Leveraging Machine Learning for Anticipatory Data Delivery in Extreme Scale In-situ Workflows

2019 ◽  
Author(s):  
Pradeep Subedi ◽  
Philip E Davis ◽  
Manish Parashar
Keyword(s):  
Machine Learning ◽  
Data Delivery ◽  
Extreme Scale

scholarly journals Global soil moisture data derived through machine learning trained with in-situ measurements

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sungmin O. ◽  
Rene Orth
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Large Scale ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

scholarly journals Snow Depth Fusion Based on Machine Learning Methods for the Northern Hemisphere

2021 ◽  
Vol 13 (7) ◽  
pp. 1250
Author(s):  
Yanxing Hu ◽  
Tao Che ◽  
Liyun Dai ◽  
Lin Xiao
Keyword(s):  
Machine Learning ◽  
Northern Hemisphere ◽  
Snow Depth ◽  
Learning Algorithm ◽  
Random Forest Regression ◽  
Machine Learning Methods ◽  
Long Time ◽  
In Situ Observations ◽  
Input Variables

In this study, a machine learning algorithm was introduced to fuse gridded snow depth datasets. The input variables of the machine learning method included geolocation (latitude and longitude), topographic data (elevation), gridded snow depth datasets and in situ observations. A total of 29,565 in situ observations were used to train and optimize the machine learning algorithm. A total of five gridded snow depth datasets—Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E) snow depth, Global Snow Monitoring for Climate Research (GlobSnow) snow depth, Long time series of daily snow depth over the Northern Hemisphere (NHSD) snow depth, ERA-Interim snow depth and Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2) snow depth—were used as input variables. The first three snow depth datasets are retrieved from passive microwave brightness temperature or assimilation with in situ observations, while the last two are snow depth datasets obtained from meteorological reanalysis data with a land surface model and data assimilation system. Then, three machine learning methods, i.e., Artificial Neural Networks (ANN), Support Vector Regression (SVR), and Random Forest Regression (RFR), were used to produce a fused snow depth dataset from 2002 to 2004. The RFR model performed best and was thus used to produce a new snow depth product from the fusion of the five snow depth datasets and auxiliary data over the Northern Hemisphere from 2002 to 2011. The fused snow-depth product was verified at five well-known snow observation sites. The R2 of Sodankylä, Old Aspen, and Reynolds Mountains East were 0.88, 0.69, and 0.63, respectively. At the Swamp Angel Study Plot and Weissfluhjoch observation sites, which have an average snow depth exceeding 200 cm, the fused snow depth did not perform well. The spatial patterns of the average snow depth were analyzed seasonally, and the average snow depths of autumn, winter, and spring were 5.7, 25.8, and 21.5 cm, respectively. In the future, random forest regression will be used to produce a long time series of a fused snow depth dataset over the Northern Hemisphere or other specific regions.

scholarly journals Evaluating Machine Learning and Geostatistical Methods for Spatial Gap-filling of Monthly ESA CCI Soil Moisture in China

2021 ◽  
Vol 13 (14) ◽  
pp. 2848
Author(s):  
Hao Sun ◽  
Qian Xu
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Water Resource Management ◽  
Large Scale ◽  
Gap Filling ◽  
Spatial Continuity ◽  
Study Results ◽  
Data Gaps

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

scholarly journals Identification of abnormal tribological regimes using a microphone and semi-supervised machine-learning algorithm

Friction ◽  
2021 ◽  
Author(s):  
Vigneashwara Pandiyan ◽  
Josef Prost ◽  
Georg Vorlaufer ◽  
Markus Varga ◽  
Kilian Wasmer
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Industrial Applications ◽  
Supervised Machine Learning ◽  
Normal Operation ◽  
Promising Alternative ◽  
Wear And Tear ◽  
Validation Procedure ◽  
And Performance

AbstractFunctional surfaces in relative contact and motion are prone to wear and tear, resulting in loss of efficiency and performance of the workpieces/machines. Wear occurs in the form of adhesion, abrasion, scuffing, galling, and scoring between contacts. However, the rate of the wear phenomenon depends primarily on the physical properties and the surrounding environment. Monitoring the integrity of surfaces by offline inspections leads to significant wasted machine time. A potential alternate option to offline inspection currently practiced in industries is the analysis of sensors signatures capable of capturing the wear state and correlating it with the wear phenomenon, followed by in situ classification using a state-of-the-art machine learning (ML) algorithm. Though this technique is better than offline inspection, it possesses inherent disadvantages for training the ML models. Ideally, supervised training of ML models requires the datasets considered for the classification to be of equal weightage to avoid biasing. The collection of such a dataset is very cumbersome and expensive in practice, as in real industrial applications, the malfunction period is minimal compared to normal operation. Furthermore, classification models would not classify new wear phenomena from the normal regime if they are unfamiliar. As a promising alternative, in this work, we propose a methodology able to differentiate the abnormal regimes, i.e., wear phenomenon regimes, from the normal regime. This is carried out by familiarizing the ML algorithms only with the distribution of the acoustic emission (AE) signals captured using a microphone related to the normal regime. As a result, the ML algorithms would be able to detect whether some overlaps exist with the learnt distributions when a new, unseen signal arrives. To achieve this goal, a generative convolutional neural network (CNN) architecture based on variational auto encoder (VAE) is built and trained. During the validation procedure of the proposed CNN architectures, we were capable of identifying acoustics signals corresponding to the normal and abnormal wear regime with an accuracy of 97% and 80%. Hence, our approach shows very promising results for in situ and real-time condition monitoring or even wear prediction in tribological applications.

Machine Learning Powered by Principal Component Descriptors as the Key for Sorted Structural Fit of XANES

2021 ◽  
Author(s):  
Andrea Martini ◽  
Alexander A. Guda ◽  
Sergey A. Guda ◽  
Aram L. Bugaev ◽  
Olga V. Safonova ◽  
...  
Keyword(s):  
Machine Learning ◽  
Structural Analysis ◽  
Synchrotron Radiation ◽  
Free Electron Laser ◽  
Principal Component ◽  
Analytical Tool ◽  
X Ray ◽  
Radiation Sources ◽  
X Ray Absorption

Modern synchrotron radiation sources and free electron laser made X-ray absorption spectroscopy (XAS) an analytical tool for the structural analysis of materials under in situ or operando conditions. Fourier approach...

scholarly journals Predicting land deformation by integrating InSAR data and cone penetration testing through machine learning techniques

2020 ◽  
Vol 382 ◽  
pp. 525-529
Author(s):  
Melika Sajadian ◽  
Ana Teixeira ◽  
Faraz S. Tehrani ◽  
Mathias Lemmens
Keyword(s):  
Machine Learning ◽  
Soil Mechanics ◽  
Machine Learning Techniques ◽  
Cone Penetration ◽  
Penetration Testing ◽  
Cone Penetration Testing ◽  
Learning Techniques ◽  
Land Deformation ◽  
Spatio Temporal

Abstract. Built environments developed on compressible soils are susceptible to land deformation. The spatio-temporal monitoring and analysis of these deformations are necessary for sustainable development of cities. Techniques such as Interferometric Synthetic Aperture Radar (InSAR) or predictions based on soil mechanics using in situ characterization, such as Cone Penetration Testing (CPT) can be used for assessing such land deformations. Despite the combined advantages of these two methods, the relationship between them has not yet been investigated. Therefore, the major objective of this study is to reconcile InSAR measurements and CPT measurements using machine learning techniques in an attempt to better predict land deformation.

Sign in / Sign up

Export Citation Format

Share Document