Global Automated Quality Control of In Situ Soil Moisture Data from the International Soil Moisture Network

2013 ◽  
Vol 12 (3) ◽  
pp. vzj2012.0097 ◽  
Author(s):  
W.A. Dorigo ◽  
A. Xaver ◽  
M. Vreugdenhil ◽  
A. Gruber ◽  
A. Hegyiová ◽  
...  
Keyword(s):  
Quality Control ◽  
Soil Moisture ◽  
Soil Moisture Data ◽  
Automated Quality Control

scholarly journals Automated Quality Control of In Situ Soil Moisture from the North American Soil Moisture Database Using NLDAS-2 Products

2015 ◽  
Vol 54 (6) ◽  
pp. 1267-1282 ◽  
Author(s):  
Youlong Xia ◽  
Trent W. Ford ◽  
Yihua Wu ◽  
Steven M. Quiring ◽  
Michael B. Ek
Keyword(s):  
Quality Control ◽  
Soil Moisture ◽  
North American ◽  
Soil Layer ◽  
Control Procedure ◽  
Multiple Model ◽  
Quality Control Procedure ◽  
The North ◽  
Automated Quality Control

AbstractThe North American Soil Moisture Database (NASMD) was initiated in 2011 to provide support for developing climate forecasting tools, calibrating land surface models, and validating satellite-derived soil moisture algorithms. The NASMD has collected data from over 30 soil moisture observation networks providing millions of in situ soil moisture observations in all 50 states, as well as Canada and Mexico. It is recognized that the quality of measured soil moisture in NASMD is highly variable because of the diversity of climatological conditions, land cover, soil texture, and topographies of the stations, and differences in measurement devices (e.g., sensors) and installation. It is also recognized that error, inaccuracy, and imprecision in the data can have significant impacts on practical operations and scientific studies. Therefore, developing an appropriate quality control procedure is essential to ensure that the data are of the best quality. In this study, an automated quality control approach is developed using the North American Land Data Assimilation System, phase 2 (NLDAS-2), Noah soil porosity, soil temperature, and fraction of liquid and total soil moisture to flag erroneous and/or spurious measurements. Overall results show that this approach is able to flag unreasonable values when the soil is partially frozen. A validation example using NLDAS-2 multiple model soil moisture products at the 20-cm soil layer showed that the quality control procedure had a significant positive impact in Alabama, North Carolina, and west Texas. It had a greater impact in colder regions, particularly during spring and autumn. Over 433 NASMD stations have been quality controlled using the methodology proposed in this study, and the algorithm will be implemented to control data quality from the other ~1200 NASMD stations in the near future.

The International Soil Moisture Network: an open-source data hosting facility in support of hydrological research

2020 ◽  
Author(s):  
Irene Himmelbauer ◽  
Daniel Aberer ◽  
Lukas Schremmer ◽  
Ivana Petrakovic ◽  
Luca Zappa ◽  
...  
Keyword(s):  
Quality Control ◽  
Soil Moisture ◽  
Open Source ◽  
Measurement Techniques ◽  
European Space Agency ◽  
Web Portal ◽  
Soil Moisture Data ◽  
In Situ Data ◽  
Spatial Coverage

<p><span>The International Soil Moisture Network (ISMN, </span><span></span><span>) is an international cooperation to establish and maintain an open-source global data hosting facility, providing in-situ soil moisture data as well as accompanying soil variables. This database is an essential means for validating and improving global satellite soil moisture products as well as land surface -, climate- , and hydrological models.</span></p><p><span>For hydrological validation, the quality of used in-situ data is essential. The various independent local and regional in situ networks often do not follow standardized measurement techniques or protocols, collect their data in different units, at different depths and at various sampling rates. Besides, quality control is rarely applied and accessing the data is often not easy or feasible.</span></p><p><span>The ISMN was created to address the above-mentioned issues. Within the ISMN, in situ soil moisture measurements (surface and sub-surface) are collected, harmonized in terms of units and sampling rates, advanced quality control is applied and the data is then stored in a database and made available online, where users can download it for free. </span></p><p><span>Since its establishment in 2009 and with continuous financial support through the European Space Agency (ESA), the ISMN evolved into a widely used in situ data source growing continuously (in terms of data volume and users). Historic measurements starting in 1952 up to near–real time are available through the ISMN web portal. Currently, the ISMN consists of 60 networks with more than 2500 stations spread all over the globe. With a </span><span><span>steadily growing user community more than 3200 registered users strong</span></span><span> the value of the ISMN as a well-established and rich source of in situ soil moisture observations is well recognized. In fact, the ISMN is widely used in variety of scientific fields (e.g. climate, water, agriculture, disasters, ecosystems, weather, biodiversity, etc.). </span></p><p> <span>Our partner networks range from networks with a handful of stations to networks that are composed of over 400 sites, are supported with half yearly provider reports on statistical data about their network (e.g.: data download statistic, flagging statistic, etc.). </span></p><p><span>About 10’000 datasets are available through the web portal. However, the spatial coverage of in situ observations still needs to be improved. For example, in Africa and South America only sparse data are available. Innovative ideas, such as the inclusion of soil moisture data from low cost sensors (GROW observatory ) collected by citizen scientists, holds the potential of closing this gap, thus providing new information and knowledge.</span></p><p><span>In this session , we want to give an overview of the ISMN, its unique features and its support of data provider, who are willing to openly share their data, as well as hydrological researcher in need of freely available datasets.</span></p>

The International Soil Moisture Network in assistance of EO soil moisture validation products, services and models

2020 ◽  
Author(s):  
Daniel Aberer ◽  
Irene Himmelbauer ◽  
Lukas Schremmer ◽  
Ivana Petrakovic ◽  
Wouter Dorigo ◽  
...  
Keyword(s):  
Quality Control ◽  
Soil Moisture ◽  
Real Time ◽  
Land Surface ◽  
Measurement Techniques ◽  
European Space Agency ◽  
Web Portal ◽  
Soil Moisture Data ◽  
Spatial Coverage

<p>The International Soil Moisture Network (ISMN, https://ismn.geo.tuwien.ac.at/) is an international cooperation to establish and maintain a unique centralized global data hosting facility, making in situ soil moisture data easily and freely accessible. This database is an essential means for validating and improving global satellite soil moisture products, land surface -, climate- , and hydrological models. </p><p>In situ measurements are crucial to calibrate and validate satellite soil moisture products. For a meaningful comparison with remotely sensed data and reliable validation results, the quality of the reference data is essential. The various independent local and regional in situ networks often do not follow standardized measurement techniques or protocols, collecting their data in different units, at different depths and at various sampling rates. Besides, quality control is rarely applied and accessing the data is often not easy or feasible.</p><p>The ISMN has been created to address the above-mentioned issues and is building a stable base to assist EO products, services and models. Within the ISMN, in situ soil moisture measurements (surface and sub-surface) are collected, harmonized in terms of units and sampling rates, advanced quality control is applied and the data is then stored in a database and made available online, where users can download it for free.</p><p>Founded in 2009, the ISMN has grown to a widely used in situ data source including 61 networks with more than 2600 stations distributed on a global scale and a steadily growing user community > 3200 registered users strong. Time series with hourly timestamps from 1952 – up to near real time are stored in the database and are available through the ISMN web portal, including daily near-real time updates from 6 networks (> 900 stations). With continuous financial support through the European Space Agency (formerly SMOS and IDEAS+ programs, currently QA4EO program), the ISMN evolved into a platform of benchmark data for several operational services such as ESA CCI Soil Moisture, the Copernicus Climate Change (C3S), the Copernicus Global Land Service (CGLS) and the online validation service Quality Assurance for Soil Moisture (QA4SM). In general, ISMN data is widely used in a variety of scientific fields (e.g. climate, water, agriculture, disasters, ecosystems, weather, biodiversity, etc.).</p><p>About 10’000 datasets are available through the web portal. However, the spatial coverage of in situ observations still needs to be improved. For example, in Africa and South America only sparse data are available. Innovative ideas, such as the inclusion of soil moisture data from low cost sensors (eventually) collected by citizen scientists, holds the potential of closing this gap, thus providing new information and knowledge.</p><p>In this session, we give an overview of the ISMN, its unique features and its benefits for validating satellite soil moisture products.</p>

scholarly journals Global soil moisture data derived through machine learning trained with in-situ measurements

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sungmin O. ◽  
Rene Orth
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Large Scale ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

Exploring trends for the H SAF ASCAT root-zone soil moisture data records

2021 ◽  
Author(s):  
David Fairbairn ◽  
Patricia de Rosnay ◽  
Peter Weston
Keyword(s):  
Soil Moisture ◽  
Root Zone ◽  
Weather Prediction ◽  
Climate Change Impacts ◽  
Soil Moisture Data ◽  
Negative Root ◽  
In Situ Observations ◽  
Prediction Systems ◽  
Water Scarce

<p>Environmental (e.g. floods, droughts) and weather prediction systems rely on an accurate representation of soil moisture (SM). The EUMETSAT H SAF aims to provide high quality satellite-based hydrological products, including SM.<br>ECMWF is producing ASCAT root zone SM for H SAF. The production relies on an Extended Kalman filter to retrieve root zone SM from surface SM satellite data. A 10 km sampling reanalysis product (1992-2020) forced by ERA5 atmospheric fields (H141/H142) is produced for H SAF, which assimilates ERS/SCAT (1992-2006) and ASCAT-A/B/C (2007-2020) derived surface SM. The root-zone SM performance is validated using sparse in situ observations globally and generally demonstrates a positive and consistent correlation over the period. A negative trend in root-zone SM is found during summer and autumn months over much of Europe during the period (1992-2020). This is consistent with expected climate change impacts and is particularly alarming over the water-scarce Mediterranean region. The recent hot and dry summer of 2019 and dry spring of 2020 are well captured by negative root-zone SM anomalies. Plans for the future H SAF data record products will be presented, including the assimilation of high-resolution EPS-SCA-derived soil moisture data.</p>

scholarly journals An Evaluation of the EnKF vs. EnOI and the Assimilation of SMAP, SMOS and ESA CCI Soil Moisture Data over the Contiguous US

2019 ◽  
Vol 11 (5) ◽  
pp. 478 ◽  
Author(s):  
Jostein Blyverket ◽  
Paul Hamer ◽  
Laurent Bertino ◽  
Clément Albergel ◽  
David Fairbairn ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Surface Soil ◽  
Open Loop ◽  
Surface Zone ◽  
Surface Soil Moisture ◽  
Atmospheric Forcing ◽  
Significance Level ◽  
Soil Moisture Data ◽  
In Situ Observations

A number of studies have shown that assimilation of satellite derived soil moisture using the ensemble Kalman Filter (EnKF) can improve soil moisture estimates, particularly for the surface zone. However, the EnKF is computationally expensive since an ensemble of model integrations have to be propagated forward in time. Here, assimilating satellite soil moisture data from the Soil Moisture Active Passive (SMAP) mission, we compare the EnKF with the computationally cheaper ensemble Optimal Interpolation (EnOI) method over the contiguous United States (CONUS). The background error–covariance in the EnOI is sampled in two ways: (i) by using the stochastic spread from an ensemble open-loop run, and (ii) sampling from the model spinup climatology. Our results indicate that the EnKF is only marginally superior to one version of the EnOI. Furthermore, the assimilation of SMAP data using the EnKF and EnOI is found to improve the surface zone correlation with in situ observations at a 95 % significance level. The EnKF assimilation of SMAP data is also found to improve root-zone correlation with independent in situ data at the same significance level; however this improvement is dependent on which in situ network we are validating against. We evaluate how the quality of the atmospheric forcing affects the analysis results by prescribing the land surface data assimilation system with either observation corrected or model derived precipitation. Surface zone correlation skill increases for the analysis using both the corrected and model derived precipitation, but only the latter shows an improvement at the 95 % significance level. The study also suggests that assimilation of satellite derived surface soil moisture using the EnOI can correct random errors in the atmospheric forcing and give an analysed surface soil moisture close to that of an open-loop run using observation derived precipitation. Importantly, this shows that estimates of soil moisture could be improved using a combination of assimilating SMAP using the computationally cheap EnOI while using model derived precipitation as forcing. Finally, we assimilate three different Level-2 satellite derived soil moisture products from the European Space Agency Climate Change Initiative (ESA CCI), SMAP and SMOS (Soil Moisture and Ocean Salinity) using the EnOI, and then compare the relative performance of the three resulting analyses against in situ soil moisture observations. In this comparison, we find that all three analyses offer improvements over an open-loop run when comparing to in situ observations. The assimilation of SMAP data is found to perform marginally better than the assimilation of SMOS data, while assimilation of the ESA CCI data shows the smallest improvement of the three analysis products.

Estimation of Soil Moisture Content Using Deep Learning and High-Resolution Satellite Imagery (Sentinel-1 and 2)

2020 ◽  
Author(s):  
Soo-Jin Lee ◽  
Yang-Won Lee
Keyword(s):  
Soil Moisture ◽  
Deep Learning ◽  
High Resolution ◽  
Forest Fires ◽  
Optimal Solution ◽  
Local Scale ◽  
Soil Moisture Data ◽  
Local Optimal Solution ◽  
Salinity Data

<p>Soil moisture is an important factor affecting global circulation (climate, carbon, and water), disasters (drought, floods, and forest fires), and crop growth, so the production of soil moisture data is important. Currently, satellite-based soil moisture data is available from NASA’ SMAP (Soil Moisture Active Passive) and ESA’ SMOS (Soil Moisture and Ocean Salinity) data. Since these data are based on passive microwave sensor, they have low spatial resolution. Therefore, it is difficult to observe the distribution of soil moisture on a local scale. The purpose of this study is to produce high resolution soil moisture for monitoring on a local scale. For this purpose, we performed soil moisture modeling using high resolution satellite data (Sentinel-1 SAR (synthetic-aperture radar), Sentinel-2 MSI (multispectral instrument)) and deep learning. Deep learning is a method improving the problems of traditional neural networks such as overfitting, gradient vanishing, and local optimal solution through development of learning methods such as dropout, ReLU (Rectified Linear Unit), and so on. Recently, it has been used for estimation of surface hydrologic factors (soil moisture, evapotranspiration, etc.). The study area is an agricultural area located in Manitoba and Saskatoon, Canada. In-situ soil moisture data was constructed from RISMA (Real-Time In-Situ Soil Monitoring for Agriculture). In order to develop an optimal soil moisture model, various condition experiments on hyper-parameters affecting the performance of model were carried out and their performance was evaluated.</p>

Simulating Gross Primary Productivity of vegetation under soil water stress using in-situ and reanalysis soil moisture data inputs

2020 ◽  
Author(s):  
Timothy Lam ◽  
Amos P. K. Tai
Keyword(s):  
Soil Moisture ◽  
Water Stress ◽  
Soil Water ◽  
Primary Productivity ◽  
Gross Primary Productivity ◽  
Efficiency Coefficient ◽  
Community Land Model ◽  
Soil Water Stress ◽  
Soil Moisture Data

<p>This study utilises in-situ and reanalysis soil moisture data inputs from various sources to evaluate the effect of soil water stress on Gross Primary Productivity (GPP) of different Plant Functional Types (PFTs) using Terrestrial Ecosystem Model in R (TEMIR), which is under development by Tai Group of Atmosphere-Biosphere Interactions (Tai et al. in prep.). An empirical soil water stress function with reference to Community Land Model (CLM) Version 4.5 is employed to quantify water stress experienced by vegetation which hinders stomatal conductance and thus carboxylation rate. The model results are compared against observations at FLUXNET sites in semi-arid regions across the globe at daily timescale where in-situ GPP data is available and water stress inhibits plant functions to some extent. By dividing the soil into two layers (topsoil and root zone), GPP simulation improves significantly comparing with using single layer bulk soil (Modified Nash-Sutcliffe Model Efficiency Coefficient N increases from -0.686 to -0.586). Such upgrade is particularly substantial for vegetation with shallow roots such as grass PFTs. Despite this improvement, the model is characterised by an overall overestimation of GPP when water stress occurs, and inconsistency of accuracy subject to PFTs and degree of water stress experienced. This study informs responses of various PFTs to soil water stress, capacity of TEMIR in simulating the responses, and possible drawbacks of empirical soil water stress functions, and highlights the importance of topsoil moisture data input for vegetation drought monitoring.</p><p>Keywords: Soil water stress, Terrestrial model representation, Photosynthesis, In-situ data, Reanalysis data, FLUXNET</p>

ASCAT soil wetness index validation through in situ and modeled soil moisture data in central Italy

2010 ◽  
Vol 114 (11) ◽  
pp. 2745-2755 ◽  
Author(s):  
L. Brocca ◽  
F. Melone ◽  
T. Moramarco ◽  
W. Wagner ◽  
S. Hasenauer
Keyword(s):  
Soil Moisture ◽  
Central Italy ◽  
Wetness Index ◽  
Soil Moisture Data ◽  
Soil Wetness

scholarly journals Comparison of Surface Soil Moisture from SMOS Satellite and Ground Measurements

2014 ◽  
Vol 28 (3) ◽  
pp. 359-369 ◽  
Author(s):  
Bogusław Usowicz ◽  
Wojciech Marczewski ◽  
Jerzy B. Usowicz ◽  
Mateusz I. Lukowski ◽  
Jerzy Lipiec
Keyword(s):  
Land Use ◽  
Soil Moisture ◽  
Soil Cover ◽  
Natural Vegetation ◽  
Sustainable Land Use ◽  
Concordance Correlation ◽  
Marsh Soil ◽  
Soil Moisture Data ◽  
Ocean Salinity

Abstract Soil moisture datasets at various scales are needed for sustainable land use and water management. The aim of this study was to compare soil moisture ocean salinity satellite and in situ soil moisture data for the Podlasie and Polesie regions in Eastern Poland. Both regions have similar climatic and topographic conditions but are different in land use, vegetation, and soil cover. The test sites were located on agricultural fields on sandy soils and natural vegetation on marshy soils that prevail in the Podlasie and Polesie regions, respectively. The soil moisture ocean salinity soil moisture data were obtained from radiometric measurements (1.4 GHz) and the ground soil moisture from sensors at a depth of 5 cm during the years 2010-2011. In general, temporal patterns of soil moisture from both satellite and ground measurements followed the rainfall trend. The regression coefficients, Bland-Altman analysis, concordance correlation coefficient, and total deviation index showed that the agreement between ground and soil moisture ocean salinity derived soil moisture data is better for the Podlasie than the Polesie region. The lower agreement in Polesie was attributed mostly to the presence of the widespread natural vegetation on the wetter marsh soil along with minor contribution of agriculturally used drier coarse-textured soils.

Sign in / Sign up

Export Citation Format

Share Document