Impact of the numbers of observations and calibration parameters on equifinality, model performance, and output and parameter uncertainty

2015 ◽  
Vol 29 (19) ◽  
pp. 4220-4237 ◽  
Author(s):  
Younggu Her ◽  
Indrajeet Chaubey
Keyword(s):  
Parameter Uncertainty ◽  
Model Performance ◽  
Calibration Parameters

scholarly journals The effect of input data resolution and complexity on the uncertainty of hydrological predictions in a humid vegetated watershed

2018 ◽  
Vol 22 (11) ◽  
pp. 5947-5965 ◽  
Author(s):  
Linh Hoang ◽  
Rajith Mukundan ◽  
Karen E. B. Moore ◽  
Emmet M. Owens ◽  
Tammo S. Steenhuis
Keyword(s):  
Land Use ◽  
Parameter Uncertainty ◽  
Input Data ◽  
Hydrological Modeling ◽  
Assessment Tool ◽  
Model Performance ◽  
Data Complexity ◽  
Delaware River ◽  
Output Uncertainty ◽  
Data Resolution

Abstract. Uncertainty in hydrological modeling is of significant concern due to its effects on prediction and subsequent application in watershed management. Similar to other distributed hydrological models, model uncertainty is an issue in applying the Soil and Water Assessment Tool (SWAT). Previous research has shown how SWAT predictions are affected by uncertainty in parameter estimation and input data resolution. Nevertheless, little information is available on how parameter uncertainty and output uncertainty are affected by input data of varying complexity. In this study, SWAT-Hillslope (SWAT-HS), a modified version of SWAT capable of predicting saturation-excess runoff, was applied to assess the effects of input data with varying degrees of complexity on parameter uncertainty and output uncertainty. Four digital elevation model (DEM) resolutions (1, 3, 10 and 30 m) were tested for their ability to predict streamflow and saturated areas. In a second analysis, three soil maps and three land use maps were used to build nine SWAT-HS setups from simple to complex (fewer to more soil types/land use classes), which were then compared to study the effect of input data complexity on model prediction/output uncertainty. The case study was the Town Brook watershed in the upper reaches of the West Branch Delaware River in the Catskill region, New York, USA. Results show that DEM resolution did not impact parameter uncertainty or affect the simulation of streamflow at the watershed outlet but significantly affected the spatial pattern of saturated areas, with 10m being the most appropriate grid size to use for our application. The comparison of nine model setups revealed that input data complexity did not affect parameter uncertainty. Model setups using intermediate soil/land use specifications were slightly better than the ones using simple information, while the most complex setup did not show any improvement from the intermediate ones. We conclude that improving input resolution and complexity may not necessarily improve model performance or reduce parameter and output uncertainty, but using multiple temporal and spatial observations can aid in finding the appropriate parameter sets and in reducing prediction/output uncertainty.

scholarly journals Multi-basin and regional calibration based identification of distributed precipitation–runoff models for hourly runoff simulation: calibration and transfer of full and partial parameters

2015 ◽  
Vol 47 (2) ◽  
pp. 239-259 ◽  
Author(s):  
Teklu T. Hailegeorgis ◽  
Knut Alfredsen
Keyword(s):  
Performance Measures ◽  
Parameter Uncertainty ◽  
Weighted Average ◽  
Runoff Simulation ◽  
Temporal Validation ◽  
Simulation Based ◽  
And Performance ◽  
Hbv Model ◽  
Calibration Parameters ◽  
Better Than

Identification of distributed precipitation–runoff models for hourly runoff simulation based on transfer of full parameters (FP) and partial parameters (PP) are lacking for boreal mid-Norway. We evaluated storage–discharge relationships based model (Kirchmod), the Basic-Grid-Model (BGM) and a simplified Hydrologiska Byråns Vattenbalansavdelning (HBV) model for multi-basins (26 catchments). A regional calibration objective function, which uses all streamflow records in the region, was used to optimize local calibration parameters for each catchment and regional parameters yielding maximum regional weighted average (MRWA) performance measures (PM). Based on regional median Nash–Sutcliffe efficiency (NSE) and NSEln (for log-transformed series) for the calibration and validation periods, the Kirchmod model performed better than the others. Parsimony of the Kirchmod model provided less parameter uncertainty for the FP case but did not guarantee parameter identifiability. Tradeoffs between parsimony and performance were observed despite advantages of parsimony to reduce parameter correlations for the PP, which requires preliminary sensitivity analysis to identify which parameters to transfer. There are potential advantages of using the MRWA method for parameter transfer in space. However, temporal validation indicated marked deterioration of the PM. The tradeoffs between parameter transfers in space and time substantiate both spatial and temporal validation of the regional calibration methodology.

scholarly journals The effect of input data complexity on the uncertainty in simulated streamflow in a humid, mountainous watershed

2018 ◽  
Author(s):  
Linh Hoang ◽  
Rajith Mukundan ◽  
Karen E. B. Moore ◽  
Emmet M. Owens ◽  
Tammo S. Steenhuis
Keyword(s):  
New York ◽  
Land Use ◽  
Parameter Uncertainty ◽  
Input Data ◽  
Assessment Tool ◽  
Model Performance ◽  
Data Complexity ◽  
Delaware River ◽  
Output Uncertainty ◽  
Uncertainty Model

Abstract. Uncertainty in hydrological and water quality modelling is of significant concern due to its effects on prediction and subsequent application in watershed management. Similar to other distributed hydrological models, model uncertainty is an issue in applying the Soil and Water Assessment Tool (SWAT). Previous research has shown how SWAT predictions are affected by uncertainty in parameter estimation and input data resolution. Nevertheless, little information is available on how parameter uncertainty and output uncertainty are affected by input data of varying complexity. In this study, SWAT-Hillslope (SWAT-HS), a modified version of SWAT capable of predicting saturation excess runoff was applied to assess the effects of input data with varying degrees of complexity on parameter uncertainty and output uncertainty. Four digital elevation model (DEM) resolutions (1, 3, 10 and 30 m) were tested for their ability to predict streamflow and saturated areas. In a second analysis, three soil maps and three land use maps were used to build nine SWAT-HS setups from simple to complex (fewer to more soil types/ land use classes), which were then compared to study the effect of input data complexity on model prediction/output uncertainty. The case study was the Town Brook watershed in the upper reaches of the West Branch Delaware River in the Catskill Region, New York, USA. Results show that DEM resolution did not impact parameter uncertainty or affect the simulation of streamflow at the watershed outlet but significantly affected the spatial pattern of saturated areas, with 10 m being the most appropriate grid size to use for our application. The comparison of nine model setups revealed that input data complexity did not affect parameter uncertainty. Model setups using intermediate soil/land use specifications were slightly better than the ones using simple information, while the most complex setup did not show any improvement from the intermediate ones. We conclude that increasing spatial input details may not necessarily improve model performance or reduce parameter and output uncertainty, but using multiple temporal and spatial observations can aid in finding the appropriate parameter sets and in reducing prediction/output uncertainty.

Preposterior Analysis to Select Experimental Responses for Improving Identifiability in Model Uncertainty Quantification

2013 ◽  
Author(s):  
Zhen Jiang ◽  
Wei Chen ◽  
Daniel W. Apley
Keyword(s):  
Uncertainty Quantification ◽  
Parameter Uncertainty ◽  
Process Model ◽  
Monte Carlo Analysis ◽  
Simply Supported ◽  
Sources Of Uncertainty ◽  
Physical Experiments ◽  
Bayesian Monte Carlo ◽  
Engineering Modeling ◽  
Calibration Parameters

In physics-based engineering modeling and uncertainty quantification, distinguishing the effects of two main sources of uncertainty — calibration parameter uncertainty and model discrepancy — is challenging. Previous research has shown that identifiability can sometimes be improved by experimentally measuring multiple responses of the system that share a mutual dependence on a common set of calibration parameters. In this paper, we address the issue of how to select the most appropriate subset of responses to measure experimentally, to best enhance identifiability. We propose a preposterior analysis approach that, prior to conducting the physical experiments but after conducting computer simulations, can predict the degree of identifiability that will result using different subsets of responses to measure experimentally. We quantify identifiability via the posterior covariance of the calibration parameters, and predict it via the preposterior covariance from a modular Bayesian Monte Carlo analysis of a multi-response Gaussian process model. The proposed method is applied to a simply supported beam example to select two out of six responses to best improve identifiability. The estimated preposterior covariance is compared to the actual posterior covariance to demonstrate the effectiveness of the method.

Improving Identifiability in Model Calibration Using Multiple Responses

2012 ◽  
Vol 134 (10) ◽  
Author(s):  
Paul D. Arendt ◽  
Daniel W. Apley ◽  
Wei Chen ◽  
David Lamb ◽  
David Gorsich
Keyword(s):  
Parameter Uncertainty ◽  
Model Calibration ◽  
Primary Sources ◽  
Posterior Distributions ◽  
Multiple Responses ◽  
Sources Of Uncertainty ◽  
Physical Experiments ◽  
Single Response ◽  
Engineering Modeling ◽  
Calibration Parameters

In physics-based engineering modeling, the two primary sources of model uncertainty, which account for the differences between computer models and physical experiments, are parameter uncertainty and model discrepancy. Distinguishing the effects of the two sources of uncertainty can be challenging. For situations in which identifiability cannot be achieved using only a single response, we propose to improve identifiability by using multiple responses that share a mutual dependence on a common set of calibration parameters. To that end, we extend the single response modular Bayesian approach for calculating posterior distributions of the calibration parameters and the discrepancy function to multiple responses. Using an engineering example, we demonstrate that including multiple responses can improve identifiability (as measured by posterior standard deviations) by an amount that ranges from minimal to substantial, depending on the characteristics of the specific responses that are combined.

scholarly journals Multi-station calibration strategy for evaluation and sensitivity analysis of the snowmelt runoff model using MODIS satellite images

2021 ◽  
Author(s):  
Vahid Nourani ◽  
Amin Afkhaminia ◽  
Soghra Andaryani ◽  
Yongqiang Zhang
Keyword(s):  
River Flow ◽  
Model Performance ◽  
Surface Flow ◽  
Snowmelt Runoff ◽  
Single Station ◽  
Moderate Resolution Imaging Spectroradiometer ◽  
Increasing Temperature ◽  
Calibration Techniques ◽  
Runoff Model ◽  
Calibration Parameters

Abstract In this study, the snowmelt runoff model (SRM) was employed to estimate the effect of snow on the surface flow of Aji-Chay basin, northwest Iran. Two calibration techniques were adopted to enhance the calibration. The multi-station calibration (MSC) and single-station calibration (SSC) strategies applied to investigate their effects on the modeling accuracy. The runoff coefficients (cs and cr) were selected as calibration parameters because of their uncertainty in such an extended basin. To determine the most substantial input of the model which is the snow-covered area (SCA) from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor imagery, MOD10A2 images were collected with spatial and temporal resolutions of 500 meters and 8 days, respectively. The results show an average of 15% improvement in the model performance in the MSC strategy from the data period of 2008–2012. Also, an appropriate agreement with physical characteristics of the study area could be seen for the calibration parameters. The contribution of snowmelt in the river flow reaches its peak in April and May, then with increasing temperature, the contribution decreased gradually. Furthermore, analysis of parameters indicates that the SRM is sensitive to recession coefficient and runoff coefficients.

scholarly journals Improving Weather Predictability by Including Land Surface Model Parameter Uncertainty

2016 ◽  
Vol 144 (4) ◽  
pp. 1551-1569 ◽  
Author(s):  
Rene Orth ◽  
Emanuel Dutra ◽  
Florian Pappenberger
Keyword(s):  
Soil Moisture ◽  
Parameter Uncertainty ◽  
Land Surface ◽  
Soil Depth ◽  
Model Performance ◽  
Land Surface Model ◽  
Stomatal Resistance ◽  
Forecast Skill ◽  
Surface Model ◽  
Earth System

Abstract The land surface forms an important component of Earth system models and interacts nonlinearly with other parts such as ocean and atmosphere. To capture the complex and heterogeneous hydrology of the land surface, land surface models include a large number of parameters impacting the coupling to other components of the Earth system model. Focusing on ECMWF’s land surface model Hydrology Tiled ECMWF Scheme of Surface Exchanges over Land (HTESSEL), the authors present in this study a comprehensive parameter sensitivity evaluation using multiple observational datasets in Europe. The authors select six poorly constrained effective parameters (surface runoff effective depth, skin conductivity, minimum stomatal resistance, maximum interception, soil moisture stress function shape, and total soil depth) and explore their sensitivity to model outputs such as soil moisture, evapotranspiration, and runoff using uncoupled simulations and coupled seasonal forecasts. Additionally, the authors investigate the possibility to construct ensembles from the multiple land surface parameters. In the uncoupled runs the authors find that minimum stomatal resistance and total soil depth have the most influence on model performance. Forecast skill scores are moreover sensitive to the same parameters as HTESSEL performance in the uncoupled analysis. The authors demonstrate the robustness of these findings by comparing multiple best-performing parameter sets and multiple randomly chosen parameter sets. The authors find better temperature and precipitation forecast skill with the best-performing parameter perturbations demonstrating representativeness of model performance across uncoupled (and hence less computationally demanding) and coupled settings. Finally, the authors construct ensemble forecasts from ensemble members derived with different best-performing parameterizations of HTESSEL. This incorporation of parameter uncertainty in the ensemble generation yields an increase in forecast skill, even beyond the skill of the default system.

scholarly journals Exploring the parameter space of the COSMO-CLM v5.0 regional climate model for the Central Asia CORDEX domain

2020 ◽  
Vol 13 (11) ◽  
pp. 5779-5797
Author(s):  
Emmanuele Russo ◽  
Silje Lund Sørland ◽  
Ingo Kirchner ◽  
Martijn Schaap ◽  
Christoph C. Raible ◽  
...  
Keyword(s):  
Central Asia ◽  
Parameter Space ◽  
Parameter Uncertainty ◽  
Climate Model ◽  
Regional Climate ◽  
Model Performance ◽  
Small Scale ◽  
Model Parameters ◽  
Study Results ◽  
Parameter Values

Abstract. The parameter uncertainty of a climate model represents the spectrum of the results obtained by perturbing its empirical and unconfined parameters used to represent subgrid-scale processes. In order to assess a model's reliability and to better understand its limitations and sensitivity to different physical processes, the spread of model parameters needs to be carefully investigated. This is particularly true for regional climate models (RCMs), whose performance is domain dependent. In this study, the parameter space of the Consortium for Small-scale Modeling CLimate Mode (COSMO-CLM) RCM is investigated for the Central Asia Coordinated Regional Climate Downscaling Experiment (CORDEX) domain, using a perturbed physics ensemble (PPE) obtained by performing 1-year simulations with different parameter values. The main goal is to characterize the parameter uncertainty of the model and to determine the most sensitive parameters for the region. Moreover, the presented experiments are used to study the effect of several parameters on the simulation of selected variables for subregions characterized by different climate conditions, assessing by which degree it is possible to improve model performance by properly selecting parameter inputs in each case. Finally, the paper explores the model parameter sensitivity over different domains, tackling the question of transferability of an RCM model setup to different regions of study. Results show that only a subset of model parameters present relevant changes in model performance for different parameter values. Importantly, for almost all parameter inputs, the model shows an opposite behaviour among different clusters and regions. This indicates that conducting a calibration of the model against observations to determine optimal parameter values for the Central Asia domain is particularly challenging: in this case, the use of objective calibration methods is highly necessary. Finally, the sensitivity of the model to parameter perturbation for Central Asia is different than the one observed for Europe, suggesting that an RCM should be retuned, and its parameter uncertainty properly investigated, when setting up model experiments for different domains of study.

Predictors of Purchasing a Hearing Aid After an Evaluation Period: A Prospective Study in Dutch Older Hearing Aid Candidates

2019 ◽  
Vol 28 (3S) ◽  
pp. 802-805 ◽  
Author(s):  
Marieke Pronk ◽  
Janine F. J. Meijerink ◽  
Sophia E. Kramer ◽  
Martijn W. Heymans ◽  
Jana Besser
Keyword(s):  
Help Seeking ◽  
Hearing Aid ◽  
Model Performance ◽  
Area Under The Curve ◽  
Secondary Data ◽  
Support Program ◽  
Evaluation Period ◽  
A Prospective Study ◽  
Study Participants ◽  
Baseline Covariates

Purpose The current study aimed to identify factors that distinguish between older (50+ years) hearing aid (HA) candidates who do and do not purchase HAs after having gone through an HA evaluation period (HAEP). Method Secondary data analysis of the SUpport PRogram trial was performed ( n = 267 older, 1st-time HA candidates). All SUpport PRogram participants started an HAEP shortly after study enrollment. Decision to purchase an HA by the end of the HAEP was the outcome of interest of the current study. Participants' baseline covariates (22 in total) were included as candidate predictors. Multivariable logistic regression modeling (backward selection and reclassification tables) was used. Results Of all candidate predictors, only pure-tone average (average of 1, 2, and 4 kHz) hearing loss emerged as a significant predictor (odds ratio = 1.03, 95% confidence interval [1.03, 1.17]). Model performance was weak (Nagelkerke R 2 = .04, area under the curve = 0.61). Conclusions These data suggest that, once HA candidates have decided to enter an HAEP, factors measured early in the help-seeking journey do not predict well who will and will not purchase an HA. Instead, factors that act during the HAEP may hold this predictive value. This should be examined.

Constructing and Deconstructing Concepts

2016 ◽  
Vol 63 (5) ◽  
pp. 249-262 ◽  
Author(s):  
Charles A. Doan ◽  
Ronaldo Vigo
Keyword(s):  
Information Theory ◽  
Model Performance ◽  
Unsupervised Classification ◽  
Family Resemblance ◽  
Category Membership ◽  
Formal Models ◽  
Human Beings ◽  
One Dimensional ◽  
Relational Patterns ◽  
Optimal Classification

Abstract. Several empirical investigations have explored whether observers prefer to sort sets of multidimensional stimuli into groups by employing one-dimensional or family-resemblance strategies. Although one-dimensional sorting strategies have been the prevalent finding for these unsupervised classification paradigms, several researchers have provided evidence that the choice of strategy may depend on the particular demands of the task. To account for this disparity, we propose that observers extract relational patterns from stimulus sets that facilitate the development of optimal classification strategies for relegating category membership. We conducted a novel constrained categorization experiment to empirically test this hypothesis by instructing participants to either add or remove objects from presented categorical stimuli. We employed generalized representational information theory (GRIT; Vigo, 2011b , 2013a , 2014 ) and its associated formal models to predict and explain how human beings chose to modify these categorical stimuli. Additionally, we compared model performance to predictions made by a leading prototypicality measure in the literature.

Sign in / Sign up

Export Citation Format

Share Document