scholarly journals The STONE Curve: A ROC‐Derived Model Performance Assessment Tool

2020 ◽  
Vol 7 (8) ◽  
Author(s):  
Michael W. Liemohn ◽  
Abigail R. Azari ◽  
Natalia Y. Ganushkina ◽  
Lutz Rastätter
Keyword(s):  
Performance Assessment ◽  
Assessment Tool ◽  
Model Performance

scholarly journals The STONE curve: A ROC-derived model performance assessment tool

2020 ◽  
Author(s):  
Michael W. Liemohn ◽  
Abigail R. Azari ◽  
Natalia Yu Ganushkina ◽  
Lutz Rastatter
Keyword(s):  
Performance Assessment ◽  
Assessment Tool ◽  
Model Performance

scholarly journals The STONE curve: A ROC-derived model performance assessment tool

2020 ◽  
Author(s):  
Michael W. Liemohn ◽  
Abigail R. Azari ◽  
Natalia Yu Ganushkina ◽  
Lutz Rastatter
Keyword(s):  
Performance Assessment ◽  
Assessment Tool ◽  
Model Performance

Indicators for Measuring Performance of Financial Cooperatives in Nepal

2017 ◽  
Vol 2 (1-2) ◽  
pp. 66-86
Author(s):  
Nav Raj Simkhada
Keyword(s):  
Focus Group ◽  
Performance Measurement ◽  
Performance Assessment ◽  
Financial Institutions ◽  
Assessment Tool ◽  
Expert Interviews ◽  
Focus Group Discussions ◽  
Group Discussions ◽  
Institutional Assessment ◽  
Financial Cooperatives

Comprehensive institutional assessment tool helps to appraise performance of an organization and adopt appropriate strategies for enhancing performance. Different organizations demand different indicators and standards for appraising performance. Different tools such as PEARLS and CAMEL have been prescribed measure performance of financial institutions. These tools were developed in different contexts and are not adopted in Nepali cooperative sector. The objective of this paper is to identify and recommend different indicators for measuring performance of financial cooperatives in Nepal. Expert interviews and focus group discussions were applied to explore the indicators for performance assessment. The identified indicators were piloted with randomly selected 210 cooperatives. The findings showed that 32 financial ratios under eight performance measurement dimensions and 25 self-governance related indicators are needed to assess the performance of financial cooperatives in Nepal and elsewhere. Implications of the findings are discussed and limitations of the study are highlighted,

scholarly journals Inconsistent strategies to spin up models in CMIP5: implications for ocean biogeochemical model performance assessment

2016 ◽  
Vol 9 (5) ◽  
pp. 1827-1851 ◽  
Author(s):  
Roland Séférian ◽  
Marion Gehlen ◽  
Laurent Bopp ◽  
Laure Resplandy ◽  
James C. Orr ◽  
...  
Keyword(s):  
Performance Assessment ◽  
Initial Conditions ◽  
Model Performance ◽  
Cmip5 Models ◽  
Biogeochemical Model ◽  
Earth System ◽  
Model Intercomparison ◽  
System Models ◽  
Marine Biogeochemistry ◽  
Earth System Models

Abstract. During the fifth phase of the Coupled Model Intercomparison Project (CMIP5) substantial efforts were made to systematically assess the skill of Earth system models. One goal was to check how realistically representative marine biogeochemical tracer distributions could be reproduced by models. In routine assessments model historical hindcasts were compared with available modern biogeochemical observations. However, these assessments considered neither how close modeled biogeochemical reservoirs were to equilibrium nor the sensitivity of model performance to initial conditions or to the spin-up protocols. Here, we explore how the large diversity in spin-up protocols used for marine biogeochemistry in CMIP5 Earth system models (ESMs) contributes to model-to-model differences in the simulated fields. We take advantage of a 500-year spin-up simulation of IPSL-CM5A-LR to quantify the influence of the spin-up protocol on model ability to reproduce relevant data fields. Amplification of biases in selected biogeochemical fields (O2, NO3, Alk-DIC) is assessed as a function of spin-up duration. We demonstrate that a relationship between spin-up duration and assessment metrics emerges from our model results and holds when confronted with a larger ensemble of CMIP5 models. This shows that drift has implications for performance assessment in addition to possibly aliasing estimates of climate change impact. Our study suggests that differences in spin-up protocols could explain a substantial part of model disparities, constituting a source of model-to-model uncertainty. This requires more attention in future model intercomparison exercises in order to provide quantitatively more correct ESM results on marine biogeochemistry and carbon cycle feedbacks.

scholarly journals The effect of input data resolution and complexity on the uncertainty of hydrological predictions in a humid vegetated watershed

2018 ◽  
Vol 22 (11) ◽  
pp. 5947-5965 ◽  
Author(s):  
Linh Hoang ◽  
Rajith Mukundan ◽  
Karen E. B. Moore ◽  
Emmet M. Owens ◽  
Tammo S. Steenhuis
Keyword(s):  
Land Use ◽  
Parameter Uncertainty ◽  
Input Data ◽  
Hydrological Modeling ◽  
Assessment Tool ◽  
Model Performance ◽  
Data Complexity ◽  
Delaware River ◽  
Output Uncertainty ◽  
Data Resolution

Abstract. Uncertainty in hydrological modeling is of significant concern due to its effects on prediction and subsequent application in watershed management. Similar to other distributed hydrological models, model uncertainty is an issue in applying the Soil and Water Assessment Tool (SWAT). Previous research has shown how SWAT predictions are affected by uncertainty in parameter estimation and input data resolution. Nevertheless, little information is available on how parameter uncertainty and output uncertainty are affected by input data of varying complexity. In this study, SWAT-Hillslope (SWAT-HS), a modified version of SWAT capable of predicting saturation-excess runoff, was applied to assess the effects of input data with varying degrees of complexity on parameter uncertainty and output uncertainty. Four digital elevation model (DEM) resolutions (1, 3, 10 and 30 m) were tested for their ability to predict streamflow and saturated areas. In a second analysis, three soil maps and three land use maps were used to build nine SWAT-HS setups from simple to complex (fewer to more soil types/land use classes), which were then compared to study the effect of input data complexity on model prediction/output uncertainty. The case study was the Town Brook watershed in the upper reaches of the West Branch Delaware River in the Catskill region, New York, USA. Results show that DEM resolution did not impact parameter uncertainty or affect the simulation of streamflow at the watershed outlet but significantly affected the spatial pattern of saturated areas, with 10m being the most appropriate grid size to use for our application. The comparison of nine model setups revealed that input data complexity did not affect parameter uncertainty. Model setups using intermediate soil/land use specifications were slightly better than the ones using simple information, while the most complex setup did not show any improvement from the intermediate ones. We conclude that improving input resolution and complexity may not necessarily improve model performance or reduce parameter and output uncertainty, but using multiple temporal and spatial observations can aid in finding the appropriate parameter sets and in reducing prediction/output uncertainty.

scholarly journals Assessing the Uncertainties of Four Precipitation Products for Swat Modeling in Mekong River Basin

2019 ◽  
Vol 11 (3) ◽  
pp. 304 ◽  
Author(s):  
Xiongpeng Tang ◽  
Jianyun Zhang ◽  
Chao Gao ◽  
Gebdang Ruben ◽  
Guoqing Wang
Keyword(s):  
River Basin ◽  
Assessment Tool ◽  
Model Performance ◽  
Mekong River ◽  
Distributed Hydrological Model ◽  
Parameter Uncertainties ◽  
Mekong River Basin ◽  
Daily Streamflow ◽  
Precipitation Estimates ◽  
Better Than

Using hydrological simulation to evaluate the accuracy of satellite-based and reanalysis precipitation products always suffer from a large uncertainty. This study evaluates four widely used global precipitation products with high spatial and temporal resolutions [i.e., AgMERRA (AgMIP modern-Era Retrospective Analysis for Research and Applications), MSWEP (Multi-Source Weighted-Ensemble Precipitation), PERSIANN-CDR (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Climate Data Record), and TMPA (Tropical Rainfall Measuring Mission 3B42 Version7)] against gauge observations with six statistical metrics over Mekong River Basin (MRB). Furthermore, the Soil and Water Assessment Tool (SWAT), a widely used semi-distributed hydrological model, is calibrated using different precipitation inputs. Both model performance and uncertainties of parameters and prediction have been quantified. The following findings were obtained: (1) The MSWEP and TMPA precipitation products have good accuracy with higher CC, POD, and lower ME and RMSE, and the AgMERRA precipitation estimates perform better than PERSIANN-CDR in this rank; and (2) out of the six different climate regions of MRB, all six metrics are worse than that in the whole MRB. The AgMERRA can better reproduce the occurrence and contributions at different precipitation densities, and the MSWEP has the best performance in Cwb, Cwa, Aw, and Am regions that belong to the low latitudes. (3) Daily streamflow predictions obtained using MSWEP precipitation estimates are better than those simulated by other three products in term of both the model performance and parameter uncertainties; and (4) although MSWEP better captures the precipitation at different intensities in different climatic regions, the performance can still be improved, especially in the regions with higher altitude.

scholarly journals Evaluation of a new clinical performance assessment tool : A reliability study

2012 ◽  
Vol 68 (3) ◽  
Author(s):  
C. Joseph ◽  
J. Frantz ◽  
C. Hendricks ◽  
M. Smith
Keyword(s):  
Clinical Practice ◽  
Performance Assessment ◽  
Internal Consistency ◽  
Quantitative Research ◽  
Assessment Tool ◽  
Clinical Performance ◽  
Intraclass Correlation ◽  
Cross Sectional ◽  
Clinical Educators ◽  
Clinical Performance Assessment

Clinical practice is an essential requirement of any graduatephysiotherapy programme. For this purpose, valid and reliable assessment toolsare paramount for the measurement of key competencies in the real-worldsetting. This study aims to determine the internal consistency and inter-raterreliability of a newly developed and validated clinical performance assessmentform. A cross-sectional quantitative research design was used, which includedpaired evaluations of 32 (17 treatment and 15 assessment) student examinationsperformed by two independent clinical educators. Chronbachs alpha was computedto assess internal consistency and intraclass correlation coefficient (ICC’s) withconfidence intervals of 95% were computed to determine the percentage agreement between paired examiners. Thedegree of internal consistency was substantial for all key performance areas of both examinations, except for timeand organisational management (0.21) and professionalism (0.42) in the treatment and evaluation examinationsrespectively. The overall internal consistency was 0.89 and 0.73 for both treatment and assessment examinations,indicating substantial agreement. With regard to agreement between raters, the ICC’s for the overall marks were0.90 and 0.97 for both treatment and assessment examinations. Clinical educators demonstrated a high level ofreliability in the assessment of students’ competence using the newly developed clinical performance assessment form.These findings greatly underscore the reliability of results obtained through observation of student examinations, andadd another tool to the basket of ensuring quality assurance in physiotherapy clinical practice assessment.

Sign in / Sign up

Export Citation Format

Share Document