Decision Making in the Presence of Measurement Uncertainty: an Approach in Terms of the Theory of Evidence

Abstract Knowledge of the measurement uncertainty of test results is fundamentally important for laboratories, their customers and all parties using and interpreting these results. In conformity assessment, a measurement result is used to decide if an item of interest conforms to a specified requirement. Because of measurement uncertainty, there is always the risk of incorrectly deciding whether or not an item conforms to a specified requirement based on the measured value of a property of the item. Conformity assessment can be quite challenging when the entity measured is so close to the tolerance limits of the specification that its uncertainty, however estimated, critically affects decision-making. In such cases, different decision rules can be used to make statements of conformity. The aim of this paper is to provide a survey of methods for the evaluation of measurement uncertainty in testing, as well as to stress the need for appropriate estimation of measurement uncertainty. This paper also aims to assist testing laboratories in understanding the different decision rules used in conformity assessment and level of risk (such as false accept and false reject) associated with the decision rule employed.

Download Full-text

Risk calculations for conformity assessment in practice

19th International Congress of Metrology (CIM2019) ◽

10.1051/metrology/201916001 ◽

2019 ◽

Author(s):

Alexandre Allard ◽

Nicolas Fischer ◽

Ian Smith ◽

Peter Harris ◽

Leslie Pendrill

Keyword(s):

Decision Making ◽

Measurement Uncertainty ◽

Decision Rule ◽

Prior Information ◽

Conformity Assessment ◽

Global Risk ◽

Wrong Decision ◽

Iso 17025 ◽

Incorrect Decision ◽

Testing Laboratories

In 2012, the Joint Committee for Guides in Metrology (JCGM) published novel guidance on the consideration of measurement uncertainty for decision-making in conformity assessment (JCGM 106:2012). The two situations of making a wrong decision are considered: the risk of accepting a non-conforming item, denoted as the customer risk, and the risk of rejecting a conforming item, denoted as the producer risk. In 2017, the revision of ISO 17025 obliged calibration and testing laboratories to “document the decision rule employed, taking into account the level of risk (such as false accept and false reject and statistical assumptions) associated with the decision rule employed, and apply the decision rule” in the context of the decision made about the conformity of an item. However, JCGM 106:2012 can in some cases be perceived as quite difficult to apply for non-statisticians as it mainly relies on calculations involving probability distributions. In order to facilitate uptake of the methodology of JCGM 106:2012, EURAMET is funding the project EMPIR 17SIP05 “CASoft” (2018 – 2020), involving the National Measurement Institutes from France, Sweden and the UK. The objective is to make the methodology accessible to organisations involved in decision-making in conformity assessment: calibration and testing laboratories, industrialists and regulation authorities. Where the customer or producer are concerned, there are two kinds of risks arising from measurement uncertainty: specific risk which concerns the risk of an incorrect decision for a particular item and global risk which is the risk of an incorrect decision for any item chosen at random. Both kinds of risk may involve prior information, taken into account through a so-called prior probability distribution, introducing the concept of a Bayesian evaluation of the risks. If a calibration and testing laboratory performing the measurement has difficulty accessing prior information, it is likely that the industrialist in control of production processes will have some idea of the quality of the items produced. In this paper, the two problems of estimating the specific and global risks are addressed. The consideration of prior information is also discussed through a practical example as well as the use of software implementing the methodology, which will be made publically available at the end of the project.

Download Full-text

Handling uncertainty in the hydroinformatic process

Journal of Hydroinformatics ◽

10.2166/hydro.2003.0019 ◽

2003 ◽

Vol 5 (4) ◽

pp. 215-232 ◽

Cited By ~ 48

Author(s):

Jim W. Hall

Keyword(s):

Decision Making ◽

Probability Distributions ◽

Possibility Theory ◽

Point Of View ◽

Fuzzy Decision Making ◽

Uncertain Information ◽

Fuzzy Decision ◽

Competing Models ◽

Theory Of Evidence ◽

Significant Bearing

Hydroinformatics combines topics of modelling and decision-making, both of which have attracted a great deal of attention outside hydroinformatics from the point of view of uncertainty. Epistemic uncertainties are due to the inevitably incomplete evidence about the dependability of a model or set of competing models. Inherent uncertainties are due to the varying information content inherent in measurements or model predictions, be they probabilistic or fuzzy. Decision-making in management of the aquatic environment is, more often than not, a complex, discursive, multi-player process. The requirement for hydroinformatics systems is to support rather than replace human judgment in this process, a requirement that has significant bearing on the treatment of uncertainty. Furthermore, a formal language is required to encode uncertainty in computer systems. We therefore review the modern mathematics of uncertainty, starting first with probability theory and then extending to fuzzy set theory and possibility theory, the theory of evidence (and its random set counterpart), which generalises probability and possibility theory, and higher-order generalisations. A simple example from coastal hydraulics illustrates how a range of types of uncertain information (including probability distributions, interval measurements and fuzzy sets) can be handled in the types of algebraic or numerical functions that form the kernel of most hydroinformatic systems.

Download Full-text

Rebuilding global fisheries under uncertainty

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.1902657116 ◽

2019 ◽

Vol 116 (32) ◽

pp. 15985-15990 ◽

Cited By ~ 9

Author(s):

Milad Memarzadeh ◽

Gregory L. Britten ◽

Boris Worm ◽

Carl Boettiger

Keyword(s):

Decision Making ◽

Environmental Change ◽

Measurement Uncertainty ◽

Population Models ◽

Decision Making Process ◽

Future Prospects ◽

Economic Returns ◽

Stock Status ◽

Decision Methods

Current and future prospects for successfully rebuilding global fisheries remain debated due to uncertain stock status, variable management success, and disruptive environmental change. While scientists routinely account for some of this uncertainty in population models, the mechanisms by which this translates into decision-making and policy are problematic and can lead to unintentional overexploitation. Here, we explicitly track the role of measurement uncertainty and environmental variation in the decision-making process for setting catch quotas. Analyzing 109 well-sampled stocks from all oceans, we show that current practices may attain 55% recovery on average, while richer decision methods borrowed from robotics yield 85% recovery of global stocks by midcentury, higher economic returns, and greater robustness to environmental surprises. These results challenge the consensus that global fisheries can be rebuilt by existing approaches alone, while also underscoring that rebuilding stocks may still be achieved by improved decision-making tools that optimally manage this uncertainty.

Download Full-text