scholarly journals Robust scoring rules

2020 ◽  
Vol 15 (3) ◽  
pp. 955-987 ◽  
Author(s):  
Elias Tsakas
Keyword(s):  
Mere Exposure ◽  
Scoring Rules ◽  
Belief Updating ◽  
Scoring Rule ◽  
Belief Elicitation ◽  
Scale Down ◽  
The Subject ◽  
Arbitrary Precision

Is it possible to guarantee that the mere exposure of a subject to a belief elicitation task will not affect the very same beliefs that we are trying to elicit? In this paper, we introduce mechanisms that make it simultaneously strictly dominant for the subject (a) not to acquire any information that could potentially lead to belief updating as a response to the incentives provided by the mechanism itself, and (b) to report his beliefs truthfully. Such mechanisms are called robust scoring rules. We prove that robust scoring rules always exist under mild assumptions on the subject's costs for acquiring information. Moreover, every scoring rule can become approximately robust, in the sense that if we scale down the incentives sufficiently, we will approximate with arbitrary precision the beliefs that the subject would have held if he had not been confronted with the belief‐elicitation task.

scholarly journals Evaluating probabilistic forecasts of football matches: the case against the ranked probability score

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Edward Wheatcroft
Keyword(s):  
Scoring Rules ◽  
Brier Score ◽  
Sporting Events ◽  
Forecast Performance ◽  
Scoring Rule ◽  
Probability Score ◽  
Probabilistic Forecasts ◽  
Non Local ◽  
Evaluating Forecasts ◽  
Non Locality

Abstract A scoring rule is a function of a probabilistic forecast and a corresponding outcome used to evaluate forecast performance. There is some debate as to which scoring rules are most appropriate for evaluating forecasts of sporting events. This paper focuses on forecasts of the outcomes of football matches. The ranked probability score (RPS) is often recommended since it is ‘sensitive to distance’, that is it takes into account the ordering in the outcomes (a home win is ‘closer’ to a draw than it is to an away win). In this paper, this reasoning is disputed on the basis that it adds nothing in terms of the usual aims of using scoring rules. A local scoring rule is one that only takes the probability placed on the outcome into consideration. Two simulation experiments are carried out to compare the performance of the RPS, which is non-local and sensitive to distance, the Brier score, which is non-local and insensitive to distance, and the Ignorance score, which is local and insensitive to distance. The Ignorance score outperforms both the RPS and the Brier score, casting doubt on the value of non-locality and sensitivity to distance as properties of scoring rules in this context.

scholarly journals Characterising scoring rules by their solution in iteratively undominated strategies

2021 ◽  
Author(s):  
Christian Basteck
Keyword(s):  
Monotonicity Property ◽  
Scoring Rules ◽  
The Other ◽  
Approval Voting ◽  
Property A ◽  
Borda Rule ◽  
Scoring Rule ◽  
Direct Mechanism ◽  
Social Choice Correspondence ◽  
Choice Correspondence

AbstractWe characterize voting procedures according to the social choice correspondence they implement when voters cast ballots strategically, applying iteratively undominated strategies. In elections with three candidates, the Borda Rule is the unique positional scoring rule that satisfies unanimity (U) (i.e., elects a candidate whenever it is unanimously preferred) and is majoritarian after eliminating a worst candidate (MEW)(i.e., if there is a unanimously disliked candidate, the majority-preferred among the other two is elected). In a larger class of rules, Approval Voting is characterized by a single axiom that implies both U and MEW but is weaker than Condorcet-consistency (CON)—it is the only direct mechanism scoring rule that is majoritarian after eliminating a Pareto-dominated candidate (MEPD)(i.e., if there is a Pareto-dominated candidate, the majority-preferred among the other two is elected); among all finite scoring rules that satisfy MEPD, Approval Voting is the most decisive. However, it fails a desirable monotonicity property: a candidate that is elected for some preference profile, may lose the election once she gains further in popularity. In contrast, the Borda Rule is the unique direct mechanism scoring rule that satisfies U, MEW and monotonicity (MON). There exists no direct mechanism scoring rule that satisfies both MEPD and MON and no finite scoring rule satisfying CON.

scholarly journals Belief Elicitation with a Synchronized Lottery Choice Menu That Is Invariant to Risk Attitudes

2016 ◽  
Vol 8 (1) ◽  
pp. 110-139 ◽  
Author(s):  
Charles A. Holt ◽  
Angela M. Smith
Keyword(s):  
Information Processing ◽  
Incentive Compatibility ◽  
Risk Attitudes ◽  
Two Stage ◽  
Processing Task ◽  
Scoring Rule ◽  
Belief Elicitation ◽  
Lottery Choices

This paper uses a Bayesian information processing task to compare belief elicitation mechanisms including a quadratic scoring rule, a Becker-DeGroot-Marschak pricing procedure, and a two-stage menu of lottery choices that is structured to identify a precise point of probability indifference. The choice menu yields a higher incidence of correct Bayesian responses and lower belief error averages. Unlike the quadratic scoring rule, the binary payoffs for the lottery choice mechanism are synchronized to provide theoretical incentive-compatibility regardless of risk attitudes. In addition, the choice menu structure is more transparent and intuitive than the Becker-DeGroot-Marschak procedure. (JEL C91, D44, D81, D83)

Mere Exposure to Stimulus and Response Terms and Paired-Associate Learning

1977 ◽  
Vol 41 (3_suppl) ◽  
pp. 1167-1171
Author(s):  
J. M. Innes
Keyword(s):  
Significant Interaction ◽  
Paired Associate ◽  
Paired Associate Learning ◽  
Mere Exposure ◽  
Stimulus Term ◽  
Associate Learning ◽  
The Subject

Subjects learned four-item paired-associate lists by the method of anticipation. Items were composed from Turkish-like words which the subject had either never seen previously or which had been exposed 16 times in a previous familiarization process designed to minimize the induction of verbal satiation. Analysis of the data indicated significant facilitation of learning due to familiarization with the stimulus term and a significant interaction of stimulus and response terms. The experiment demonstrates the likely importance of the encoding of stimulus terms in the learning of paired-associate lists and shows that mere exposure to materials can modify later response to those materials.

Tlie Measurement of Performance in Probabilistic Diagnosis

1978 ◽  
Vol 17 (04) ◽  
pp. 238-246 ◽  
Author(s):  
J. Hilden ◽  
J. D. F. Habbema ◽  
B. Bjerregaard
Keyword(s):  
Abdominal Pain ◽  
Acute Abdominal Pain ◽  
Continuous Functions ◽  
Error Rates ◽  
Scoring Rules ◽  
Evaluation Tools ◽  
Discriminatory Ability ◽  
Scoring Rule ◽  
Probability Prediction ◽  
Probabilistic Diagnosis

Within the framework of diagnostic probability prediction the problem of measuring discriminatory ability is operationally defined as the problem of measuring the agreement between probabilistic predictions and actual outcomes. We present a number of so-called scoring rules developed to this end. Most of these are continuous functions of the assigned probabilities. Discontinuous rules, including conventional non-error rates, are discussed by way of contrast. The concept of properness of a scoring-rule is discussed and the desirability of properness is argued. Separate sections deal with the problems connected with uncommon diseases and methods utilizing subdivisions of the patient material. The distinction between the three concepts of discriminatory ability, sharpness and reliability is explained. The evaluation tools developed are applied to previously presented data from the Copenhagen Acute Abdominal Pain Study.

scholarly journals Neural Network Identifiability for a Family of Sigmoidal Nonlinearities

2021 ◽  
Author(s):  
Verner Vlačić ◽  
Helmut Bölcskei
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Uniform Norm ◽  
Feed Forward Neural Network ◽  
The Subject ◽  
Hidden Layer ◽  
Arbitrary Precision ◽  
Necessary And Sufficient ◽  
Fully Connected

AbstractThis paper addresses the following question of neural network identifiability: Does the input–output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? The existing literature on the subject (Sussman in Neural Netw 5(4):589–593, 1992; Albertini et al. in Artificial neural networks for speech and vision, 1993; Fefferman in Rev Mat Iberoam 10(3):507–555, 1994) suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided that the networks under consideration satisfy certain “genericity conditions.” The results in Sussman (1992) and Albertini et al. (1993) apply to networks with a single hidden layer and in Fefferman (1994) the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

THE HIDDEN BEAUTY OF THE QUADRATIC MARKET SCORING RULE: A UNIFORM LIQUIDITY MARKET MAKER, WITH VARIATIONS

2012 ◽  
Vol 1 (2) ◽  
pp. 111-125 ◽  
Author(s):  
Michael Abramovicz
Keyword(s):  
Scoring Rules ◽  
Prediction Markets ◽  
Market Maker ◽  
Scoring Rule ◽  
Market Makers ◽  
Market Participants ◽  
Aggregate Information ◽  
Spectrum Market

For some applications, prediction markets that rely entirely on voluntary transactions between individual participants may provide insufficient liquidity to aggregate information effectively, especially where the number of participants is small. A solution to this problem is to rely on an automated market maker, which allows participants to buy from or sell to the house. Robin Hanson has described a class of automated market makers called market scoring rules. This Article examines a member of this class that has received little attention, the quadratic market scoring rule. Its prime virtue is that it provides uniform liquidity across the probability or prediction spectrum. Market participants will thus have the same incentive to do research that is expected to produce an expected change in the market prediction, regardless of the current prediction. Formulas are provided for implementing the quadratic market scoring rule, as well as variations, for example to implement conditional markets.

scholarly journals A construction principle for proper scoring rules

2021 ◽  
Vol 8 (24) ◽  
pp. 297-301
Author(s):  
Jonas Brehmer
Keyword(s):  
Predictive Distribution ◽  
Scoring Rules ◽  
Parametric Family ◽  
Scoring Rule ◽  
Proper Scoring Rules ◽  
Construction Principle ◽  
Probabilistic Forecasts ◽  
New Scoring

Proper scoring rules enable decision-theoretically principled comparisons of probabilistic forecasts. New scoring rules can be constructed by identifying the predictive distribution with an element of a parametric family and then applying a known scoring rule. We introduce a condition which ensures propriety in this construction and thereby obtain novel proper scoring rules.

A Note on Sandroni-Shmaya Belief Elicitation Mechanism

2017 ◽  
Vol 10 (2) ◽  
pp. 14-21 ◽  
Author(s):  
Arthur Carvalho
Keyword(s):  
Economic Theory ◽  
Scoring Rules ◽  
Risk Attitudes ◽  
Risk And Uncertainty ◽  
Proper Scoring Rules ◽  
Incentive Compatible ◽  
Decisions Under Risk ◽  
Belief Elicitation ◽  
Proper Scoring Rule ◽  
Extra Assumption

Incentive-compatible methods for eliciting beliefs, such as proper scoring rules, often rely on strong assumptions about how humans behave when making decisions under risk and uncertainty. For example, standard proper scoring rules assume that humans are risk neutral, an assumption that is often violated in practice. Under such an assumption, proper scoring rules induce honest reporting of beliefs, in a sense that experts maximize their expected scores from a proper scoring rule by honestly reporting their beliefs.Sandroni and Shmaya [Economic Theory Bulletin, volume 1, issue 1, 2013] suggested a remarkable mechanism based on proper scoring rules that induces honest reporting of beliefs without any assumptions on experts’ risk attitudes. In particular, the authors claimed that the mechanism relies only on the natural assumptions of probabilistic sophistication and dominance. We suggest in this paper that the reduction of compound lotteries axiom is another assumption required for Sandroni and Shmaya’s mechanism to induce honest reporting of beliefs. We further elaborate on the implications of such an extra assumption in light of recent findings regarding the reduction of compound lotteries axiom.

Sign in / Sign up

Export Citation Format

Share Document