A Comment on Diagnostic Tools for Counterfactual Inference

2009 ◽  
Vol 17 (1) ◽  
pp. 89-106 ◽  
Author(s):  
Nicholas Sambanis ◽  
Alexander Michaelides
Keyword(s):  
Monte Carlo ◽  
Political Science ◽  
Diagnostic Tools ◽  
Counterfactual Analysis ◽  
Data Sets ◽  
Explanatory Variables ◽  
Monte Carlo Experiments ◽  
Data Points

We evaluate two diagnostic tools used to determine if counterfactual analysis requires extrapolation. Counterfactuals based on extrapolation are model dependent and might not support empirically valid inferences. The diagnostics help researchers identify those counterfactual “what if” questions that are empirically plausible. We show, through simple Monte Carlo experiments, that these diagnostics will often detect extrapolation, suggesting that there is a risk of biased counterfactual inference when there is no such risk of extrapolation bias in the data. This is because the diagnostics are affected by what we call the n/k problem: as the number of data points relative to the number of explanatory variables decreases, the diagnostics are more likely to detect the risk of extrapolation bias even when such risk does not exist. We conclude that the diagnostics provide too severe a test for many data sets used in political science.

scholarly journals Testing a Parametric Model Against a Semiparametric Alternative

1994 ◽  
Vol 10 (5) ◽  
pp. 821-848 ◽  
Author(s):  
Joel L. Horowitz ◽  
Wolfgang Härdle
Keyword(s):  
Monte Carlo ◽  
Alternative Model ◽  
Random Variable ◽  
Parametric Model ◽  
Conditional Moment ◽  
Moment Conditions ◽  
Explanatory Variables ◽  
Conditional Moments ◽  
Monte Carlo Experiments ◽  
The Mean

This paper describes a method for testing a parametric model of the mean of a random variable Y conditional on a vector of explanatory variables X against a semiparametric alternative. The test is motivated by a conditional moment test against a parametric alternative and amounts to replacing the parametric alternative model with a semiparametric model. The resulting semiparametric test is consistent against a larger set of alternatives than are parametric conditional moments tests based on finitely many moment conditions. The results of Monte Carlo experiments and an application illustrate the usefulness of the new test.

scholarly journals NONPARAMETRIC SIGNIFICANCE TESTING

2000 ◽  
Vol 16 (4) ◽  
pp. 576-601 ◽  
Author(s):  
Pascal Lavergne ◽  
Quang Vuong
Keyword(s):  
Monte Carlo ◽  
Null Hypothesis ◽  
Kernel Method ◽  
Significance Testing ◽  
Local Alternatives ◽  
Test Statistic ◽  
Explanatory Variables ◽  
Monte Carlo Experiments ◽  
Standard Normal ◽  
Complete Set

A procedure for testing the significance of a subset of explanatory variables in a nonparametric regression is proposed. Our test statistic uses the kernel method. Under the null hypothesis of no effect of the variables under test, we show that our test statistic has an nhp2/2 standard normal limiting distribution, where p2 is the dimension of the complete set of regressors. Our test is one-sided, consistent against all alternatives and detects local alternatives approaching the null at rate slower than n−1/2h−p2/4. Our Monte-Carlo experiments indicate that it outperforms the test proposed by Fan and Li (1996, Econometrica 64, 865–890).

scholarly journals Estimation of Stochastic Frontier Models with Fixed Effects through Monte Carlo Maximum Likelihood

2011 ◽  
Vol 2011 ◽  
pp. 1-13 ◽  
Author(s):  
Grigorios Emvalomatis ◽  
Spiro E. Stefanou ◽  
Alfons Oude Lansink
Keyword(s):  
Monte Carlo ◽  
Random Effects ◽  
Fixed Effects ◽  
Likelihood Function ◽  
Stochastic Frontier ◽  
General Context ◽  
Explanatory Variables ◽  
Monte Carlo Experiments ◽  
Incidental Parameters ◽  
Stochastic Frontier Models

Estimation of nonlinear fixed-effects models is plagued by the incidental parameters problem. This paper proposes a procedure for choosing appropriate densities for integrating the incidental parameters from the likelihood function in a general context. The densities are based on priors that are updated using information from the data and are robust to possible correlation of the group-specific constant terms with the explanatory variables. Monte Carlo experiments are performed in the specific context of stochastic frontier models to examine and compare the sampling properties of the proposed estimator with those of the random-effects and correlated random-effects estimators. The results suggest that the estimator is unbiased even in short panels. An application to a cross-country panel of EU manufacturing industries is presented as well. The proposed estimator produces a distribution of efficiency scores suggesting that these industries are highly efficient, while the other estimators suggest much poorer performance.

When BLUE is not best: non-normal errors and the linear model

2018 ◽  
Vol 8 (1) ◽  
pp. 136-148 ◽  
Author(s):  
Daniel K. Baissa ◽  
Carlisle Rainey
Keyword(s):  
Monte Carlo ◽  
Statistical Theory ◽  
Monte Carlo Simulations ◽  
Least Squares ◽  
Political Science ◽  
Linear Model ◽  
Linear Models ◽  
Least Squares Estimators ◽  
Data Points ◽  
Least Squares Estimates

AbstractResearchers in political science often estimate linear models of continuous outcomes using least squares. While it is well known that least-squares estimates are sensitive to single, unusual data points, this knowledge has not led to careful practices when using least-squares estimators. Using statistical theory and Monte Carlo simulations, we highlight the importance of using more robust estimators along with variable transformations. We also discuss several approaches to detect, summarize, and communicate the influence of particular data points.

scholarly journals Sequential Tests for Large-Scale Learning

2016 ◽  
Vol 28 (1) ◽  
pp. 45-70 ◽  
Author(s):  
Anoop Korattikara ◽  
Yutian Chen ◽  
Max Welling
Keyword(s):  
Monte Carlo ◽  
Large Scale ◽  
Data Sets ◽  
Posterior Inference ◽  
Sequential Hypothesis ◽  
Inference Algorithms ◽  
Sequential Tests ◽  
Wrong Direction ◽  
Data Points ◽  
Context Of Learning

We argue that when faced with big data sets, learning and inference algorithms should compute updates using only subsets of data items. We introduce algorithms that use sequential hypothesis tests to adaptively select such a subset of data points. The statistical properties of this subsampling process can be used to control the efficiency and accuracy of learning or inference. In the context of learning by optimization, we test for the probability that the update direction is no more than 90 degrees in the wrong direction. In the context of posterior inference using Markov chain Monte Carlo, we test for the probability that our decision to accept or reject a sample is wrong. We experimentally evaluate our algorithms on a number of models and data sets.

scholarly journals Goodness-of-Fit Tests for Bivariate Time Series of Counts

Econometrics ◽  
2021 ◽  
Vol 9 (1) ◽  
pp. 10
Author(s):  
Šárka Hudecová ◽  
Marie Hušková ◽  
Simos G. Meintanis
Keyword(s):  
Goodness Of Fit ◽  
Probability Generating Function ◽  
Parametric Bootstrap ◽  
Real Data ◽  
Data Sets ◽  
Test Statistics ◽  
Finite Sample ◽  
Generalized Poisson ◽  
Goodness Of Fit Tests ◽  
Monte Carlo Experiments

This article considers goodness-of-fit tests for bivariate INAR and bivariate Poisson autoregression models. The test statistics are based on an L2-type distance between two estimators of the probability generating function of the observations: one being entirely nonparametric and the second one being semiparametric computed under the corresponding null hypothesis. The asymptotic distribution of the proposed tests statistics both under the null hypotheses as well as under alternatives is derived and consistency is proved. The case of testing bivariate generalized Poisson autoregression and extension of the methods to dimension higher than two are also discussed. The finite-sample performance of a parametric bootstrap version of the tests is illustrated via a series of Monte Carlo experiments. The article concludes with applications on real data sets and discussion.

scholarly journals Statistical Tests of Symbolic Dynamics

Mathematics ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 817
Author(s):  
Fernando López ◽  
Mariano Matilla-García ◽  
Jesús Mur ◽  
Manuel Ruiz Marín
Keyword(s):  
Monte Carlo ◽  
Symbolic Dynamics ◽  
Null Hypothesis ◽  
Statistical Tests ◽  
General Applicability ◽  
Monte Carlo Experiments ◽  
The Mean ◽  
Theoretical Application ◽  
New Statistics ◽  
General Method

A novel general method for constructing nonparametric hypotheses tests based on the field of symbolic analysis is introduced in this paper. Several existing tests based on symbolic entropy that have been used for testing central hypotheses in several branches of science (particularly in economics and statistics) are particular cases of this general approach. This family of symbolic tests uses few assumptions, which increases the general applicability of any symbolic-based test. Additionally, as a theoretical application of this method, we construct and put forward four new statistics to test for the null hypothesis of spatiotemporal independence. There are very few tests in the specialized literature in this regard. The new tests were evaluated with the mean of several Monte Carlo experiments. The results highlight the outstanding performance of the proposed test.

scholarly journals ON GENERATING DIGITAL ELEVATION MODELS FROM LIDAR DATA – RESOLUTION VERSUS ACCURACY AND TOPOGRAPHIC WETNESS INDEX INDICES IN NORTHERN PEATLANDS

2012 ◽  
Vol 38 (2) ◽  
pp. 57-69 ◽  
Author(s):  
Abdulghani Hasan ◽  
Petter Pilesjö ◽  
Andreas Persson
Keyword(s):  
Large Scale ◽  
Drainage Area ◽  
Data Sets ◽  
Topographic Wetness Index ◽  
Absolute Deviation ◽  
Digital Elevation ◽  
Elevation Data ◽  
Scale Modelling ◽  
Data Points ◽  
Emission Modelling

Global change and GHG emission modelling are dependent on accurate wetness estimations for predictions of e.g. methane emissions. This study aims to quantify how the slope, drainage area and the TWI vary with the resolution of DEMs for a flat peatland area. Six DEMs with spatial resolutions from 0.5 to 90 m were interpolated with four different search radiuses. The relationship between accuracy of the DEM and the slope was tested. The LiDAR elevation data was divided into two data sets. The number of data points facilitated an evaluation dataset with data points not more than 10 mm away from the cell centre points in the interpolation dataset. The DEM was evaluated using a quantile-quantile test and the normalized median absolute deviation. It showed independence of the resolution when using the same search radius. The accuracy of the estimated elevation for different slopes was tested using the 0.5 meter DEM and it showed a higher deviation from evaluation data for steep areas. The slope estimations between resolutions showed differences with values that exceeded 50%. Drainage areas were tested for three resolutions, with coinciding evaluation points. The model ability to generate drainage area at each resolution was tested by pair wise comparison of three data subsets and showed differences of more than 50% in 25% of the evaluated points. The results show that consideration of DEM resolution is a necessity for the use of slope, drainage area and TWI data in large scale modelling.

scholarly journals Inter- and Intralaboratory Comparison of JC Polyomavirus Antibody Testing Using Two Different Virus-Like Particle-Based Assays

2014 ◽  
Vol 21 (11) ◽  
pp. 1581-1588 ◽  
Author(s):  
Piotr Kardas ◽  
Mohammadreza Sadeghi ◽  
Fabian H. Weissbach ◽  
Tingting Chen ◽  
Lea Hedman ◽  
...  
Keyword(s):  
Risk Stratification ◽  
Data Sets ◽  
Antibody Testing ◽  
Jc Polyomavirus ◽  
Virus Like Particle ◽  
Interlaboratory Variability ◽  
Increased Risk ◽  
Basic Protocol ◽  
Data Points ◽  
Reference Serum

ABSTRACTJC polyomavirus (JCPyV) can cause progressive multifocal leukoencephalopathy (PML), a debilitating, often fatal brain disease in immunocompromised patients. JCPyV-seropositive multiple sclerosis (MS) patients treated with natalizumab have a 2- to 10-fold increased risk of developing PML. Therefore, JCPyV serology has been recommended for PML risk stratification. However, different antibody tests may not be equivalent. To study intra- and interlaboratory variability, sera from 398 healthy blood donors were compared in 4 independent enzyme-linked immunoassay (ELISA) measurements generating >1,592 data points. Three data sets (Basel1, Basel2, and Basel3) used the same basic protocol but different JCPyV virus-like particle (VLP) preparations and introduced normalization to a reference serum. The data sets were also compared with an independent method using biotinylated VLPs (Helsinki1). VLP preadsorption reducing ≥35% activity was used to identify seropositive sera. The results indicated that Basel1, Basel2, Basel3, and Helsinki1 were similar regarding overall data distribution (P= 0.79) and seroprevalence (58.0, 54.5, 54.8, and 53.5%, respectively;P= 0.95). However, intra-assay intralaboratory comparison yielded 3.7% to 12% discordant results, most of which were close to the cutoff (0.080 < optical density [OD] < 0.250) according to Bland-Altman analysis. Introduction of normalization improved overall performance and reduced discordance. The interlaboratory interassay comparison between Basel3 and Helsinki1 revealed only 15 discordant results, 14 (93%) of which were close to the cutoff. Preadsorption identified specificities of 99.44% and 97.78% and sensitivities of 99.54% and 95.87% for Basel3 and Helsinki1, respectively. Thus, normalization to a preferably WHO-approved reference serum, duplicate testing, and preadsorption for samples around the cutoff may be necessary for reliable JCPyV serology and PML risk stratification.

Sign in / Sign up

Export Citation Format

Share Document