scholarly journals Finite-Sample Bounds on the Accuracy of Plug-In Estimators of Fisher Information

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 545
Author(s):  
Wei Cao ◽  
Alex Dytso ◽  
Michael Fauß ◽  
H. Vincent Poor
Keyword(s):  
Fisher Information ◽  
Gaussian Noise ◽  
Minimum Mean Square Error ◽  
Score Function ◽  
Random Variable ◽  
Rates Of Convergence ◽  
Upper Bounds ◽  
Regularity Conditions ◽  
Performance Bounds ◽  
Finite Sample

Finite-sample bounds on the accuracy of Bhattacharya’s plug-in estimator for Fisher information are derived. These bounds are further improved by introducing a clipping step that allows for better control over the score function. This leads to superior upper bounds on the rates of convergence, albeit under slightly different regularity conditions. The performance bounds on both estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown’s identity, two corresponding estimators of the minimum mean-square error are proposed.

scholarly journals Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 722
Author(s):  
Xin Li ◽  
Dongya Wu
Keyword(s):  
Additive Noise ◽  
Rates Of Convergence ◽  
Upper Bounds ◽  
High Dimensional ◽  
Regularity Conditions ◽  
Errors In Variables ◽  
Lower And Upper Bounds ◽  
Information Theoretic ◽  
Minimax Rates ◽  
Sparse Vector

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the ℓp(1≤p<∞)-losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when p=2, which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the ℓ2-loss.

scholarly journals Estimation in Partial Functional Linear Spatial Autoregressive Model

Mathematics ◽  
2020 ◽  
Vol 8 (10) ◽  
pp. 1680
Author(s):  
Yuping Hu ◽  
Siyu Wu ◽  
Sanying Feng ◽  
Junliang Jin
Keyword(s):  
Autoregressive Model ◽  
Estimation Method ◽  
Random Variable ◽  
Regularity Conditions ◽  
Functional Regression ◽  
Finite Sample ◽  
Spatial Autoregressive Model ◽  
Explanatory Variables ◽  
Spatial Autoregressive ◽  
Slope Function

Functional regression allows for a scalar response to be dependent on a functional predictor; however, not much work has been done when response variables are dependence spatial variables. In this paper, we introduce a new partial functional linear spatial autoregressive model which explores the relationship between a scalar dependence spatial response variable and explanatory variables containing both multiple real-valued scalar variables and a function-valued random variable. By means of functional principal components analysis and the instrumental variable estimation method, we obtain the estimators of the parametric component and slope function of the model. Under some regularity conditions, we establish the asymptotic normality for the parametric component and the convergence rate for slope function. At last, we illustrate the finite sample performance of our proposed methods with some simulation studies.

scholarly journals On Asymptotic Efficiency of the M2M4 Signal-to-Noise Estimator for Deterministic Complex Sinusoids

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 4950
Author(s):  
Gianmarco Romano
Keyword(s):  
Covariance Matrix ◽  
Gaussian Noise ◽  
Asymptotic Efficiency ◽  
Noise Power ◽  
Finite Sample ◽  
Signal To Noise ◽  
Numerical Examples ◽  
Unknown Phase ◽  
The Moment ◽  
Asymptotically Efficient

The moment-based M2M4 signal-to-noise (SNR) estimator was proposed for a complex sinusoidal signal with a deterministic but unknown phase corrupted by additive Gaussian noise by Sekhar and Sreenivas. The authors studied its performances only through numerical examples and concluded that the proposed estimator is asymptotically efficient and exhibits finite sample super-efficiency for some combinations of signal and noise power. In this paper, we derive the analytical asymptotic performances of the proposed M2M4 SNR estimator, and we show that, contrary to what it has been concluded by Sekhar and Sreenivas, the proposed estimator is neither (asymptotically) efficient nor super-efficient. We also show that when dealing with deterministic signals, the covariance matrix needed to derive asymptotic performances must be explicitly derived as its known general form for random signals cannot be extended to deterministic signals. Numerical examples are provided whose results confirm the analytical findings.

scholarly journals Generalized Wavelet Fisher’s Information of1/fαSignals

2015 ◽  
Vol 2015 ◽  
pp. 1-9 ◽  
Author(s):  
Julio Ramírez-Pacheco ◽  
Homero Toral-Cruz ◽  
Luis Rizo-Domínguez ◽  
Joaquin Cortez-Gonzalez
Keyword(s):  
Fisher Information ◽  
Gaussian Noise ◽  
Structural Breaks ◽  
Wavelet Domain ◽  
Fractional Gaussian Noise ◽  
Domain Definition ◽  
Potential Applications ◽  
Definition Of ◽  
The Time Domain ◽  
Fisher's Information

This paper defines the generalized wavelet Fisher information of parameterq. This information measure is obtained by generalizing the time-domain definition of Fisher’s information of Furuichi to the wavelet domain and allows to quantify smoothness and correlation, among other signals characteristics. Closed-form expressions of generalized wavelet Fisher information for1/fαsignals are determined and a detailed discussion of their properties, characteristics and their relationship with waveletq-Fisher information are given. Information planes of1/fsignals Fisher information are obtained and, based on these, potential applications are highlighted. Finally, generalized wavelet Fisher information is applied to the problem of detecting and locating weak structural breaks in stationary1/fsignals, particularly for fractional Gaussian noise series. It is shown that by using a joint Fisher/F-Statistic procedure, significant improvements in time and accuracy are achieved in comparison with the sole application of theF-statistic.

scholarly journals Distributional Replication

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 1063
Author(s):  
Brendan K. Beare
Keyword(s):  
Hedge Fund ◽  
Minimum Cost ◽  
Random Variable ◽  
Optimal Approximation ◽  
Regularity Conditions ◽  
Return Distribution ◽  
Continuous Random Variable ◽  
Hedge Fund Replication ◽  
Market Index ◽  
Fund Return

A function which transforms a continuous random variable such that it has a specified distribution is called a replicating function. We suppose that functions may be assigned a price, and study an optimization problem in which the cheapest approximation to a replicating function is sought. Under suitable regularity conditions, including a bound on the entropy of the set of candidate approximations, we show that the optimal approximation comes close to achieving distributional replication, and close to achieving the minimum cost among replicating functions. We discuss the relevance of our results to the financial literature on hedge fund replication; in this case, the optimal approximation corresponds to the cheapest portfolio of market index options which delivers the hedge fund return distribution.

scholarly journals A Bi-Level Weibull Model with Applications to Two Ordered Events

2020 ◽  
Vol 8 (2) ◽  
Author(s):  
Shuguang Song ◽  
Hanlin Liu ◽  
Mimi Zhang ◽  
Min Xie
Keyword(s):  
Maximum Likelihood ◽  
Estimation Method ◽  
Mixture Distribution ◽  
Likelihood Estimation ◽  
Random Variable ◽  
Weibull Model ◽  
Reliability Function ◽  
The Other ◽  
Regularity Conditions ◽  
Maximum Likelihood Estimation Method

In this paper, we propose and study a new bivariate Weibull model, called Bi-levelWeibullModel, which arises when one failure occurs after the other. Under some specific regularity conditions, the reliability function of the second event can be above the reliability function of the first event, and is always above the reliability function of the transformed first event, which is a univariate Weibull random variable. This model is motivated by a common physical feature that arises fromseveral real applications. The two marginal distributions are a Weibull distribution and a generalized three-parameter Weibull mixture distribution. Some useful properties of the model are derived, and we also present the maximum likelihood estimation method. A real example is provided to illustrate the application of the model.

scholarly journals Universal inference

2020 ◽  
Vol 117 (29) ◽  
pp. 16880-16890 ◽  
Author(s):  
Larry Wasserman ◽  
Aaditya Ramdas ◽  
Sivaraman Balakrishnan
Keyword(s):  
Maximum Likelihood ◽  
Likelihood Ratio ◽  
Likelihood Ratio Statistic ◽  
Model Misspecification ◽  
Null Distribution ◽  
Regularity Conditions ◽  
Ratio Test ◽  
Finite Sample ◽  
Confidence Sets ◽  
Nonparametric Models

We propose a general method for constructing confidence sets and hypothesis tests that have finite-sample guarantees without regularity conditions. We refer to such procedures as “universal.” The method is very simple and is based on a modified version of the usual likelihood-ratio statistic that we call “the split likelihood-ratio test” (split LRT) statistic. The (limiting) null distribution of the classical likelihood-ratio statistic is often intractable when used to test composite null hypotheses in irregular statistical models. Our method is especially appealing for statistical inference in these complex setups. The method we suggest works for any parametric model and also for some nonparametric models, as long as computing a maximum-likelihood estimator (MLE) is feasible under the null. Canonical examples arise in mixture modeling and shape-constrained inference, for which constructing tests and confidence sets has been notoriously difficult. We also develop various extensions of our basic methods. We show that in settings when computing the MLE is hard, for the purpose of constructing valid tests and intervals, it is sufficient to upper bound the maximum likelihood. We investigate some conditions under which our methods yield valid inferences under model misspecification. Further, the split LRT can be used with profile likelihoods to deal with nuisance parameters, and it can also be run sequentially to yield anytime-valid P values and confidence sequences. Finally, when combined with the method of sieves, it can be used to perform model selection with nested model classes.

scholarly journals Optimal Short-Term Population Coding: When Fisher Information Fails

2002 ◽  
Vol 14 (10) ◽  
pp. 2317-2351 ◽  
Author(s):  
M. Bethge ◽  
D. Rotermund ◽  
K. Pawelzik
Keyword(s):  
Fisher Information ◽  
Dynamic Range ◽  
Mean Squared Error ◽  
Average Rate ◽  
Neuronal Response ◽  
Minimum Mean Square Error ◽  
Population Coding ◽  
Neuronal Firing ◽  
Tuning Curves ◽  
Efficient Coding

Efficient coding has been proposed as a first principle explaining neuronal response properties in the central nervous system. The shape of optimal codes, however, strongly depends on the natural limitations of the particular physical system. Here we investigate how optimal neuronal encoding strategies are influenced by the finite number of neurons N (place constraint), the limited decoding time window length T (time constraint), the maximum neuronal firing rate fmax (power constraint), and the maximal average rate fmax (energy constraint). While Fisher information provides a general lower bound for the mean squared error of unbiased signal reconstruction, its use to characterize the coding precision is limited. Analyzing simple examples, we illustrate some typical pitfalls and thereby show that Fisher information provides a valid measure for the precision of a code only if the dynamic range (fmin T, fmax T) is sufficiently large. In particular, we demonstrate that the optimal width of gaussian tuning curves depends on the available decoding time T. Within the broader class of unimodal tuning functions, it turns out that the shape of a Fisher-optimal coding scheme is not unique. We solve this ambiguity by taking the minimum mean square error into account, which leads to flat tuning curves. The tuning width, however, remains to be determined by energy constraints rather than by the principle of efficient coding.

Convergence rates in the law of large numbers. II

1968 ◽  
Vol 64 (2) ◽  
pp. 485-488 ◽  
Author(s):  
V. K. Rohatgi
Keyword(s):  
Convergence Rates ◽  
Random Variables ◽  
Almost Sure Convergence ◽  
Random Variable ◽  
Rates Of Convergence ◽  
Independent Random Variables ◽  
Large Numbers ◽  
Present Context ◽  
Image Position ◽  
Uniformly Bounded

Let {Xn: n ≥ 1} be a sequence of independent random variables and write Suppose that the random vairables Xn are uniformly bounded by a random variable X in the sense thatSet qn(x) = Pr(|Xn| > x) and q(x) = Pr(|Xn| > x). If qn ≤ q and E|X|r < ∞ with 0 < r < 2 then we have (see Loève(4), 242)where ak = 0, if 0 < r < 1, and = EXk if 1 ≤ r < 2 and ‘a.s.’ stands for almost sure convergence. the purpose of this paper is to study the rates of convergence ofto zero for arbitrary ε > 0. We shall extend to the present context, results of (3) where the case of identically distributed random variables was treated. The techniques used here are strongly related to those of (3).

Design Sensitivity Method for Sampling-Based RBDO With Varying Standard Deviation

2015 ◽  
Vol 138 (1) ◽  
Author(s):  
Hyunkyoo Cho ◽  
K. K. Choi ◽  
Ikjin Lee ◽  
David Lamb
Keyword(s):  
Standard Deviation ◽  
Design Variable ◽  
Score Function ◽  
Random Variable ◽  
Probability Of Failure ◽  
Design Sensitivity ◽  
Mean Value ◽  
Reliability Based Design ◽  
The Mean ◽  
Sensitivity Method

Conventional reliability-based design optimization (RBDO) uses the mean of input random variable as its design variable; and the standard deviation (STD) of the random variable is a fixed constant. However, the constant STD may not correctly represent certain RBDO problems well, especially when a specified tolerance of the input random variable is present as a percentage of the mean value. For this kind of design problem, the STD of the input random variable should vary as the corresponding design variable changes. In this paper, a method to calculate the design sensitivity of the probability of failure for RBDO with varying STD is developed. For sampling-based RBDO, which uses Monte Carlo simulation (MCS) for reliability analysis, the design sensitivity of the probability of failure is derived using a first-order score function. The score function contains the effect of the change in the STD in addition to the change in the mean. As copulas are used for the design sensitivity, correlated input random variables also can be used for RBDO with varying STD. Moreover, the design sensitivity can be calculated efficiently during the evaluation of the probability of failure. Using a mathematical example, the accuracy and efficiency of the developed design sensitivity method are verified. The RBDO result for mathematical and physical problems indicates that the developed method provides accurate design sensitivity in the optimization process.

Sign in / Sign up

Export Citation Format

Share Document