Convergence in Lp Norm

This chapter looks in detail at proofs of the weak law of large numbers (convergence in probability) using the technique of establishing convergence in Lp‐norm. The extension to a proof of almost‐sure convergence is given, and then special results for martingale differences, mixingales, and approximable processes. These results are proved in array notation to allow general forms of heterogeneity.

One-dimensional system arising in stochastic gradient descent

We consider stochastic differential equations of the form $dX_t = |f(X_t)|/t^{\gamma} dt+1/t^{\gamma} dB_t$, where f(x) behaves comparably to $|x|^k$ in a neighborhood of the origin, for $k\in [1,\infty)$. We show that there exists a threshold value $ \,{:}\,{\raise-1.5pt{=}}\, \tilde{\gamma}$ for $\gamma$, depending on k, such that if $\gamma \in (1/2, \tilde{\gamma})$, then $\mathbb{P}(X_t\rightarrow 0) = 0$, and for the rest of the permissible values of $\gamma$, $\mathbb{P}(X_t\rightarrow 0)>0$. These results extend to discrete processes that satisfy $X_{n+1}-X_n = f(X_n)/n^\gamma +Y_n/n^\gamma$. Here, $Y_{n+1}$ are martingale differences that are almost surely bounded.This result shows that for a function F whose second derivative at degenerate saddle points is of polynomial order, it is always possible to escape saddle points via the iteration $X_{n+1}-X_n =F'(X_n)/n^\gamma +Y_n/n^\gamma$ for a suitable choice of $\gamma$.

EFFICIENT TWO-STEP GENERALIZED EMPIRICAL LIKELIHOOD ESTIMATION AND TESTS WITH MARTINGALE DIFFERENCES

This paper considers two-step generalized empirical likelihood (GEL) estimation and tests with martingale differences when there is a computationally simple $\sqrt n-$ consistent estimator of nuisance parameters or the nuisance parameters can be eliminated with an estimating function of parameters of interest. As an initial estimate might have asymptotic impact on final estimates, we propose general C(α)-type transformed moments to eliminate the impact, and use them in the GEL framework to construct estimation and tests robust to initial estimates. This two-step approach can save computational burden as the numbers of moments and parameters are reduced. A properly constructed two-step GEL (TGEL) estimator of parameters of interest is asymptotically as efficient as the corresponding joint GEL estimator. TGEL removes several higher-order bias terms of a corresponding two-step generalized method of moments. Our moment functions at the true parameters are martingales, thus they cover some spatial and time series models. We investigate tests for parameter restrictions in the TGEL framework, which are locally as powerful as those in the joint GEL framework when the two-step estimator is efficient.

Deviation inequalities for Banach space valued martingales differences sequences and random fields

We establish deviation inequalities for the maxima of partial sums of a martingale differences sequence, and of an orthomartingale differences random field. These inequalities can be used to give rates for linear regression and the law of large numbers.