scholarly journals A lower bound on the probability that a binomial random variable is exceeding its mean

2016 ◽  
Vol 119 ◽  
pp. 305-309 ◽  
Author(s):  
Christos Pelekis ◽  
Jan Ramon
Keyword(s):  
Lower Bound ◽  
Random Variable ◽  
Binomial Random Variable

The Stochastic Delta Rule: Faster and More Accurate Deep Learning Through Adaptive Weight Noise

2020 ◽  
Vol 32 (5) ◽  
pp. 1018-1032 ◽  
Author(s):  
Noah Frazier-Logue ◽  
Stephen José Hanson
Keyword(s):  
Prediction Error ◽  
Saddle Points ◽  
Model Averaging ◽  
Random Variable ◽  
Mean Values ◽  
Delta Rule ◽  
Binomial Random Variable ◽  
Optimal Network ◽  
Training Error ◽  
Feature Discovery

Multilayer neural networks have led to remarkable performance on many kinds of benchmark tasks in text, speech, and image processing. Nonlinear parameter estimation in hierarchical models is known to be subject to overfitting and misspecification. One approach to these estimation and related problems (e.g., saddle points, colinearity, feature discovery) is called Dropout. The Dropout algorithm removes hidden units according to a binomial random variable with probability [Formula: see text] prior to each update, creating random “shocks” to the network that are averaged over updates (thus creating weight sharing). In this letter, we reestablish an older parameter search method and show that Dropout is a special case of this more general model, stochastic delta rule (SDR), published originally in 1990. Unlike Dropout, SDR redefines each weight in the network as a random variable with mean [Formula: see text] and standard deviation [Formula: see text]. Each weight random variable is sampled on each forward activation, consequently creating an exponential number of potential networks with shared weights (accumulated in the mean values). Both parameters are updated according to prediction error, thus resulting in weight noise injections that reflect a local history of prediction error and local model averaging. SDR therefore implements a more sensitive local gradient-dependent simulated annealing per weight converging in the limit to a Bayes optimal network. We run tests on standard benchmarks (CIFAR and ImageNet) using a modified version of DenseNet and show that SDR outperforms standard Dropout in top-5 validation error by approximately 13% with DenseNet-BC 121 on ImageNet and find various validation error improvements in smaller networks. We also show that SDR reaches the same accuracy that Dropout attains in 100 epochs in as few as 40 epochs, as well as improvements in training error by as much as 80%.

scholarly journals A refinement of normal approximation to Poisson binomial

2005 ◽  
Vol 2005 (5) ◽  
pp. 717-728 ◽  
Author(s):  
K. Neammanee
Keyword(s):  
Normal Distribution ◽  
Normal Approximation ◽  
Random Variables ◽  
Random Variable ◽  
Standard Normal Distribution ◽  
Bernoulli Random Variables ◽  
Binomial Random Variable ◽  
Standard Normal ◽  
Correction Terms ◽  
Better Than

LetX1,X2,…,Xnbe independent Bernoulli random variables withP(Xj=1)=1−P(Xj=0)=pjand letSn:=X1+X2+⋯+Xn.Snis called a Poisson binomial random variable and it is well known that the distribution of a Poisson binomial random variable can be approximated by the standard normal distribution. In this paper, we use Taylor's formula to improve the approximation by adding some correction terms. Our result is better than before and is of order1/nin the casep1=p2=⋯=pn.

scholarly journals On Tails of Perpetuities

2010 ◽  
Vol 47 (4) ◽  
pp. 1191-1194 ◽  
Author(s):  
Paweł Hitczenko
Keyword(s):  
Lower Bound ◽  
Difference Equation ◽  
Upper Bound ◽  
Random Variable ◽  
Stochastic Difference Equation

We establish an upper bound on the tails of a random variable that arises as a solution of a stochastic difference equation. In the nonnegative case our bound is similar to a lower bound obtained in Goldie and Grübel (1996).

scholarly journals Nondecreasing Lower Bound on the Poisson Cumulative Distribution Function for z Standard Deviations Above the Mean

2013 ◽  
Vol 50 (4) ◽  
pp. 909-917
Author(s):  
M. Bondareva
Keyword(s):  
Distribution Function ◽  
Lower Bound ◽  
Cumulative Distribution Function ◽  
Random Variable ◽  
Step Function ◽  
Cumulative Distribution ◽  
Control Parameters ◽  
Poisson Random Variable ◽  
The Mean ◽  
Standard Deviations

In this paper we discuss a nondecreasing lower bound for the Poisson cumulative distribution function (CDF) at z standard deviations above the mean λ, where z and λ are parameters. This is important because the normal distribution as an approximation for the Poisson CDF may overestimate or underestimate its value. A sharp nondecreasing lower bound in the form of a step function is constructed. As a corollary of the bound's properties, for a given percent α and parameter λ, the minimal z is obtained such that, for any Poisson random variable with the mean greater or equal to λ, its αth percentile is at most z standard deviations above its mean. For Poisson distributed control parameters, the corollary allows simple policies measuring performance in terms of standard deviations from a benchmark.

Negative binomial sums of random variables and discounted reward processes

1998 ◽  
Vol 35 (3) ◽  
pp. 589-599
Author(s):  
William L. Cooper
Keyword(s):  
Differential Equation ◽  
Negative Binomial ◽  
Infinite Horizon ◽  
Random Variables ◽  
Random Variable ◽  
Random Time ◽  
Sums Of Random Variables ◽  
Total Reward ◽  
Binomial Random Variable ◽  
Binomial Sums

Given a sequence of random variables (rewards), the Haviv–Puterman differential equation relates the expected infinite-horizon λ-discounted reward and the expected total reward up to a random time that is determined by an independent negative binomial random variable with parameters 2 and λ. This paper provides an interpretation of this proven, but previously unexplained, result. Furthermore, the interpretation is formalized into a new proof, which then yields new results for the general case where the rewards are accumulated up to a time determined by an independent negative binomial random variable with parameters k and λ.

A note on stochastic ordering of order statistics

1997 ◽  
Vol 34 (03) ◽  
pp. 785-789 ◽  
Author(s):  
Chunsheng Ma
Keyword(s):  
Order Statistics ◽  
Stochastic Ordering ◽  
Random Variable ◽  
Sufficient Condition ◽  
Necessary And Sufficient Condition ◽  
Stochastic Comparisons ◽  
Homogeneous Population ◽  
Independent Components ◽  
Binomial Random Variable ◽  
Necessary And Sufficient

A necessary and sufficient condition is obtained for a Poisson binomial random variable to be stochastically larger (or smaller) than a binomial random variable. It is then used to deal with the stochastic comparisons of order statistics from heterogeneous populations with those from a homogeneous population. The result has obvious applications in the stochastic comparisons of lifetimes of k-out-of-n systems having independent components.

Sign in / Sign up

Export Citation Format

Share Document