scholarly journals Mutual information decay for factors of i.i.d.

2018 ◽  
Vol 39 (11) ◽  
pp. 3015-3030
Author(s):  
BALÁZS GERENCSÉR ◽  
VIKTOR HARANGI
Keyword(s):  
Mutual Information ◽  
Upper Bound ◽  
Upper Bounds ◽  
Interesting Phenomenon ◽  
Regular Tree ◽  
Normalized Mutual Information ◽  
Distributed Processes ◽  
Independent And Identically Distributed

This paper is concerned with factors of independent and identically distributed processes on the $d$ -regular tree for $d\geq 3$ . We study the mutual information of values on two given vertices. If the vertices are neighbors (i.e. their distance is $1$ ), then a known inequality between the entropy of a vertex and the entropy of an edge provides an upper bound for the (normalized) mutual information. In this paper we obtain upper bounds for vertices at an arbitrary distance $k$ , of order $(d-1)^{-k/2}$ . Although these bounds are sharp, we also show that an interesting phenomenon occurs here: for any fixed process, the rate of mutual information decay is much faster, essentially of order $(d-1)^{-k}$ .

scholarly journals Information Bottleneck Analysis by a Conditional Mutual Information Bound

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 974
Author(s):  
Taro Tezuka ◽  
Shizuma Namekawa
Keyword(s):  
Mutual Information ◽  
Supervised Learning ◽  
Upper Bound ◽  
Latent Variables ◽  
Upper Bounds ◽  
Conditional Mutual Information ◽  
Information Bottleneck ◽  
Input Matching ◽  
Bottleneck Analysis ◽  
Information Bound

Task-nuisance decomposition describes why the information bottleneck loss I(z;x)−βI(z;y) is a suitable objective for supervised learning. The true category y is predicted for input x using latent variables z. When n is a nuisance independent from y, I(z;n) can be decreased by reducing I(z;x) since the latter upper bounds the former. We extend this framework by demonstrating that conditional mutual information I(z;x|y) provides an alternative upper bound for I(z;n). This bound is applicable even if z is not a sufficient representation of x, that is, I(z;y)≠I(x;y). We used mutual information neural estimation (MINE) to estimate I(z;x|y). Experiments demonstrated that I(z;x|y) is smaller than I(z;x) for layers closer to the input, matching the claim that the former is a tighter bound than the latter. Because of this difference, the information plane differs when I(z;x|y) is used instead of I(z;x).

scholarly journals Target space entanglement in quantum mechanics of fermions and matrices

2021 ◽  
Vol 2021 (8) ◽  
Author(s):  
Sotaro Sugishita
Keyword(s):  
Mutual Information ◽  
Ground State ◽  
Entanglement Entropy ◽  
Algebraic Approach ◽  
Upper Bounds ◽  
Wave Functions ◽  
Target Space ◽  
Single Interval ◽  
Area Law ◽  
Formula Expression

Abstract We consider entanglement of first-quantized identical particles by adopting an algebraic approach. In particular, we investigate fermions whose wave functions are given by the Slater determinants, as for singlet sectors of one-matrix models. We show that the upper bounds of the general Rényi entropies are N log 2 for N particles or an N × N matrix. We compute the target space entanglement entropy and the mutual information in a free one-matrix model. We confirm the area law: the single-interval entropy for the ground state scales as $$ \frac{1}{3} $$ 1 3 log N in the large N model. We obtain an analytical $$ \mathcal{O}\left({N}^0\right) $$ O N 0 expression of the mutual information for two intervals in the large N expansion.

Upper bounds on the energy dissipation in turbulent precession

1996 ◽  
Vol 321 ◽  
pp. 335-370 ◽  
Author(s):  
R. R. Kerswell
Keyword(s):  
Experimental Data ◽  
Energy Dissipation ◽  
Viscous Dissipation ◽  
Upper Bound ◽  
Dissipation Rate ◽  
Oblate Spheroid ◽  
Upper Bounds ◽  
Precession Rate ◽  
Long Cylinder

Rigorous upper bounds on the viscous dissipation rate are identified for two commonly studied precessing fluid-filled configurations: an oblate spheroid and a long cylinder. The latter represents an interesting new application of the upper-bounding techniques developed by Howard and Busse. A novel ‘background’ method recently introduced by Doering & Constantin is also used to deduce in both instances an upper bound which is independent of the fluid's viscosity and the forcing precession rate. Experimental data provide some evidence that the observed viscous dissipation rate mirrors this behaviour at sufficiently high precessional forcing. Implications are then discussed for the Earth's precessional response.

scholarly journals The Essential Dimension of Stacks of Parabolic Vector Bundles over Curves

2012 ◽  
Vol 10 (3) ◽  
pp. 455-488 ◽  
Author(s):  
Indranil Biswas ◽  
Ajneet Dhillon ◽  
Nicole Lemire
Keyword(s):  
Lower Bounds ◽  
Upper Bound ◽  
Vector Bundles ◽  
Upper Bounds ◽  
Essential Dimension ◽  
Parabolic Structure ◽  
Moduli Stack

AbstractWe find upper bounds on the essential dimension of the moduli stack of parabolic vector bundles over a curve. When there is no parabolic structure, we improve the known upper bound on the essential dimension of the usual moduli stack. Our calculations also give lower bounds on the essential dimension of the semistable locus inside the moduli stack of vector bundles of rank r and degree d without parabolic structure.

ances and covariances obtained from REML are normally distributed with expectation vector and variance-covariance matrix equal to the fol-low  ing, r  espectiv    ely,   When σˆ > 0.04, let νˆ = δˆ + σˆ + σˆ − 2ωˆ + σˆ − (1 + c (7.6) be an estimate for the (7.3) reference-scaled metric in accordance with FDA Guidance (2001) and using a REML UN model. Then (Patter-son, 2003; Patterson and Jones, 2002b), this estimate is asymptotically normally distributed and unbiased with E[νˆ ] = δ +σ − (1 + c and Var[νˆ ] = 4σ + l + 4l + (1 + c ) (l )+ 2l −2(1+c − 2(1+c +4(1+c −2(1+c . Similarly, for the constant-scaled metric, when σˆ ≤ 0.04, νˆ = δˆ + σˆ + σˆ − 2ωˆ + σˆ − σˆ − 0.04(c ) (7.7) E[νˆ ] = δ +σ − 0.04(c ) Var[νˆ ] = 4σ + l + 4l + 2l − 2l − 4l + 4l − 2l . The required asymptotic upper bound √ of the 90% confidence interval can √ then be calculated as νˆ + 1.645× V̂ ar[νˆ ] or νˆ + 1.645× V̂ ar[νˆ ], where the variances are obtained by ‘plugging in’ the estimated values of the variances and covariances obtained from SAS proc mixed into the formulae for Var[νˆ ] or Var[νˆ ]. The necessary SAS code to do this is given in Appendix B. The output reveals that σˆ = 0.0714 and the upper bound is−0.060 for log(AUC). For log(Cmax), σˆ = 0.1060 and the upper bound is −0.055. As both of these upper bounds are below zero, IBE can be claimed.

2003 ◽  
pp. 367-367
Keyword(s):  
Confidence Interval ◽  
Covariance Matrix ◽  
Upper Bound ◽  
Upper Bounds ◽  
Fda Guidance ◽  
Asymptotic Upper Bound ◽  
Variance Covariance Matrix ◽  
Normally Distributed

Models of arithmetic and upper bounds for arithmetic sets

1994 ◽  
Vol 59 (3) ◽  
pp. 977-983 ◽  
Author(s):  
Alistair H. Lachlan ◽  
Robert I. Soare
Keyword(s):  
Upper Bound ◽  
Upper Bounds ◽  
Models Of Arithmetic

AbstractWe settle a question in the literature about degrees of models of true arithmetic and upper bounds for the arithmetic sets. We prove that there is a model of true arithmetic whose degree is not a uniform upper bound for the arithmetic sets. The proof involves two forcing constructions.

Sign in / Sign up

Export Citation Format

Share Document