scholarly journals Understanding and Predicting Nonlinear Turbulent Dynamical Systems with Information Theory

Atmosphere ◽  
2019 ◽  
Vol 10 (5) ◽  
pp. 248
Author(s):  
Nan Chen ◽  
Xiao Hou ◽  
Qin Li ◽  
Yingda Li
Keyword(s):  
Information Theory ◽  
Dynamical Systems ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Model Error ◽  
High Dimensions ◽  
New Approach ◽  
Fluctuation Dissipation ◽  
Spectrum Density ◽  
Non Gaussian

Complex nonlinear turbulent dynamical systems are ubiquitous in many areas. Quantifying the model error and model uncertainty plays an important role in understanding and predicting complex dynamical systems. In the first part of this article, a simple information criterion is developed to assess the model error in imperfect models. This effective information criterion takes into account the information in both the equilibrium statistics and the temporal autocorrelation function, where the latter is written in the form of the spectrum density that permits the quantification via information theory. This information criterion facilitates the study of model reduction, stochastic parameterizations, and intermittent events. In the second part of this article, a new efficient method is developed to improve the computation of the linear response via the Fluctuation Dissipation Theorem (FDT). This new approach makes use of a Gaussian Mixture (GM) to describe the unperturbed probability density function in high dimensions and avoids utilizing Gaussian approximations in computing the statistical response, as is widely used in the quasi-Gaussian (qG) FDT. Testing examples show that this GM FDT outperforms qG FDT in various strong non-Gaussian regimes.

scholarly journals Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part II: Applications

2013 ◽  
Vol 141 (6) ◽  
pp. 1761-1785 ◽  
Author(s):  
Thomas Sondergaard ◽  
Pierre F. J. Lermusiaux
Keyword(s):  
Gaussian Mixture Models ◽  
Mean Field ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Sudden Expansion ◽  
Test Case ◽  
Field Equations ◽  
Gaussian Statistics ◽  
Non Gaussian ◽  
Orthogonal Field

Abstract The properties and capabilities of the Gaussian Mixture Model–Dynamically Orthogonal filter (GMM-DO) are assessed and exemplified by applications to two dynamical systems: 1) the double well diffusion and 2) sudden expansion flows; both of which admit far-from-Gaussian statistics. The former test case, or twin experiment, validates the use of the Expectation-Maximization (EM) algorithm and Bayesian Information Criterion with GMMs in a filtering context; the latter further exemplifies its ability to efficiently handle state vectors of nontrivial dimensionality and dynamics with jets and eddies. For each test case, qualitative and quantitative comparisons are made with contemporary filters. The sensitivity to input parameters is illustrated and discussed. Properties of the filter are examined and its estimates are described, including the equation-based and adaptive prediction of the probability densities; the evolution of the mean field, stochastic subspace modes, and stochastic coefficients; the fitting of GMMs; and the efficient and analytical Bayesian updates at assimilation times and the corresponding data impacts. The advantages of respecting nonlinear dynamics and preserving non-Gaussian statistics are brought to light. For realistic test cases admitting complex distributions and with sparse or noisy measurements, the GMM-DO filter is shown to fundamentally improve the filtering skill, outperforming simpler schemes invoking the Gaussian parametric distribution.

scholarly journals Detecting multiple generalized change-points by isolating single ones

Metrika ◽  
2021 ◽  
Author(s):  
Andreas Anastasiou ◽  
Piotr Fryzlewicz
Keyword(s):  
Model Selection ◽  
Linear Trend ◽  
Information Criterion ◽  
Change Points ◽  
Isolation Technique ◽  
New Approach ◽  
Piecewise Constant ◽  
Practical Performance ◽  
R Packages ◽  
The Times

AbstractWe introduce a new approach, called Isolate-Detect (ID), for the consistent estimation of the number and location of multiple generalized change-points in noisy data sequences. Examples of signal changes that ID can deal with are changes in the mean of a piecewise-constant signal and changes, continuous or not, in the linear trend. The number of change-points can increase with the sample size. Our method is based on an isolation technique, which prevents the consideration of intervals that contain more than one change-point. This isolation enhances ID’s accuracy as it allows for detection in the presence of frequent changes of possibly small magnitudes. In ID, model selection is carried out via thresholding, or an information criterion, or SDLL, or a hybrid involving the former two. The hybrid model selection leads to a general method with very good practical performance and minimal parameter choice. In the scenarios tested, ID is at least as accurate as the state-of-the-art methods; most of the times it outperforms them. ID is implemented in the R packages IDetect and breakfast, available from CRAN.

Pseudorandom Number Generators Based on Chaotic Dynamical Systems

2001 ◽  
Vol 08 (02) ◽  
pp. 137-146 ◽  
Author(s):  
Janusz Szczepański ◽  
Zbigniew Kotulski
Keyword(s):  
Dynamical Systems ◽  
Communication Systems ◽  
Initial Conditions ◽  
Pseudorandom Number ◽  
New Approach ◽  
Chaotic Dynamical Systems ◽  
Pseudorandom Number Generators ◽  
Transmission Of Information ◽  
Extreme Sensitivity ◽  
Contemporary Technology

Pseudorandom number generators are used in many areas of contemporary technology such as modern communication systems and engineering applications. In recent years a new approach to secure transmission of information based on the application of the theory of chaotic dynamical systems has been developed. In this paper we present a method of generating pseudorandom numbers applying discrete chaotic dynamical systems. The idea of construction of chaotic pseudorandom number generators (CPRNG) intrinsically exploits the property of extreme sensitivity of trajectories to small changes of initial conditions, since the generated bits are associated with trajectories in an appropriate way. To ensure good statistical properties of the CPRBG (which determine its quality) we assume that the dynamical systems used are also ergodic or preferably mixing. Finally, since chaotic systems often appear in realistic physical situations, we suggest a physical model of CPRNG.

scholarly journals Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part I: Theory and Scheme

2013 ◽  
Vol 141 (6) ◽  
pp. 1737-1760 ◽  
Author(s):  
Thomas Sondergaard ◽  
Pierre F. J. Lermusiaux
Keyword(s):  
Data Assimilation ◽  
Mixture Models ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Planck Equation ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Field Equations ◽  
Dynamical Equations ◽  
Prior Probabilities

Abstract This work introduces and derives an efficient, data-driven assimilation scheme, focused on a time-dependent stochastic subspace that respects nonlinear dynamics and captures non-Gaussian statistics as it occurs. The motivation is to obtain a filter that is applicable to realistic geophysical applications, but that also rigorously utilizes the governing dynamical equations with information theory and learning theory for efficient Bayesian data assimilation. Building on the foundations of classical filters, the underlying theory and algorithmic implementation of the new filter are developed and derived. The stochastic Dynamically Orthogonal (DO) field equations and their adaptive stochastic subspace are employed to predict prior probabilities for the full dynamical state, effectively approximating the Fokker–Planck equation. At assimilation times, the DO realizations are fit to semiparametric Gaussian Mixture Models (GMMs) using the Expectation-Maximization algorithm and the Bayesian Information Criterion. Bayes’s law is then efficiently carried out analytically within the evolving stochastic subspace. The resulting GMM-DO filter is illustrated in a very simple example. Variations of the GMM-DO filter are also provided along with comparisons with related schemes.

FREQUENCY SYNCHRONIZATION IN NETWORKS OF COUPLED OSCILLATORS, A MONOTONE DYNAMICAL SYSTEMS APPROACH

2009 ◽  
Vol 19 (12) ◽  
pp. 4107-4116 ◽  
Author(s):  
WEN-XIN QIN
Keyword(s):  
Dynamical Systems ◽  
Coupling Strength ◽  
Coupled Oscillators ◽  
Systems Approach ◽  
System Size ◽  
Coupling Scheme ◽  
Frequency Synchronization ◽  
Nonlinear Coupling ◽  
New Approach ◽  
Monotone Dynamical Systems

We propose a new approach to investigate the frequency synchronization in networks of coupled oscillators. By making use of the theory of monotone dynamical systems, we show that frequency synchronization occurs in networks of coupled oscillators, provided the coupling scheme is symmetric, connected, and strongly cooperative. Our criterion is independent of the system size, the coupling strength and the details of the connections, and applies also to nonlinear coupling schemes.

scholarly journals Nonmonotonic Generalization Bias of Gaussian Mixture Models

2000 ◽  
Vol 12 (6) ◽  
pp. 1411-1427 ◽  
Author(s):  
Shotaro Akaho ◽  
Hilbert J. Kappen
Keyword(s):  
Symmetry Breaking ◽  
Gaussian Mixture Models ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
General Tendency ◽  
Adaptive Parameters ◽  
Neural Information ◽  
The Difference ◽  
Training Error ◽  
Validation Experiments

Theories of learning and generalization hold that the generalization bias, defined as the difference between the training error and the generalization error, increases on average with the number of adaptive parameters. This article, however, shows that this general tendency is violated for a gaussian mixture model. For temperatures just below the first symmetry breaking point, the effective number of adaptive parameters increases and the generalization bias decreases. We compute the dependence of the neural information criterion on temperature around the symmetry breaking. Our results are confirmed by numerical cross-validation experiments.

scholarly journals A sieve algorithm based on overlattices

2014 ◽  
Vol 17 (A) ◽  
pp. 49-70 ◽  
Author(s):  
Anja Becker ◽  
Nicolas Gama ◽  
Antoine Joux
Keyword(s):  
Heuristic Algorithm ◽  
Orthonormal Basis ◽  
Complexity Analysis ◽  
Computer Architectures ◽  
High Dimensions ◽  
Average Case ◽  
Solution Vector ◽  
New Approach ◽  
Bounded Norm

AbstractIn this paper, we present a heuristic algorithm for solving exact, as well as approximate, shortest vector and closest vector problems on lattices. The algorithm can be seen as a modified sieving algorithm for which the vectors of the intermediate sets lie in overlattices or translated cosets of overlattices. The key idea is hence no longer to work with a single lattice but to move the problems around in a tower of related lattices. We initiate the algorithm by sampling very short vectors in an overlattice of the original lattice that admits a quasi-orthonormal basis and hence an efficient enumeration of vectors of bounded norm. Taking sums of vectors in the sample, we construct short vectors in the next lattice. Finally, we obtain solution vector(s) in the initial lattice as a sum of vectors of an overlattice. The complexity analysis relies on the Gaussian heuristic. This heuristic is backed by experiments in low and high dimensions that closely reflect these estimates when solving hard lattice problems in the average case.This new approach allows us to solve not only shortest vector problems, but also closest vector problems, in lattices of dimension$\def \xmlpi #1{}\def \mathsfbi #1{\boldsymbol {\mathsf {#1}}}\let \le =\leqslant \let \leq =\leqslant \let \ge =\geqslant \let \geq =\geqslant \def \Pr {\mathit {Pr}}\def \Fr {\mathit {Fr}}\def \Rey {\mathit {Re}}n$in time$2^{0.3774\, n}$using memory$2^{0.2925\, n}$. Moreover, the algorithm is straightforward to parallelize on most computer architectures.

An Alternative Information Theory Approach for Modelling Spatial Interaction

1987 ◽  
Vol 19 (3) ◽  
pp. 385-394 ◽  
Author(s):  
J R Roy
Keyword(s):  
Information Theory ◽  
Shannon Entropy ◽  
Information Gain ◽  
Spatial Interaction ◽  
Theory Approach ◽  
Combined Approach ◽  
New Approach ◽  
Forecasting Models ◽  
Kullback Information ◽  
Alternative Approaches

In the use of information theory for the development of forecasting models, two alternative approaches can be used, based either on Shannon entropy or on Kullback information gain. In this paper, a new approach is presented, which combines the usually superior statistical inference powers of the Kullback procedure with the advantages of the availability of calibrated ‘elasticity’ parameters in the Shannon approach. Situations are discussed where the combined approach is preferable to either of the two existing procedures, and the principles are illustrated with the help of a small numerical example.

Sign in / Sign up

Export Citation Format

Share Document