scholarly journals Information Theory and Symbolic Analysis: Theory and Applications

Entropy ◽  
2021 ◽  
Vol 23 (10) ◽  
pp. 1361
Author(s):  
Mariano Matilla-García ◽  
Manuel Ruiz Marín
Keyword(s):  
Information Theory ◽  
Recent Literature ◽  
Complex Dynamics ◽  
Symbolic Analysis ◽  
Permutation Entropy ◽  
Network Structures ◽  
Correlation Integral ◽  
Analysis Theory ◽  
Entropy Measures

Symbolic analysis has been developed and used successfully in very diverse fields. In recent literature, the contributions of symbolic analysis to the study of complex dynamics and network structures are easy to find, especially those based on symbolic entropy measures (such as permutation entropy) and the symbolic correlation integral (connected with Renyi and Tsallis entropies)[...]

scholarly journals Employing entropy measures to identify visitors in multi-occupancy environments

2020 ◽  
Author(s):  
Aadel Howedi ◽  
Ahmad Lotfi ◽  
Amir Pourabdollah
Keyword(s):  
Home Environment ◽  
Daily Living ◽  
Wearable Sensors ◽  
Permutation Entropy ◽  
Care Workers ◽  
Entropy Measures ◽  
Novel Method ◽  
High Degree ◽  
Motion Detectors ◽  
To Receive

AbstractHuman activity recognition (HAR) is used to support older adults to live independently in their own homes. Once activities of daily living (ADL) are recognised, gathered information will be used to identify abnormalities in comparison with the routine activities. Ambient sensors, including occupancy sensors and door entry sensors, are often used to monitor and identify different activities. Most of the current research in HAR focuses on a single-occupant environment when only one person is monitored, and their activities are categorised. The assumption that home environments are occupied by one person all the time is often not true. It is common for a resident to receive visits from family members or health care workers, representing a multi-occupancy environment. Entropy analysis is an established method for irregularity detection in many applications; however, it has been rarely applied in the context of ADL and HAR. In this paper, a novel method based on different entropy measures, including Shannon Entropy, Permutation Entropy, and Multiscale-Permutation Entropy, is employed to investigate the effectiveness of these entropy measures in identifying visitors in a home environment. This research aims to investigate whether entropy measures can be utilised to identify a visitor in a home environment, solely based on the information collected from motion detectors [e.g., passive infra-red] and door entry sensors. The entropy measures are tested and evaluated based on a dataset gathered from a real home environment. Experimental results are presented to show the effectiveness of entropy measures to identify visitors and the time of their visits without the need for employing extra wearable sensors to tag the visitors. The results obtained from the experiments show that the proposed entropy measures could be used to detect and identify a visitor in a home environment with a high degree of accuracy.

scholarly journals A review of entropy measures for uncertainty quantification of stochastic processes

2019 ◽  
Vol 11 (6) ◽  
pp. 168781401985735 ◽  
Author(s):  
Alireza Namdari ◽  
Zhaojun (Steven) Li
Keyword(s):  
Stochastic Process ◽  
Stochastic Processes ◽  
Uncertainty Quantification ◽  
Lithium Ion ◽  
Second Law Of Thermodynamics ◽  
Approximate Entropy ◽  
Sample Entropy ◽  
Permutation Entropy ◽  
Battery Capacity ◽  
Entropy Measures

Entropy is originally introduced to explain the inclination of intensity of heat, pressure, and density to gradually disappear over time. Based on the concept of entropy, the Second Law of Thermodynamics, which states that the entropy of an isolated system is likely to increase until it attains its equilibrium state, is developed. More recently, the implication of entropy has been extended beyond the field of thermodynamics, and entropy has been applied in many subjects with probabilistic nature. The concept of entropy is applicable and useful in characterizing the behavior of stochastic processes since it represents the uncertainty, ambiguity, and disorder of the processes without being restricted to the forms of the theoretical probability distributions. In order to measure and quantify the entropy, the existing probability of every event in the stochastic process must be determined. Different entropy measures have been studied and presented including Shannon entropy, Renyi entropy, Tsallis entropy, Sample entropy, Permutation entropy, Approximate entropy, and Transfer entropy. This review surveys the general formulations of the uncertainty quantification based on entropy as well as their various applications. The results of the existing studies show that entropy measures are powerful predictors for stochastic processes with uncertainties. In addition, we examine the stochastic process of lithium-ion battery capacity data and attempt to determine the relation between the changes in battery capacity over different cycles and two entropy measures: Sample entropy and Approximate entropy.

Causality and information transfer in systems with extreme events

2020 ◽  
Author(s):  
Milan Palus
Keyword(s):  
Information Theory ◽  
Extreme Events ◽  
Information Transfer ◽  
Complex Dynamics ◽  
Probability Distributions ◽  
Mathematical Formulation ◽  
Multiple Time Scales ◽  
Multiple Time ◽  
Theoretic Approach ◽  
Heavy Tailed

<p>The mathematical formulation of causality in measurable terms of predictability was given by the father of cybernetics N. Wiener [1] and formulated for time series by C.W.J. Granger [2]. The Granger causality is based on the evaluation of predictability in bivariate autoregressive models. This concept has been generalized for nonlinear systems using methods rooted in information theory [3,4]. The information-theoretic approach, defining causality as information transfer, has been successful in many applications and generalized to multivariate data and causal networks [e.g., 5]. This approach, rooted in the information theory of Shannon, usually ignores two important properties of complex systems, such as the Earth climate: the systems evolve on multiple time scales and their variables have heavy-tailed probability distributions. While the multiscale character of complex dynamics, such as air temperature variability, can be studied within the Shannonian framework [6, 7], the entropy concepts of Rényi and Tsallis have been proposed to cope with variables with heavy-tailed probability distributions. We will discuss how such non-Shannonian entropy concepts can be applied in inference of causality in systems with heavy-tailed probability distributions and extreme events, using examples from the climate system.</p><p>This study was supported by the Czech Science Foundation, project GA19-16066S.</p><p> </p><p> [1] N. Wiener, in: E. F. Beckenbach (Editor), Modern Mathematics for Engineers (McGraw-Hill, New York, 1956)</p><p>[2] C.W.J. Granger, Econometrica 37 (1969) 424</p><p>[3] K. Hlaváčková-Schindler et al., Phys. Rep. 441 (2007)  1</p><p>[4] M. Paluš, M. Vejmelka, Phys. Rev. E 75 (2007) 056211</p><p>[5] J. Runge et al., Nature Communications 6 (2015) 8502</p><p>[6] M. Paluš, Phys. Rev. Lett. 112 (2014) 078702</p><p> [7] N. Jajcay, J. Hlinka, S. Kravtsov, A. A. Tsonis, M. Paluš, Geophys. Res. Lett. 43(2) (2016) 902–909</p>

scholarly journals Adaptive Neuro-Fuzzy Inference System for Classification of Background EEG Signals from ESES Patients and Controls

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Zhixian Yang ◽  
Yinghua Wang ◽  
Gaoxiang Ouyang
Keyword(s):  
Fuzzy Inference System ◽  
Slow Wave Sleep ◽  
Fuzzy Inference ◽  
Permutation Entropy ◽  
Eeg Signals ◽  
Inference System ◽  
Control Subjects ◽  
Neuro Fuzzy ◽  
Entropy Measures ◽  
Background Eeg

Background electroencephalography (EEG), recorded with scalp electrodes, in children with electrical status epilepticus during slow-wave sleep (ESES) syndrome and control subjects has been analyzed. We considered 10 ESES patients, all right-handed and aged 3–9 years. The 10 control individuals had the same characteristics of the ESES ones but presented a normal EEG. Recordings were undertaken in the awake and relaxed states with their eyes open. The complexity of background EEG was evaluated using the permutation entropy (PE) and sample entropy (SampEn) in combination with the ANOVA test. It can be seen that the entropy measures of EEG are significantly different between the ESES patients and normal control subjects. Then, a classification framework based on entropy measures and adaptive neuro-fuzzy inference system (ANFIS) classifier is proposed to distinguish ESES and normal EEG signals. The results are promising and a classification accuracy of about 89% is achieved.

HARMONIC BALANCE APPROACH TO PREDICT PERIOD-DOUBLING BIFURCATIONS IN NEARLY SYMMETRIC CNNs

2003 ◽  
Vol 12 (04) ◽  
pp. 435-459 ◽  
Author(s):  
MAURO DI MARCO ◽  
MAURO FORTI ◽  
ALBERTO TESI
Keyword(s):  
Neural Networks ◽  
Computer Simulations ◽  
Harmonic Balance ◽  
Recent Literature ◽  
Complex Dynamics ◽  
Period Doubling ◽  
Third Order ◽  
Small Perturbations ◽  
Basic Issue ◽  
Period Doubling Bifurcations

This paper further investigates a basic issue that has received attention in the recent literature, namely, the robustness of complete stability of standard Cellular Neural Networks (CNNs) with respect to small perturbations of the nominal symmetric interconnections. More specifically, a class of third-order CNNs with a nominal symmetric interconnection matrix is considered, and the Harmonic Balance (HB) method is exploited for addressing the possible existence of period-doubling bifurcations, and complex dynamics, for small perturbations of the nominal interconnections. The main result is that there are indeed parameter sets close to symmetry for which period-doubling bifurcations are predicted by the HB method. Moreover, the predictions are found to be reliable and accurate by means of computer simulations.

scholarly journals Alternative Entropy Measures and Generalized Khinchin–Shannon Inequalities

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1618
Author(s):  
Rubem P. Mondaini ◽  
Simão C. de Albuquerque Neto
Keyword(s):  
Information Theory ◽  
Algebraic Structure ◽  
Physical Systems ◽  
Special Cases ◽  
Entropy Measures ◽  
The Rich ◽  
Two Parameter

The Khinchin–Shannon generalized inequalities for entropy measures in Information Theory, are a paradigm which can be used to test the Synergy of the distributions of probabilities of occurrence in physical systems. The rich algebraic structure associated with the introduction of escort probabilities seems to be essential for deriving these inequalities for the two-parameter Sharma–Mittal set of entropy measures. We also emphasize the derivation of these inequalities for the special cases of one-parameter Havrda–Charvat’s, Rényi’s and Landsberg–Vedral’s entropy measures.

scholarly journals Further improvements of some bounds on entropy measures in information theory

1999 ◽  
pp. 599-611
Author(s):  
M. Matić ◽  
Charles E. M. Pearce ◽  
Josip Pečarić
Keyword(s):  
Information Theory ◽  
Entropy Measures

scholarly journals 𝑯𝑵- Entropy: A New Measureof Information And Its Properties

2022 ◽  
Vol 24 (1) ◽  
pp. 105-118
Author(s):  
Mervat Mahdy ◽  
◽  
Dina S. Eltelbany ◽  
Hoda Mohammed ◽  
◽  
...  
Keyword(s):  
Information Theory ◽  
Shannon Entropy ◽  
Basic Concept ◽  
Random Quantity ◽  
Numerical Results ◽  
Alternative Measure ◽  
Weighted Version ◽  
Amount Of Information ◽  
Entropy Measures ◽  
Cumulative Residual

Entropy measures the amount of uncertainty and dispersion of an unknown or random quantity, this concept introduced at first by Shannon (1948), it is important for studies in many areas. Like, information theory: entropy measures the amount of information in each message received, physics: entropy is the basic concept that measures the disorder of the thermodynamical system, and others. Then, in this paper, we introduce an alternative measure of entropy, called 𝐻𝑁- entropy, unlike Shannon entropy, this proposed measure of order α and β is more flexible than Shannon. Then, the cumulative residual 𝐻𝑁- entropy, cumulative 𝐻𝑁- entropy, and weighted version have been introduced. Finally, comparison between Shannon entropy and 𝐻𝑁- entropy and numerical results have been introduced.

scholarly journals Entropy Measures for Data Analysis: Theory, Algorithms and Applications

Entropy ◽  
2019 ◽  
Vol 21 (10) ◽  
pp. 935
Author(s):  
Karsten Keller
Keyword(s):  
Data Analysis ◽  
Analysis Theory ◽  
Entropy Measures ◽  
Non Linear

Entropies and entropy-like quantities are playing an increasing role in modern non-linear data analysis and beyond [...]

Sign in / Sign up

Export Citation Format

Share Document