scholarly journals Low Complexity Estimation Method of Rényi Entropy for Ergodic Sources

Entropy ◽  
2018 ◽  
Vol 20 (9) ◽  
pp. 657 ◽  
Author(s):  
Young-Sik Kim
Keyword(s):  
Estimation Method ◽  
Low Complexity ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Entropy Measure ◽  
General Representation ◽  
Iterative Estimation ◽  
Random Samples ◽  
Complexity Estimation ◽  
Special Case

Since the entropy is a popular randomness measure, there are many studies for the estimation of entropies for given random samples. In this paper, we propose an estimation method of the Rényi entropy of order α . Since the Rényi entropy of order α is a generalized entropy measure including the Shannon entropy as a special case, the proposed estimation method for Rényi entropy can detect any significant deviation of an ergodic stationary random source’s output. It is shown that the expected test value of the proposed scheme is equivalent to the Rényi entropy of order α . After deriving a general representation of parameters of the proposed estimator, we discuss on the particular orders of Rényi entropy such as α → 1 , α = 1 / 2 , and α = 2 . Because the Rényi entropy of order 2 is the most popular one, we present an iterative estimation method for the application with stringent resource restrictions.

Rényi entropy measure of noise-aided information transmission in a binary channel

2010 ◽  
Vol 81 (5) ◽  
Author(s):  
François Chapeau-Blondeau ◽  
David Rousseau ◽  
Agnès Delahaies
Keyword(s):  
Information Transmission ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Entropy Measure ◽  
Binary Channel

scholarly journals A Generalization of the Concavity of Rényi Entropy Power

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1593
Author(s):  
Laigang Guo ◽  
Chun-Ming Yuan ◽  
Xiao-Shan Gao
Keyword(s):  
Systematic Approach ◽  
Three Dimensional ◽  
Concave Function ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Nonlinear Heat Equation ◽  
Necessary And Sufficient Condition ◽  
Probability Densities ◽  
Necessary And Sufficient ◽  
Special Case

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in Rn is a concave function of time under certain conditions of three parameters n,p,μ, which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition Φ(n,p,μ) of n,p,μ under which the concavity of the Rényi entropy power is valid. The condition Φ(n,p,μ) contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points (n,p,μ) satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of R3, and the points satisfying the condition Φ(n,p,μ) consist a three-dimensional subset of R3. Furthermore, Φ(n,p,μ) gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.

scholarly journals Entropy Based Genetic Association Tests and Gene-Gene Interaction Tests

2011 ◽  
Vol 10 (1) ◽  
Author(s):  
Mariza de Andrade ◽  
Xin Wang
Keyword(s):  
Genetic Association ◽  
Shannon Entropy ◽  
Statistical Power ◽  
Genetic Model ◽  
Gene Interaction ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Entropy Measure ◽  
P Value ◽  
The Impact

In the past few years, several entropy-based tests have been proposed for testing either single SNP association or gene-gene interaction. These tests are mainly based on Shannon entropy and have higher statistical power when compared to standard χ2 tests. In this paper, we extend some of these tests using a more generalized entropy definition, Rényi entropy, where Shannon entropy is a special case of order 1. The order λ (>0) of Rényi entropy weights the events (genotype/haplotype) according to their probabilities (frequencies). Higher λ places more emphasis on higher probability events while smaller λ (close to 0) tends to assign weights more equally. Thus, by properly choosing the λ, one can potentially increase the power of the tests or the p-value level of significance. We conducted simulation as well as real data analyses to assess the impact of the order λ and the performance of these generalized tests. The results showed that for dominant model the order 2 test was more powerful and for multiplicative model the order 1 or 2 had similar power. The analyses indicate that the choice of λ depends on the underlying genetic model and Shannon entropy is not necessarily the most powerful entropy measure for constructing genetic association or interaction tests.

scholarly journals Information-Theoretical Quantifier of Brain Rhythm Based on Data-Driven Multiscale Representation

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Young-Seok Choi
Keyword(s):  
Cardiac Arrest ◽  
Data Driven ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Multiscale Entropy ◽  
Multiple Time Scales ◽  
Entropy Measure ◽  
Multiscale Representation ◽  
Brain Rhythm ◽  
Eeg Recordings

This paper presents a data-driven multiscale entropy measure to reveal the scale dependent information quantity of electroencephalogram (EEG) recordings. This work is motivated by the previous observations on the nonlinear and nonstationary nature of EEG over multiple time scales. Here, a new framework of entropy measures considering changing dynamics over multiple oscillatory scales is presented. First, to deal with nonstationarity over multiple scales, EEG recording is decomposed by applying the empirical mode decomposition (EMD) which is known to be effective for extracting the constituent narrowband components without a predetermined basis. Following calculation of Renyi entropy of the probability distributions of the intrinsic mode functions extracted by EMD leads to a data-driven multiscale Renyi entropy. To validate the performance of the proposed entropy measure, actual EEG recordings from ratsn=9experiencing 7 min cardiac arrest followed by resuscitation were analyzed. Simulation and experimental results demonstrate that the use of the multiscale Renyi entropy leads to better discriminative capability of the injury levels and improved correlations with the neurological deficit evaluation after 72 hours after cardiac arrest, thus suggesting an effective diagnostic and prognostic tool.Corrigendum to “Information-Theoretical Quantifier of Brain Rhythm Based on Data-Driven Multiscale Representation”

A Low-Complexity Sparse Channel Estimation Method for OFDM Systems

2010 ◽  
Vol E93-B (8) ◽  
pp. 2211-2214
Author(s):  
Bin SHENG ◽  
Pengcheng ZHU ◽  
Xiaohu YOU ◽  
Lan CHEN
Keyword(s):  
Channel Estimation ◽  
Estimation Method ◽  
Low Complexity ◽  
Ofdm Systems ◽  
Sparse Channel Estimation ◽  
Sparse Channel

scholarly journals Low-complexity beam-domain channel estimation and power allocation in hybrid architecture massive MIMO systems

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Xiao Chen ◽  
Zaichen Zhang ◽  
Liang Wu ◽  
Jian Dang
Keyword(s):  
Channel Estimation ◽  
Power Allocation ◽  
Spectral Efficiency ◽  
Communication Systems ◽  
Mimo Systems ◽  
Beam Steering ◽  
Estimation Method ◽  
Low Complexity ◽  
Estimation Accuracy ◽  
Hybrid Architecture

Abstract In this journal, we investigate the beam-domain channel estimation and power allocation in hybrid architecture massive multiple-input and multiple-output (MIMO) communication systems. First, we propose a low-complexity channel estimation method, which utilizes the beam steering vectors achieved from the direction-of-arrival (DOA) estimation and beam gains estimated by low-overhead pilots. Based on the estimated beam information, a purely analog precoding strategy is also designed. Then, the optimal power allocation among multiple beams is derived to maximize spectral efficiency. Finally, simulation results show that the proposed schemes can achieve high channel estimation accuracy and spectral efficiency.

scholarly journals Excited state Rényi entropy and subsystem distance in two-dimensional non-compact bosonic theory. Part I. Single-particle states

2020 ◽  
Vol 2020 (12) ◽  
Author(s):  
Jiaju Zhang ◽  
M.A. Rajabpour
Keyword(s):  
Excited States ◽  
Single Particle ◽  
Renyi Entropy ◽  
Rényi Entropy ◽  
Large Momentum ◽  
Particle State ◽  
Two Dimensional ◽  
Universal Form ◽  
Conformal Field ◽  
Harmonic Chain

Abstract We investigate the Rényi entropy of the excited states produced by the current and its derivatives in the two-dimensional free massless non-compact bosonic theory, which is a two-dimensional conformal field theory. We also study the subsystem Schatten distance between these states. The two-dimensional free massless non-compact bosonic theory is the continuum limit of the finite periodic gapless harmonic chains with the local interactions. We identify the excited states produced by current and its derivatives in the massless bosonic theory as the single-particle excited states in the gapless harmonic chain. We calculate analytically the second Rényi entropy and the second Schatten distance in the massless bosonic theory. We then use the wave functions of the excited states and calculate the second Rényi entropy and the second Schatten distance in the gapless limit of the harmonic chain, which match perfectly with the analytical results in the massless bosonic theory. We verify that in the large momentum limit the single-particle state Rényi entropy takes a universal form. We also show that in the limit of large momenta and large momentum difference the subsystem Schatten distance takes a universal form but it is replaced by a new corrected form when the momentum difference is small. Finally we also comment on the mutual Rényi entropy of two disjoint intervals in the excited states of the two-dimensional free non-compact bosonic theory.

scholarly journals A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria

Mathematics ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 700
Author(s):  
Belén Pérez-Sánchez ◽  
Martín González ◽  
Carmen Perea ◽  
Jose J. López-Espín
Keyword(s):  
Experimental Study ◽  
Computational Study ◽  
Estimation Method ◽  
Information Criteria ◽  
Simultaneous Equations ◽  
Estimation Methods ◽  
Entropy Measure ◽  
Economic Science ◽  
Akaike Information Criteria ◽  
Simultaneous Equations Models

Simultaneous Equations Models (SEM) is a statistical technique widely used in economic science to model the simultaneity relationship between variables. In the past years, this technique has also been used in other fields such as psychology or medicine. Thus, the development of new estimating methods is an important line of research. In fact, if we want to apply the SEM to medical problems with the main goal being to obtain the best approximation between the parameters of model and their estimations. This paper shows a computational study between different methods for estimating simultaneous equations models as well as a new method which allows the estimation of those parameters based on the optimization of the Bayesian Method of Moments and minimizing the Akaike Information Criteria. In addition, an entropy measure has been calculated as a parameter criteria to compare the estimation methods studied. The comparison between those methods is performed through an experimental study using randomly generated models. The experimental study compares the estimations obtained by the different methods as well as the efficiency when comparing solutions by Akaike Information Criteria and Entropy Measure. The study shows that the proposed estimation method offered better approximations and the entropy measured results more efficiently than the rest.

Sign in / Sign up

Export Citation Format

Share Document