scholarly journals What Can Local Transfer Entropy Tell Us about Phase-Amplitude Coupling in Electrophysiological Signals?

Entropy ◽  
2020 ◽  
Vol 22 (11) ◽  
pp. 1262
Author(s):  
Ramón Martínez-Cancino ◽  
Arnaud Delorme ◽  
Johanna Wagner ◽  
Kenneth Kreutz-Delgado ◽  
Roberto C. Sotero ◽  
...  
Keyword(s):  
Mutual Information ◽  
Time Course ◽  
Mammalian Species ◽  
Transfer Entropy ◽  
Information Theoretic ◽  
Process Dynamics ◽  
Brain Rhythm ◽  
Technical Issues ◽  
Phase Amplitude ◽  
Local Transfer

Modulation of the amplitude of high-frequency cortical field activity locked to changes in the phase of a slower brain rhythm is known as phase-amplitude coupling (PAC). The study of this phenomenon has been gaining traction in neuroscience because of several reports on its appearance in normal and pathological brain processes in humans as well as across different mammalian species. This has led to the suggestion that PAC may be an intrinsic brain process that facilitates brain inter-area communication across different spatiotemporal scales. Several methods have been proposed to measure the PAC process, but few of these enable detailed study of its time course. It appears that no studies have reported details of PAC dynamics including its possible directional delay characteristic. Here, we study and characterize the use of a novel information theoretic measure that may address this limitation: local transfer entropy. We use both simulated and actual intracranial electroencephalographic data. In both cases, we observe initial indications that local transfer entropy can be used to detect the onset and offset of modulation process periods revealed by mutual information estimated phase-amplitude coupling (MIPAC). We review our results in the context of current theories about PAC in brain electrical activity, and discuss technical issues that must be addressed to see local transfer entropy more widely applied to PAC analysis. The current work sets the foundations for further use of local transfer entropy for estimating PAC process dynamics, and extends and complements our previous work on using local mutual information to compute PAC (MIPAC).

scholarly journals What Can Local Transfer Entropy Tell us About Phase-amplitude Coupling in Electrophysiological Signals?

2020 ◽  
Author(s):  
Ramón Martínez-Cancino ◽  
Arnaud Delorme ◽  
Johanna Wagner ◽  
Kenneth Kreutz-Delgado ◽  
Roberto C. Sotero ◽  
...  
Keyword(s):  
Time Course ◽  
Mammalian Species ◽  
Transfer Entropy ◽  
Information Theoretic ◽  
Field Activity ◽  
Electrophysiological Signals ◽  
Brain Rhythm ◽  
Technical Issues ◽  
Phase Amplitude ◽  
Local Transfer

Modulation of the amplitude of high-frequency cortical field activity locked to changes in phase of a slower brain rhythm is known as phase-amplitude coupling (PAC). The study of this phenomenon has been gaining traction in neuroscience because of several reports on its appearance in normal and pathological brain processes in humans as well as across different mammalian species. This has led to the suggestion that PAC may be an intrinsic brain process that facilitates brain inter-area communication across different spatiotemporal scales. Several methods have been proposed to measure the PAC process, but few of these enable detailed study of its time course. It appears that no studies have reported details of PAC dynamics including its possible directional delay characteristics. Here, we study and characterize the use of a novel information theoretic measure that may address this limitation: local transfer entropy. We use both simulated and actual intracranial electroencephalographic data, and in both cases we observe initial indications that local transfer entropy can be used to detect the onset and offset of modulation process periods revealed by mutual information phase-amplitude coupling (MIPAC). We review our results in the context of current theories about PAC in brain electrical activity, and discuss technical issues that must be addressed to see local transfer entropy more widely applied to PAC analysis.

Interaction Between Forex and Stock Markets in India: An Entropy Approach

2008 ◽  
Vol 33 (4) ◽  
pp. 27-46 ◽  
Author(s):  
Y V Reddy ◽  
A Sebastin
Keyword(s):  
Time Series ◽  
Mutual Information ◽  
Stock Market ◽  
Transfer Entropy ◽  
The Other ◽  
Stock Index ◽  
Reverse Direction ◽  
Probability Mass Function ◽  
Information Theoretic ◽  
The Relationship

Interactions between the foreign exchange market and the stock market of a country are considered to be an important internal force of the markets in a financially liberalized environment. If causal relationship from a market to the other is not detected, then informational efficiency exists in the other whereas existence of causality implies that hedging of exposure to one market by taking position in the other market will be effective. The temporal relationship between the forex market and the stock market of developing and developed countries has been studied, especially after the East Asian financial crisis of 1997–98, using various methods like cross-correlation, cross-spectrum, and error correction model, but these methods identify only linear relations. A statistically rigorous approach to the detection of interdependence, including non-linear dynamic relationships, between time series is provided by tools defined using the information theoretic concept of entropy. Entropy is the amount of disorder in the system and also is the amount of information needed to predict the next measurement with a certain precision. The mutual information between two random variables X and Y with a joint probability mass function p(x,y) and marginal mass functions p(x) and p(y), is defined as the relative entropy between the joint distribution p(x,y) and the product distribution p(x)*p(y). Mutual information is the reduction in the uncertainty of X due to the knowledge of Y and vice versa. Since mutual information measures the deviation from independence of the variables, it has been proposed as a tool to measure the relationship between financial market segments. However, mutual information is a symmetric measure and does not contain either dynamic information or directional sense. Even time delayed mutual information does not distinguish information actually exchanged from shared information due to a common input signal or history and therefore does not quantify the actual overlap of the information content of two variables. Another information theoretic measure called transfer entropy has been introduced by Thomas Schreiber (2000) to study the relationship between dynamic systems; the concept has also been applied by some authors to study the causal structure between financial time series. In this paper, an attempt has been made to study the interaction between the stock and the forex markets in India by computing transfer entropy between daily data series of the 50 stock index of the National Stock Exchange of India Limited, viz., Nifty and the exchange rate of Indian Rupee vis- à- vis US Dollar, viz., Reserve Bank of India reference rate. The entire period–November 1995 to March 2007–selected for the study, has been divided into three sub-periods for the purpose of analysis, considering the developments that took place during these sub-periods. The results obtained reveal that: there exist only low level interactions between the stock and the forex markets of India at a time scale of a day or less, although theory suggests interactive relationship between the two markets the flow from the stock market to the forex market is more pronounced than the flow in the reverse direction.

scholarly journals Statistical Approaches for the Analysis of Dependency Among Neurons Under Noise

Entropy ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. 387 ◽  
Author(s):  
Deniz Gençağa ◽  
Sevgi Şengül Ayan ◽  
Hajar Farnoudkia ◽  
Serdar Okuyucu
Keyword(s):  
Mutual Information ◽  
Additive Noise ◽  
Transfer Entropy ◽  
Observation Data ◽  
Noise Levels ◽  
Information Theoretic ◽  
Statistical Approaches

Neuronal noise is a major factor affecting the communication between coupled neurons. In this work, we propose a statistical toolset to infer the coupling between two neurons under noise. We estimate these statistical dependencies from data which are generated by a coupled Hodgkin–Huxley (HH) model with additive noise. To infer the coupling using observation data, we employ copulas and information-theoretic quantities, such as the mutual information (MI) and the transfer entropy (TE). Copulas and MI between two variables are symmetric quantities, whereas TE is asymmetric. We demonstrate the performances of copulas and MI as functions of different noise levels and show that they are effective in the identification of the interactions due to coupling and noise. Moreover, we analyze the inference of TE values between neurons as a function of noise and conclude that TE is an effective tool for finding out the direction of coupling between neurons under the effects of noise.

scholarly journals Comparative performance of mutual information and transfer entropy for analyzing the balance of information flow and energy consumption at synapses

2020 ◽  
Author(s):  
Mireille Conrad ◽  
Renaud B Jolivet
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Information Flow ◽  
Experimental Finding ◽  
Brain Regions ◽  
Transfer Entropy ◽  
Information Flows ◽  
Experimental Conditions ◽  
Information Theoretic ◽  
Thalamic Relay

AbstractInformation theory has become an essential tool of modern neuroscience. It can however be difficult to apply in experimental contexts when acquisition of very large datasets is prohibitive. Here, we compare the relative performance of two information theoretic measures, mutual information and transfer entropy, for the analysis of information flow and energetic consumption at synapses. We show that transfer entropy outperforms mutual information in terms of reliability of estimates for small datasets. However, we also show that a detailed understanding of the underlying neuronal biophysics is essential for properly interpreting the results obtained with transfer entropy. We conclude that when time and experimental conditions permit, mutual information might provide an easier to interpret alternative. Finally, we apply both measures to the study of energetic optimality of information flow at thalamic relay synapses in the visual pathway. We show that both measures recapitulate the experimental finding that these synapses are tuned to optimally balance information flowing through them with the energetic consumption associated with that synaptic and neuronal activity. Our results highlight the importance of conducting systematic computational studies prior to applying information theoretic tools to experimental data.Author summaryInformation theory has become an essential tool of modern neuroscience. It is being routinely used to evaluate how much information flows from external stimuli to various brain regions or individual neurons. It is also used to evaluate how information flows between brain regions, between neurons, across synapses, or in neural networks. Information theory offers multiple measures to do that. Two of the most popular are mutual information and transfer entropy. While these measures are related to each other, they differ in one important aspect: transfer entropy reports a directional flow of information, as mutual information does not. Here, we proceed to a systematic evaluation of their respective performances and trade-offs from the perspective of an experimentalist looking to apply these measures to binarized spike trains. We show that transfer entropy might be a better choice than mutual information when time for experimental data collection is limited, as it appears less affected by systematic biases induced by a relative lack of data. Transmission delays and integration properties of the output neuron can however complicate this picture, and we provide an example of the effect this has on both measures. We conclude that when time and experimental conditions permit, mutual information – especially when estimated using a method referred to as the ‘direct’ method – might provide an easier to interpret alternative. Finally, we apply both measures in the biophysical context of evaluating the energetic optimality of information flow at thalamic relay synapses in the visual pathway. We show that both measures capture the original experimental finding that those synapses are tuned to optimally balance information flowing through them with the concomitant energetic consumption associated with that synaptic and neuronal activity.

scholarly journals Do galactic bars depend on environment?: an information theoretic analysis of Galaxy Zoo 2

2020 ◽  
Vol 501 (1) ◽  
pp. 994-1001
Author(s):  
Suman Sarkar ◽  
Biswajit Pandey ◽  
Snehasish Bhattacharjee
Keyword(s):  
Spatial Distribution ◽  
Mutual Information ◽  
Local Density ◽  
Statistical Significance ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Host Galaxy ◽  
Data Sets ◽  
Data Set ◽  
Information Theoretic

ABSTRACT We use an information theoretic framework to analyse data from the Galaxy Zoo 2 project and study if there are any statistically significant correlations between the presence of bars in spiral galaxies and their environment. We measure the mutual information between the barredness of galaxies and their environments in a volume limited sample (Mr ≤ −21) and compare it with the same in data sets where (i) the bar/unbar classifications are randomized and (ii) the spatial distribution of galaxies are shuffled on different length scales. We assess the statistical significance of the differences in the mutual information using a t-test and find that both randomization of morphological classifications and shuffling of spatial distribution do not alter the mutual information in a statistically significant way. The non-zero mutual information between the barredness and environment arises due to the finite and discrete nature of the data set that can be entirely explained by mock Poisson distributions. We also separately compare the cumulative distribution functions of the barred and unbarred galaxies as a function of their local density. Using a Kolmogorov–Smirnov test, we find that the null hypothesis cannot be rejected even at $75{{\ \rm per\ cent}}$ confidence level. Our analysis indicates that environments do not play a significant role in the formation of a bar, which is largely determined by the internal processes of the host galaxy.

scholarly journals Role of Mammalian Auditory Cortex in the Perception of Elementary Sound Properties

2001 ◽  
Vol 85 (6) ◽  
pp. 2350-2358 ◽  
Author(s):  
Sanjiv K. Talwar ◽  
Pawel G. Musial ◽  
George L. Gerstein
Keyword(s):  
Auditory Cortex ◽  
Time Course ◽  
Auditory Evoked Potentials ◽  
Mammalian Species ◽  
Sound Intensity ◽  
Primary Auditory Cortex ◽  
Normal Hearing ◽  
Auditory Task ◽  
Experimental Conditions

Studies in several mammalian species have demonstrated that bilateral ablations of the auditory cortex have little effect on simple sound intensity and frequency-based behaviors. In the rat, for example, early experiments have shown that auditory ablations result in virtually no effect on the rat's ability to either detect tones or discriminate frequencies. Such lesion experiments, however, typically examine an animal's performance some time after recovery from ablation surgery. As such, they demonstrate that the cortex is not essential for simple auditory behaviors in the long run. Our study further explores the role of cortex in basic auditory perception by examining whether the cortex is normally involved in these behaviors. In these experiments we reversibly inactivated the rat primary auditory cortex (AI) using the GABA agonist muscimol, while the animals performed a simple auditory task. At the same time we monitored the rat's auditory activity by recording auditory evoked potentials (AEP) from the cortical surface. In contrast to lesion studies, the rapid time course of these experimental conditions preclude reorganization of the auditory system that might otherwise compensate for the loss of cortical processing. Soon after bilateral muscimol application to their AI region, our rats exhibited an acute and profound inability to detect tones. After a few hours this state was followed by a gradual recovery of normal hearing, first of tone detection and, much later, of the ability to discriminate frequencies. Surface muscimol application, at the same time, drastically altered the normal rat AEP. Some of the normal AEP components vanished nearly instantaneously to unveil an underlying waveform, whose size was related to the severity of accompanying behavioral deficits. These results strongly suggest that the cortex is directly involved in basic acoustic processing. Along with observations from accompanying multiunit experiments that related the AEP to AI neuronal activity, our results suggest that a critical amount of activity in the auditory cortex is necessary for normal hearing. It is likely that the involvement of the cortex in simple auditory perceptions has hitherto not been clearly understood because of underlying recovery processes that, in the long-term, safeguard fundamental auditory abilities after cortical injury.

scholarly journals Bulk private curves require large conditional mutual information

2021 ◽  
Vol 2021 (9) ◽  
Author(s):  
Alex May
Keyword(s):  
Mutual Information ◽  
Boundary Region ◽  
Theoretic Approach ◽  
Strong Correlations ◽  
Conditional Mutual Information ◽  
Information Theoretic ◽  
Causal Curve ◽  
Resource Requirements ◽  
Theoretic Argument ◽  
Information Theoretic Approach

Abstract We prove a theorem showing that the existence of “private” curves in the bulk of AdS implies two regions of the dual CFT share strong correlations. A private curve is a causal curve which avoids the entanglement wedge of a specified boundary region $$ \mathcal{U} $$ U . The implied correlation is measured by the conditional mutual information $$ I\left({\mathcal{V}}_1:\left.{\mathcal{V}}_2\right|\mathcal{U}\right) $$ I V 1 : V 2 U , which is O(1/GN) when a private causal curve exists. The regions $$ {\mathcal{V}}_1 $$ V 1 and $$ {\mathcal{V}}_2 $$ V 2 are specified by the endpoints of the causal curve and the placement of the region $$ \mathcal{U} $$ U . This gives a causal perspective on the conditional mutual information in AdS/CFT, analogous to the causal perspective on the mutual information given by earlier work on the connected wedge theorem. We give an information theoretic argument for our theorem, along with a bulk geometric proof. In the geometric perspective, the theorem follows from the maximin formula and entanglement wedge nesting. In the information theoretic approach, the theorem follows from resource requirements for sending private messages over a public quantum channel.

scholarly journals Unsupervised Learning via Total Correlation Explanation

2017 ◽  
Author(s):  
Greg Ver Steeg
Keyword(s):  
Mutual Information ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Human Behavior ◽  
Learning Problems ◽  
Information Theoretic ◽  
Sensory Environment ◽  
Total Correlation ◽  
Environment Dependence ◽  
Multivariate Mutual Information

Learning by children and animals occurs effortlessly and largely without obvious supervision. Successes in automating supervised learning have not translated to the more ambiguous realm of unsupervised learning where goals and labels are not provided. Barlow (1961) suggested that the signal that brains leverage for unsupervised learning is dependence, or redundancy, in the sensory environment. Dependence can be characterized using the information-theoretic multivariate mutual information measure called total correlation. The principle of Total Cor-relation Ex-planation (CorEx) is to learn representations of data that "explain" as much dependence in the data as possible. We review some manifestations of this principle along with successes in unsupervised learning problems across diverse domains including human behavior, biology, and language.

scholarly journals An Information-Theoretic Account of Semantic Interference in Word Production

2021 ◽  
Vol 12 ◽  
Author(s):  
Richard Futrell
Keyword(s):  
Mutual Information ◽  
Semantic Similarity ◽  
Rate Distortion ◽  
Word Production ◽  
Interference Effects ◽  
Semantic Interference ◽  
Information Theoretic ◽  
Human Data ◽  
Level Model ◽  
Computational Level

I present a computational-level model of semantic interference effects in online word production within a rate–distortion framework. I consider a bounded-rational agent trying to produce words. The agent's action policy is determined by maximizing accuracy in production subject to computational constraints. These computational constraints are formalized using mutual information. I show that semantic similarity-based interference among words falls out naturally from this setup, and I present a series of simulations showing that the model captures some of the key empirical patterns observed in Stroop and Picture–Word Interference paradigms, including comparisons to human data from previous experiments.

CRMI: Confidence-rich Mutual Information for Information-theoretic Mapping

2021 ◽  
pp. 1-1
Author(s):  
Yang Xu ◽  
Ronghao Zheng ◽  
Meiqin Liu ◽  
Senlin Zhang
Keyword(s):  
Mutual Information ◽  
Information Theoretic
Sign in / Sign up

Export Citation Format

Share Document