scholarly journals Information-theoretic measures for nonlinear causality detection: application to social media sentiment and cryptocurrency prices

2020 ◽  
Vol 7 (9) ◽  
pp. 200863
Author(s):  
Z. Keskin ◽  
T. Aste
Keyword(s):  
Time Series ◽  
Information Transfer ◽  
Synthetic Data ◽  
Transfer Entropy ◽  
Significant Information ◽  
Information Theoretic ◽  
Order Of Magnitude ◽  
Information Theoretic Measures ◽  
Media Sentiment ◽  
Non Parametric

Information transfer between time series is calculated using the asymmetric information-theoretic measure known as transfer entropy. Geweke’s autoregressive formulation of Granger causality is used to compute linear transfer entropy, and Schreiber’s general, non-parametric, information-theoretic formulation is used to quantify nonlinear transfer entropy. We first validate these measures against synthetic data. Then we apply these measures to detect statistical causality between social sentiment changes and cryptocurrency returns. We validate results by performing permutation tests by shuffling the time series, and calculate the Z -score. We also investigate different approaches for partitioning in non-parametric density estimation which can improve the significance. Using these techniques on sentiment and price data over a 48-month period to August 2018, for four major cryptocurrencies, namely bitcoin (BTC), ripple (XRP), litecoin (LTC) and ethereum (ETH), we detect significant information transfer, on hourly timescales, with greater net information transfer from sentiment to price for XRP and LTC, and instead from price to sentiment for BTC and ETH. We report the scale of nonlinear statistical causality to be an order of magnitude larger than the linear case.

scholarly journals Transfer Entropy Analysis of Interactions between Bats Using Position and Echolocation Data

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1176
Author(s):  
Irena Shaffer ◽  
Nicole Abaid
Keyword(s):  
Information Transfer ◽  
Animal Species ◽  
Transfer Entropy ◽  
Signal Interference ◽  
Information Theoretic ◽  
Echolocation Calls ◽  
Primary Means ◽  
Information Theoretic Measures ◽  
Large Groups

Many animal species, including many species of bats, exhibit collective behavior where groups of individuals coordinate their motion. Bats are unique among these animals in that they use the active sensing mechanism of echolocation as their primary means of navigation. Due to their use of echolocation in large groups, bats run the risk of signal interference from sonar jamming. However, several species of bats have developed strategies to prevent interference, which may lead to different behavior when flying with conspecifics than when flying alone. This study seeks to explore the role of this acoustic sensing on the behavior of bat pairs flying together. Field data from a maternity colony of gray bats (Myotis grisescens) were collected using an array of cameras and microphones. These data were analyzed using the information theoretic measure of transfer entropy in order to quantify the interaction between pairs of bats and to determine the effect echolocation calls have on this interaction. This study expands on previous work that only computed information theoretic measures on the 3D position of bats without echolocation calls or that looked at the echolocation calls without using information theoretic analyses. Results show that there is evidence of information transfer between bats flying in pairs when time series for the speed of the bats and their turning behavior are used in the analysis. Unidirectional information transfer was found in some subsets of the data which could be evidence of a leader–follower interaction.

scholarly journals Decomposing information into copying versus transformation

2020 ◽  
Vol 17 (162) ◽  
pp. 20190623 ◽  
Author(s):  
Artemy Kolchinsky ◽  
Bernat Corominas-Murtra
Keyword(s):  
Amino Acid ◽  
Mutual Information ◽  
Information Transfer ◽  
World Systems ◽  
Information Theoretic ◽  
Versus Transformation ◽  
Information Theoretic Measures ◽  
Functional Consequences ◽  
Standard Information ◽  
Minimal Work

In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation . Copying occurs when messages are transmitted without modification, e.g. when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are modified systematically during transmission, e.g. when mutational biases occur during genetic replication. Standard information-theoretic measures do not distinguish these two modes of information transfer, although they may reflect different mechanisms and have different functional consequences. Starting from a few simple axioms, we derive a decomposition of mutual information into the information transmitted by copying versus the information transmitted by transformation. We begin with a decomposition that applies when the source and destination of the channel have the same set of messages and a notion of message identity exists. We then generalize our decomposition to other kinds of channels, which can involve different source and destination sets and broader notions of similarity. In addition, we show that copy information can be interpreted as the minimal work needed by a physical copying process, which is relevant for understanding the physics of replication. We use the proposed decomposition to explore a model of amino acid substitution rates. Our results apply to any system in which the fidelity of copying, rather than simple predictability, is of critical relevance.

scholarly journals Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

Entropy ◽  
2020 ◽  
Vol 22 (1) ◽  
pp. 102 ◽  
Author(s):  
Adrian Moldovan ◽  
Angel Caţaron ◽  
Răzvan Andonie
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Information Transfer ◽  
Feedforward Neural Networks ◽  
Theoretical Method ◽  
Transfer Entropy ◽  
Training Algorithms ◽  
Training Algorithm ◽  
Neural Connections ◽  
Feedback Connections

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.

scholarly journals Statistical Estimation of Changes in the Dominant Frequencies of Structures in Long Noisy Series of Monitoring Data

2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Fanis Moschas ◽  
Eva Steirou
Keyword(s):  
Time Series ◽  
Statistical Estimation ◽  
Statistical Approach ◽  
Natural Frequencies ◽  
Synthetic Data ◽  
The Other ◽  
Climatological Data ◽  
Pedestrian Bridge ◽  
Order Of Magnitude ◽  
Dominant Frequencies

Damage in structures is reflected in permanent changes of their natural frequencies and theoretically can be derived through measurements. Still, measurement-derived frequencies of structures usually reflect a superimposition of various effects, fluctuations due to environmental and loading conditions, noise, and possible permanent changes (damage or repair). The amplitude of the latter is usually of the same order of magnitude with the other effects; hence permanent shifts are masked by noise and cannot be identified, especially in long monitoring records. In order to overcome this problem, essential for the assessment of the structural health of various key structures, we adopt a statistical approach developed for the identification of shifts (inhomogeneities) in normally distributed climatological data, in particular the SNHT test. The efficiency of the SNHT was first tested on synthetic data and then on sets of estimates of dominant frequencies of a decaying pedestrian bridge. It was found that under certain conditions the SNHT can identify the location of shifts in dominant frequencies of structures; the amplitude of the shifts can then be easily computed. Since the efficiency of the test increases with the length of the time series, this test seems especially suitable for the analysis of long monitoring records.

Interaction Between Forex and Stock Markets in India: An Entropy Approach

2008 ◽  
Vol 33 (4) ◽  
pp. 27-46 ◽  
Author(s):  
Y V Reddy ◽  
A Sebastin
Keyword(s):  
Time Series ◽  
Mutual Information ◽  
Stock Market ◽  
Transfer Entropy ◽  
The Other ◽  
Stock Index ◽  
Reverse Direction ◽  
Probability Mass Function ◽  
Information Theoretic ◽  
The Relationship

Interactions between the foreign exchange market and the stock market of a country are considered to be an important internal force of the markets in a financially liberalized environment. If causal relationship from a market to the other is not detected, then informational efficiency exists in the other whereas existence of causality implies that hedging of exposure to one market by taking position in the other market will be effective. The temporal relationship between the forex market and the stock market of developing and developed countries has been studied, especially after the East Asian financial crisis of 1997–98, using various methods like cross-correlation, cross-spectrum, and error correction model, but these methods identify only linear relations. A statistically rigorous approach to the detection of interdependence, including non-linear dynamic relationships, between time series is provided by tools defined using the information theoretic concept of entropy. Entropy is the amount of disorder in the system and also is the amount of information needed to predict the next measurement with a certain precision. The mutual information between two random variables X and Y with a joint probability mass function p(x,y) and marginal mass functions p(x) and p(y), is defined as the relative entropy between the joint distribution p(x,y) and the product distribution p(x)*p(y). Mutual information is the reduction in the uncertainty of X due to the knowledge of Y and vice versa. Since mutual information measures the deviation from independence of the variables, it has been proposed as a tool to measure the relationship between financial market segments. However, mutual information is a symmetric measure and does not contain either dynamic information or directional sense. Even time delayed mutual information does not distinguish information actually exchanged from shared information due to a common input signal or history and therefore does not quantify the actual overlap of the information content of two variables. Another information theoretic measure called transfer entropy has been introduced by Thomas Schreiber (2000) to study the relationship between dynamic systems; the concept has also been applied by some authors to study the causal structure between financial time series. In this paper, an attempt has been made to study the interaction between the stock and the forex markets in India by computing transfer entropy between daily data series of the 50 stock index of the National Stock Exchange of India Limited, viz., Nifty and the exchange rate of Indian Rupee vis- à- vis US Dollar, viz., Reserve Bank of India reference rate. The entire period–November 1995 to March 2007–selected for the study, has been divided into three sub-periods for the purpose of analysis, considering the developments that took place during these sub-periods. The results obtained reveal that: there exist only low level interactions between the stock and the forex markets of India at a time scale of a day or less, although theory suggests interactive relationship between the two markets the flow from the stock market to the forex market is more pronounced than the flow in the reverse direction.

scholarly journals Information Theoretic Causality Detection between Financial and Sentiment Data

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 621
Author(s):  
Roberta Scaramozzino ◽  
Paola Cerchiello ◽  
Tomaso Aste
Keyword(s):  
Social Media ◽  
Stock Market ◽  
Stock Price ◽  
Transfer Entropy ◽  
Market Prices ◽  
Causal Information ◽  
Information Theoretic ◽  
P Index ◽  
Media Sentiment

The interaction between the flow of sentiment expressed on blogs and media and the dynamics of the stock market prices are analyzed through an information-theoretic measure, the transfer entropy, to quantify causality relations. We analyzed daily stock price and daily social media sentiment for the top 50 companies in the Standard & Poor (S&P) index during the period from November 2018 to November 2020. We also analyzed news mentioning these companies during the same period. We found that there is a causal flux of information that links those companies. The largest fraction of significant causal links is between prices and between sentiments, but there is also significant causal information which goes both ways from sentiment to prices and from prices to sentiment. We observe that the strongest causal signal between sentiment and prices is associated with the Tech sector.

scholarly journals Estimating Transfer Entropy in Continuous Time Between Neural Spike Trains or Other Event-Based Data

2020 ◽  
Author(s):  
David P. Shorten ◽  
Richard E. Spinney ◽  
Joseph T. Lizier
Keyword(s):  
Time Series ◽  
Continuous Time ◽  
Conditional Independence ◽  
Spike Trains ◽  
Stomatogastric Ganglion ◽  
Transfer Entropy ◽  
Information Flows ◽  
Information Theoretic ◽  
Neural Spike ◽  
Event Based

AbstractTransfer entropy (TE) is a widely used measure of directed information flows in a number of domains including neuroscience. Many real-world time series in which we are interested in information flows come in the form of (near) instantaneous events occurring over time, including the spiking of biological neurons, trades on stock markets and posts to social media. However, there exist severe limitations to the current approach to TE estimation on such event-based data via discretising the time series into time bins: it is not consistent, has high bias, converges slowly and cannot simultaneously capture relationships that occur with very fine time precision as well as those that occur over long time intervals. Building on recent work which derived a theoretical framework for TE in continuous time, we present an estimation framework for TE on event-based data and develop a k-nearest-neighbours estimator within this framework. This estimator is provably consistent, has favourable bias properties and converges orders of magnitude more quickly than the discrete-time estimator on synthetic examples. We also develop a local permutation scheme for generating null surrogate time series to test for the statistical significance of the TE and, as such, test for the conditional independence between the history of one point process and the updates of another — signifying the lack of a causal connection under certain weak assumptions. Our approach is capable of detecting conditional independence or otherwise even in the presence of strong pairwise time-directed correlations. The power of this approach is further demonstrated on the inference of the connectivity of biophysical models of a spiking neural circuit inspired by the pyloric circuit of the crustacean stomatogastric ganglion, succeeding where previous related estimators have failed.AUTHOR SUMMARYTransfer Entropy (TE) is an information-theoretic measure commonly used in neuroscience to measure the directed statistical dependence between a source and a target time series, possibly also conditioned on other processes. Along with measuring information flows, it is used for the inference of directed functional and effective networks from time series data. The currently-used technique for estimating TE on neural spike trains first time-discretises the data and then applies a straightforward or “plug-in” information-theoretic estimation procedure. This approach has numerous drawbacks: it is very biased, it cannot capture relationships occurring on both fine and large timescales simultaneously, converges very slowly as more data is obtained, and indeed does not even converge to the correct value. We present a new estimator for TE which operates in continuous time, demonstrating via application to synthetic examples that it addresses these problems, and can reliably differentiate statistically significant flows from (conditionally) independent spike trains. Further, we also apply it to more biologically-realistic spike trains obtained from a biophysical model of the pyloric circuit of the crustacean stomatogastric ganglion; our correct inference of the underlying connection structure here provides an important validation for our approach where similar methods have previously failed

scholarly journals Estimating Directed Phase-Amplitude Interactions from EEG Data through Kernel-Based Phase Transfer Entropy

2021 ◽  
Vol 11 (21) ◽  
pp. 9803
Author(s):  
Iván De La Pava Panche ◽  
Viviana Gómez-Orozco ◽  
Andrés Álvarez-Meza ◽  
David Cárdenas-Peña ◽  
Álvaro Orozco-Gutiérrez
Keyword(s):  
Working Memory ◽  
Time Series ◽  
Cognitive Task ◽  
Memory Task ◽  
Synthetic Data ◽  
Transfer Entropy ◽  
Alternative Methods ◽  
Interaction Detection ◽  
Eeg Data ◽  
Phase Amplitude

Cross-frequency interactions, a form of oscillatory neural activity, are thought to play an essential role in the integration of distributed information in the brain. Indeed, phase-amplitude interactions are believed to allow for the transfer of information from large-scale brain networks, oscillating at low frequencies, to local, rapidly oscillating neural assemblies. A promising approach to estimating such interactions is the use of transfer entropy (TE), a non-linear, information-theory-based effective connectivity measure. The conventional method involves feeding instantaneous phase and amplitude time series, extracted at the target frequencies, to a TE estimator. In this work, we propose that the problem of directed phase-amplitude interaction detection is recast as a phase TE estimation problem, under the hypothesis that estimating TE from data of the same nature, i.e., two phase time series, will improve the robustness to the common confounding factors that affect connectivity measures, such as the presence of high noise levels. We implement our proposal using a kernel-based TE estimator, defined in terms of Renyi’s α entropy, which has successfully been used to compute single-trial phase TE. We tested our approach on the synthetic data generated through a simulation model capable of producing a time series with directed phase-amplitude interactions at two given frequencies, and on EEG data from a cognitive task designed to activate working memory, a memory system whose underpinning mechanisms are thought to include phase–amplitude couplings. Our proposal detected statistically significant interactions between the simulated signals at the desired frequencies for the synthetic data, identifying the correct direction of the interaction. It also displayed higher robustness to noise than the alternative methods. The results attained for the working memory data showed that the proposed approach codes connectivity patterns based on directed phase–amplitude interactions, that allow for the different cognitive load levels of the working memory task to be differentiated.

Sign in / Sign up

Export Citation Format

Share Document