Learning minimal latent directed information trees

2012 ◽  
Author(s):  
Jalal Etesami ◽  
Negar Kiyavash ◽  
Todd P. Coleman
Keyword(s):  
Directed Information

scholarly journals Directed Data-Processing Inequalities for Systems with Feedback

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 533
Author(s):  
Milan S. Derpich ◽  
Jan Østergaard
Keyword(s):  
Mutual Information ◽  
Data Processing ◽  
Channel Coding ◽  
Communication Channel ◽  
Closed Loop ◽  
Transmission Rate ◽  
Information Rate ◽  
Random Input ◽  
Directed Information ◽  
Stochastic Mapping

We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal deterministic blocks within such systems are restricted only to be causal mappings, but are allowed to be non-linear and time varying, and randomized by their own external random input, can yield any stochastic mapping. These randomized blocks can for example represent source encoders, decoders, or even communication channels. Moreover, the involved signals can be arbitrarily distributed. Our first main result relates mutual and directed information and can be interpreted as a law of conservation of information flow. Our second main result is a pair of data-processing inequalities (one the conditional version of the other) between nested pairs of random sequences entirely within the closed loop. Our third main result introduces and characterizes the notion of in-the-loop (ITL) transmission rate for channel coding scenarios in which the messages are internal to the loop. Interestingly, in this case the conventional notions of transmission rate associated with the entropy of the messages and of channel capacity based on maximizing the mutual information between the messages and the output turn out to be inadequate. Instead, as we show, the ITL transmission rate is the unique notion of rate for which a channel code attains zero error probability if and only if such an ITL rate does not exceed the corresponding directed information rate from messages to decoded messages. We apply our data-processing inequalities to show that the supremum of achievable (in the usual channel coding sense) ITL transmission rates is upper bounded by the supremum of the directed information rate across the communication channel. Moreover, we present an example in which this upper bound is attained. Finally, we further illustrate the applicability of our results by discussing how they make possible the generalization of two fundamental inequalities known in networked control literature.

e-Search

2011 ◽  
pp. 1-18 ◽  
Author(s):  
Kuan-Pin Chiang ◽  
Ruby Roy Dholakia ◽  
Stu Westin
Keyword(s):  
Conceptual Framework ◽  
Web Sites ◽  
Information Search ◽  
Product Information ◽  
Search Behavior ◽  
Consumer Search ◽  
Online Marketplace ◽  
Theoretical Perspectives ◽  
Directed Information ◽  
Online Search Behavior

The continued success of online shopping will be determined by the degree to which consumers utilize the Internet during their decision making process, mainly the acquisition of product information. This chapter addresses consumers’ goal-directed information search in the online marketplace. To understand consumer search behavior in this unique environment, relevant theoretical perspectives are drawn to provide a conceptual framework that provides an explanation of consumers’ online search behavior. In an environment characterized by human-computer interaction, the framework includes consumers’ choice to search information online and two sets of variables – domain and system (personal) and interruptions and information load (system), affecting information search between and within Web sites. Several implications of this conceptual framework are also discussed.

Sign in / Sign up

Export Citation Format

Share Document