Principles and analytical tools for reconstruction of probabilistic dependency structures in special class

2017 ◽  
pp. 097-110
Author(s):  
O.S. Balabanov ◽  
Keyword(s):  
Mutual Information ◽  
Error Rate ◽  
Model Structure ◽  
Artificial Data ◽  
Dependency Structure ◽  
Flow Dependency ◽  
Dependency Model ◽  
Dependency Structures ◽  
Flow Graphs ◽  
Analytical Tools

We examine a problem of reconstruction of dependency structure from data. It is assumed that model structure belongs to class of "mono-flow" graphs, which is a subclass of acyclonic digraph (known as DAGs) and is super-class relatively to the poly-trees. Properties of the mono-flow dependency models are examined, especially in terms of patterns of unconditional dependencies and mutual information. We characterize the twin-association evolving among two variables. Specialized methods of inference of mono-flow dependency model are briefly reviewed. To justify correctness of model recovery from data we formulate an assumption of unconditional (marginal) edge-wise faithfulness, perhaps the most reliable one among all simple versions of Causal faithfulness assumption. On the basis of the assumption and the properties of mono-flow dependency models we derive several empirical resolutions for edge identification, which make use 2-placed statistics only. A lot of experiments with artificial data have demonstrated efficiency of the resolutions in that they correctly recover many edges and commit low error rate.

scholarly journals Input Statistics and Hebbian Cross-Talk Effects

2014 ◽  
Vol 26 (4) ◽  
pp. 654-692
Author(s):  
Anca Rădulescu
Keyword(s):  
Numerical Simulations ◽  
Cross Talk ◽  
Error Rate ◽  
Critical State ◽  
Hebbian Learning ◽  
Neural Mechanism ◽  
Prior Work ◽  
Learning Dynamics ◽  
Analytical Tools ◽  
The Brain

As an extension of prior work, we studied inspecific Hebbian learning using the classical Oja model. We used a combination of analytical tools and numerical simulations to investigate how the effects of synaptic cross talk (which we also refer to as synaptic inspecificity) depend on the input statistics. We investigated a variety of patterns that appear in dimensions higher than two (and classified them based on covariance type and input bias). We found that the effects of cross talk on learning dynamics and outcome is highly dependent on the input statistics and that cross talk may lead in some cases to catastrophic effects on learning or development. Arbitrarily small levels of cross talk are able to trigger bifurcations in learning dynamics, or bring the system in close enough proximity to a critical state, to make the effects indistinguishable from a real bifurcation. We also investigated how cross talk behaves toward unbiased (“competitive”) inputs and in which circumstances it can help the system productively resolve the competition. Finally, we discuss the idea that sophisticated neocortical learning requires accurate synaptic updates (similar to polynucleotide copying, which requires highly accurate replication). Since it is unlikely that the brain can completely eliminate cross talk, we support the proposal that is uses a neural mechanism that “proofreads” the accuracy of the updates, much as DNA proofreading lowers copying error rate.

scholarly journals A Quick and Easy Way to Estimate Entropy and Mutual Information for Neuroscience

2021 ◽  
Vol 15 ◽  
Author(s):  
Mickael Zbili ◽  
Sylvain Rama
Keyword(s):  
Mutual Information ◽  
Sampling Bias ◽  
Computationally Efficient ◽  
Mathematical Methods ◽  
Text And Image ◽  
Experimental Conditions ◽  
Histological Data ◽  
Series Of Experiments ◽  
Entropy Encoding ◽  
Analytical Tools

Calculations of entropy of a signal or mutual information between two variables are valuable analytical tools in the field of neuroscience. They can be applied to all types of data, capture non-linear interactions and are model independent. Yet the limited size and number of recordings one can collect in a series of experiments makes their calculation highly prone to sampling bias. Mathematical methods to overcome this so-called “sampling disaster” exist, but require significant expertise, great time and computational costs. As such, there is a need for a simple, unbiased and computationally efficient tool for estimating the level of entropy and mutual information. In this article, we propose that application of entropy-encoding compression algorithms widely used in text and image compression fulfill these requirements. By simply saving the signal in PNG picture format and measuring the size of the file on the hard drive, we can estimate entropy changes through different conditions. Furthermore, with some simple modifications of the PNG file, we can also estimate the evolution of mutual information between a stimulus and the observed responses through different conditions. We first demonstrate the applicability of this method using white-noise-like signals. Then, while this method can be used in all kind of experimental conditions, we provide examples of its application in patch-clamp recordings, detection of place cells and histological data. Although this method does not give an absolute value of entropy or mutual information, it is mathematically correct, and its simplicity and broad use make it a powerful tool for their estimation through experiments.

scholarly journals On the Zero-Outage Secrecy-Capacity of Dependent Fading Wiretap Channels

Entropy ◽  
2022 ◽  
Vol 24 (1) ◽  
pp. 99
Author(s):  
Eduard Jorswieck ◽  
Pin-Hsun Lin ◽  
Karl-Ludwig Besser
Keyword(s):  
Outage Probability ◽  
Fading Channels ◽  
Positive Zero ◽  
Dependence Structure ◽  
Secrecy Outage Probability ◽  
Secrecy Capacity ◽  
Secrecy Rate ◽  
Wiretap Channel ◽  
Dependency Structure ◽  
Dependency Structures

It is known that for a slow fading Gaussian wiretap channel without channel state information at the transmitter and with statistically independent fading channels, the outage probability of any given target secrecy rate is non-zero, in general. This implies that the so-called zero-outage secrecy capacity (ZOSC) is zero and we cannot transmit at any positive data rate reliably and confidentially. When the fading legitimate and eavesdropper channels are statistically dependent, this conclusion changes significantly. Our work shows that there exist dependency structures for which positive zero-outage secrecy rates (ZOSR) are achievable. In this paper, we are interested in the characterization of these dependency structures and we study the system parameters in terms of the number of observations at legitimate receiver and eavesdropper as well as average channel gains for which positive ZOSR are achieved. First, we consider the setting that there are two paths from the transmitter to the legitimate receiver and one path to the eavesdropper. We show that by introducing a proper dependence structure among the fading gains of the three paths, we can achieve a zero secrecy outage probability (SOP) for some positive secrecy rate. In this way, we can achieve a non-zero ZOSR. We conjecture that the proposed dependency structure achieves maximum ZOSR. To better understand the underlying dependence structure, we further consider the case where the channel gains are from finite alphabets and systematically and globally solve the ZOSC. In addition, we apply the rearrangement algorithm to solve the ZOSR for continuous channel gains. The results indicate that the legitimate link must have an advantage in terms of the number of antennas and average channel gains to obtain positive ZOSR. The results motivate further studies into the optimal dependency structures.

scholarly journals OPTIMAL EAVESDROPPING ON NOISY STATES IN QUANTUM KEY DISTRIBUTION

2009 ◽  
Vol 07 (01) ◽  
pp. 297-306 ◽  
Author(s):  
Z. SHADMAN ◽  
H. KAMPERMANN ◽  
T. MEYER ◽  
D. BRUß
Keyword(s):  
Mutual Information ◽  
White Noise ◽  
Error Rate ◽  
Quantum Channel ◽  
Quantum Key Distribution ◽  
Quantum Noise ◽  
Key Distribution ◽  
Qubit Error Rate ◽  
Noisy Quantum Channel

We study eavesdropping in quantum key distribution with the six state protocol, when the signal states are mixed with white noise. This situation may arise either when Alice deliberately adds noise to the signal states before they leave her lab, or in a realistic scenario where Eve cannot replace the noisy quantum channel by a noiseless one. We find Eve's optimal mutual information with Alice, for individual attacks, as a function of the qubit error rate. Our result is that added quantum noise reduces Eve's mutual information more than Bob's.

Unsupervised dependency parsing without training

2012 ◽  
Vol 18 (2) ◽  
pp. 187-203 ◽  
Author(s):  
ANDERS SØGAARD
Keyword(s):  
Language Processing ◽  
Training Data ◽  
Linguistic Knowledge ◽  
Dependency Parsing ◽  
Empirical Methods ◽  
Grammar Induction ◽  
Dependency Structure ◽  
Dependency Trees ◽  
Dependency Model ◽  
Parsing Algorithm

AbstractUsually unsupervised dependency parsers try to optimize the probability of a corpus by revising the dependency model that is assumed to have generated the corpus. In this paper we explore a different view in which a dependency structure is, among other things, a partial order on the nodes in terms of centrality or saliency. Under this assumption we directly model centrality and derive dependency trees from the ordering of words. The result is an approach to unsupervised dependency parsing that is very different from standard ones in that it requires no training data. The input words are ordered by centrality, and a parse is derived from the ranking using a simple deterministic parsing algorithm, relying on the universal dependency rules defined by Naseem et al. (Naseem, T., Chen, H., Barzilay, R., Johnson, M. 2010. Using universal linguistic knowledge to guide grammar induction. In Proceedings of Empirical Methods in Natural Language Processing, Boston, MA, USA, pp. 1234–44.). Our approach is evaluated on data from twelve different languages and is remarkably competitive.

Ordering and aging properties of systems with dependent components governed by the Archimedean copula

2021 ◽  
pp. 1-28
Author(s):  
Tanmay Sahoo ◽  
Nil Kamal Hazra
Keyword(s):  
Stochastic Orders ◽  
Archimedean Copulas ◽  
Stochastic Comparisons ◽  
Dependency Structure ◽  
Distributed Components ◽  
Wide Range ◽  
The Family ◽  
Aging Properties ◽  
Dependency Structures ◽  
Better Than

Abstract Copula is one of the widely used techniques to describe the dependency structure between components of a system. Among all existing copulas, the family of Archimedean copulas is the popular one due to its wide range of capturing the dependency structures. In this paper, we consider the systems that are formed by dependent and identically distributed components, where the dependency structures are described by Archimedean copulas. We study some stochastic comparisons results for series, parallel, and general $r$ -out-of- $n$ systems. Furthermore, we investigate whether a system of used components performs better than a used system with respect to different stochastic orders. Furthermore, some aging properties of these systems have been studied. Finally, some numerical examples are given to illustrate the proposed results.

Different Aspects of Interleaving Techniques in Wireless Communication

2015 ◽  
pp. 335-374
Author(s):  
Barnali Das ◽  
Manash Pratim Sarma ◽  
Kandarpa Kumar Sarma
Keyword(s):  
Wireless Communication ◽  
Mutual Information ◽  
Bit Error Rate ◽  
Error Rate ◽  
Communication System ◽  
Hardware Implementation ◽  
Computational Time ◽  
System Parameters ◽  
Wireless Communication System ◽  
Interleaving Techniques

This chapter describes the use of certain interleavers for use in a wireless communication set for better accuracy and constancy of the transmitted data. Different interleaver techniques and methods are explored, including the variation of associated system parameters. The performance derived is discussed and the most suitable design is ascertained which is essential for better reliability of a wireless communication system. Bit Error Rate (BER), computational time, mutual information and correlation are the parameters analysed, in case of four types of interleavers viz. general block interleaver, matrix interleaver, random interleaver and convolutional interleaver, considering a fading environment. The hardware implementation using a block interleaver is reported here as a part of this work that shows encouraging results and maybe considered to be a part of a communication system with appropriate modifications.

Sign in / Sign up

Export Citation Format

Share Document