scholarly journals Calculating the Prior Probability Distribution for a Causal Network Using Maximum Entropy: Alternative Approaches

Entropy ◽  
2011 ◽  
Vol 13 (7) ◽  
pp. 1281-1304
Author(s):  
Michael J. Markham
Keyword(s):  
Probability Distribution ◽  
Maximum Entropy ◽  
Prior Probability ◽  
Causal Network ◽  
Prior Probability Distribution ◽  
Alternative Approaches

INDEPENDENCE IN COMPLETE AND INCOMPLETE CAUSAL NETWORKS UNDER MAXIMUM ENTROPY

2008 ◽  
Vol 16 (05) ◽  
pp. 699-713 ◽  
Author(s):  
MICHAEL J. MARKHAM
Keyword(s):  
Expert System ◽  
Probability Distribution ◽  
Maximum Entropy ◽  
Linear Constraints ◽  
Causal Network ◽  
Missing Information ◽  
Causal Networks ◽  
Small Set

In an expert system having a consistent set of linear constraints it is known that the Method of Tribus may be used to determine a probability distribution which exhibits maximised entropy. The method is extended here to include independence constraints (Accommodation). The paper proceeds to discusses this extension, and its limitations, then goes on to advance a technique for determining a small set of independencies which can be added to the linear constraints required in a particular representation of an expert system called a causal network, so that the Maximum Entropy and Causal Networks methodologies give matching distributions (Emulation). This technique may also be applied in cases where no initial independencies are given and the linear constraints are incomplete, in order to provide an optimal ME fill-in for the missing information.

scholarly journals A Bayesian Inference Method Using Monte Carlo Sampling for Estimating the Number of Communities in Bipartite Networks

2019 ◽  
Vol 2019 ◽  
pp. 1-12 ◽  
Author(s):  
Guo-Zheng Wang ◽  
Li Xiong ◽  
Hu-Chen Liu
Keyword(s):  
Monte Carlo ◽  
Bayesian Inference ◽  
Probability Distribution ◽  
Community Detection ◽  
Degree Distribution ◽  
Prior Probability ◽  
Bipartite Networks ◽  
Inference Method ◽  
Prior Probability Distribution ◽  
Bayesian Inference Method

Community detection is an important analysis task for complex networks, including bipartite networks, which consist of nodes of two types and edges connecting only nodes of different types. Many community detection methods take the number of communities in the networks as a fixed known quantity; however, it is impossible to give such information in advance in real-world networks. In our paper, we propose a projection-free Bayesian inference method to determine the number of pure-type communities in bipartite networks. This paper makes the following contributions: (1) we present the first principle derivation of a practical method, using the degree-corrected bipartite stochastic block model that is able to deal with networks with broad degree distributions, for estimating the number of pure-type communities of bipartite networks; (2) a prior probability distribution is proposed over the partition of a bipartite network; (3) we design a Monte Carlo algorithm incorporated with our proposed method and prior probability distribution. We give a demonstration of our algorithm on synthetic bipartite networks including an easy case with a homogeneous degree distribution and a difficult case with a heterogeneous degree distribution. The results show that the algorithm gives the correct number of communities of synthetic networks in most cases and outperforms the projection method especially in the networks with heterogeneous degree distributions.

Implicitly localized MCMC sampler to cope with nonlocal/nonlinear data constraints in large-size inverse problems

2020 ◽  
Author(s):  
Jean-Michel Brankart
Keyword(s):  
Probability Distribution ◽  
Inverse Problems ◽  
Large Scale ◽  
Prior Probability ◽  
Practical Applications ◽  
Posterior Probability Distribution ◽  
Prior Probability Distribution ◽  
Large Size ◽  
Non Gaussian ◽  
Nonlocal Nonlinear

<p>Many practical applications involve the resolution of large-size inverse problems, without providing more than a moderate-size sample to describe the prior probability distribution. In this situation, additional information must be supplied to augment the effective dimension of the available sample, for instance using a covariance localization approach. In this study, it is suggested that covariance localization can be efficiently applied to an approximate variant of the Metropolis/Hastings algorithm, by modulating the ensemble members by the large-scale patterns of other members. Modulation is used to design a (global) proposal probability distribution (i) that can be sampled at a very low cost, (ii) that automatically accounts for a localized prior covariance, and (iii) that leads to an efficient sampler for the augmented prior probability distribution or for the posterior probability distribution. The resulting algorithm is applied to an academic example, illustrating (i) the effectiveness of covariance localization, (ii) the ability of the method to deal with nonlocal/nonlinear observation operators and non-Gaussian observation errors, (iii) the reliability, resolution and optimality of the updated ensemble, using probabilistic scores appropriate to a non-Gaussian posterior distribution, and (iv) the scalability of the algorithm as a function of the size of the problem. The codes are openly available from github.com/brankart/ensdam.</p>

MAXIMISING ENTROPY TO DEDUCE AN INITIAL PROBABILITY DISTRIBUTION FOR A CAUSAL NETWORK

1999 ◽  
Vol 07 (01) ◽  
pp. 63-80 ◽  
Author(s):  
MICHAEL J. MARKHAM ◽  
PAUL C. RHODES
Keyword(s):  
Probability Distribution ◽  
Expert Systems ◽  
Maximum Entropy ◽  
Linear Constraints ◽  
Causal Network ◽  
Conditional Probabilities ◽  
Causal Information ◽  
Non Linear ◽  
Causal Networks ◽  
Initial Probability Distribution

The desire to use Causal Networks as Expert Systems even when the causal information is incomplete and/or when non-causal information is available has led researchers to look into the possibility of utilising Maximum Entropy. If this approach is taken, the known information is supplemented by maximising entropy to provide a unique initial probability distribution which would otherwise have been a consequence of the known information and the independence relationships implied by the network. Traditional maximising techniques can be used if the constraints are linear but the independence relationships give rise to non-linear constraints. This paper extends traditional maximising techniques to incorporate those types of non-linear constraints that arise from the independence relationships and presents an algorithm for implementing the extended method. Maximising entropy does not involve the concept of "causal" information. Consequently, the extended method will accept any mutually consistent set of conditional probabilities and expressions of independence. The paper provides a small example of how this property can be used to provide complete causal information, for use in a causal network, when the known information is incomplete and not in a causal form.

Prior probabilities and thermal characteristics of heat engines

Open Physics ◽  
2012 ◽  
Vol 10 (3) ◽  
Author(s):  
Preety Aneja ◽  
Ramandeep Johal
Keyword(s):  
Probability Distribution ◽  
Bayesian Approach ◽  
Prior Information ◽  
Prior Probability ◽  
Thermal Characteristics ◽  
Complete Information ◽  
Otto Cycle ◽  
Prior Probabilities ◽  
Prior Probability Distribution ◽  
Heat Engines

AbstractThe thermal characteristics of a heat cycle are studied from a Bayesian approach. In this approach, we assign a certain prior probability distribution to an uncertain parameter of the system. Based on that prior, we study the expected behaviour of the system and it has been found that even in the absence of complete information, we obtain thermodynamic-like behaviour of the system. Two models of heat cycles, the quantum Otto cycle and the classical Otto cycle are studied from this perspective. Various expressions for thermal efficiences can be obtained with a generalised prior of the form Π(x) ∝ 1/x b. The predicted thermodynamic behaviour suggests a connection between prior information about the system and thermodynamic features of the system.

scholarly journals Numeral terms and the predictive potential of Bayesian updating

2021 ◽  
Vol 18 (3) ◽  
pp. 359-390
Author(s):  
Izabela Skoczeń ◽  
Aleksander Smywiński-Pohl
Keyword(s):  
Probability Distribution ◽  
Predictive Power ◽  
Prior Probability ◽  
Empirical Support ◽  
Bayesian Updating ◽  
Modeling Language ◽  
Human Reasoning ◽  
Descriptive Model ◽  
Language Understanding ◽  
Prior Probability Distribution

Abstract In the experiment described in the paper Noah Goodman & Andreas Stuhlmüller. 2013. Knowledge and im-plicature: Modeling language understanding as social cognition. Topics in Cognitive Science 5(1). 173–184, empirical support was provided for the predictive power of the Rational Speech Act (RSA) model concerning the interpretation of utterances employing numerals in uncertainty contexts. The RSA predicts a Bayesian interdependence between beliefs about the probability distribution of the occurrence of an event prior to receiving information and the updated probability distribution after receiving information. In this paper we analyze whether the RSA is a descriptive or a normative model. We present the results of two analogous experiments carried out in Polish. The first experiment does not replicate the original empirical results. We find that this is due to different answers on the prior probability distribution. However, the model predicts the different results on the basis of different collected priors: Bayesian updating predicts human reasoning. By contrast, the second experiment, where the answers on the prior probability distribution are as predicted, is a replication of the original results. In light of these results we conclude that the RSA is a robust, descriptive model, however, the experimental assumptions pertaining to the experimental setting adopted by Goodman and Stuhlmüller are normative.

scholarly journals Adaptive Probability Distribution Estimation Based upon Maximum Entropy

1984 ◽  
Vol R-33 (4) ◽  
pp. 353-357 ◽  
Author(s):  
James E. Miller ◽  
Richard W. Kulp ◽  
George E. Orr
Keyword(s):  
Probability Distribution ◽  
Maximum Entropy ◽  
Distribution Estimation
Sign in / Sign up

Export Citation Format

Share Document