scholarly journals Sophisticated Inference

2021 ◽  
Vol 33 (3) ◽  
pp. 713-763
Author(s):  
Karl Friston ◽  
Lancelot Da Costa ◽  
Danijar Hafner ◽  
Casper Hesp ◽  
Thomas Parr
Keyword(s):  
Free Energy ◽  
Information Gain ◽  
Energy Functional ◽  
Decision Problems ◽  
Tree Search ◽  
Value Functions ◽  
Active Inference ◽  
States Of Affairs ◽  
Variational Free Energy ◽  
Belief States

Active inference offers a first principle account of sentient behavior, from which special and important cases—for example, reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design—can be derived. Active inference finesses the exploitation-exploration dilemma in relation to prior preferences by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this letter, we consider a sophisticated kind of active inference using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about “what would happen if I did that” to “what I would believe about what would happen if I did that.” The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states as opposed to states per se. We illustrate the competence of this scheme using numerical simulations of deep decision problems.

VARIATIONAL ELECTROSTATICS FOR CHARGE SOLVATION

2008 ◽  
Vol 07 (03) ◽  
pp. 397-419 ◽  
Author(s):  
ZHEN-GANG WANG
Keyword(s):  
Free Energy ◽  
Energy Functional ◽  
Dielectric Medium ◽  
Constitutive Relationship ◽  
Coulomb Interactions ◽  
Unified Framework ◽  
Variational Free Energy ◽  
The Poisson Equation ◽  
Charge Solvation ◽  
Electrostatic Problems

We show that the equations of continuum electrostatics can be obtained entirely and simply from a variational free energy comprising the Coulomb interactions among all charged species and a spring-like term for the polarization of the dielectric medium. In this formulation, the Poisson equation, the constitutive relationship between polarization and the electric field, as well as the boundary conditions across discontinuous dielectric boundaries, are all natural consequences of the extremization of the free energy functional. This formulation thus treats the electrostatic equations and the energetics within a single unified framework, avoiding some of the pitfalls in the study of electrostatic problems. Application of this formalism to the nonequilbrium solvation free energy in electron transfer is illustrated. Our calculation reaffirms the well-known result of Marcus. We address the recent criticisms by Li and coworkers who claim that the Marcus result is incorrect, and expose some key mistakes in their approach.

Active Inference, Belief Propagation, and the Bethe Approximation

2018 ◽  
Vol 30 (9) ◽  
pp. 2530-2567 ◽  
Author(s):  
Sarah Schwöbel ◽  
Stefan Kiebel ◽  
Dimitrije Marković
Keyword(s):  
Free Energy ◽  
Belief Propagation ◽  
Mean Field ◽  
Field Approximation ◽  
Mean Field Approximation ◽  
Active Inference ◽  
Inference Process ◽  
Bethe Approximation ◽  
Variational Free Energy ◽  
The Mean

When modeling goal-directed behavior in the presence of various sources of uncertainty, planning can be described as an inference process. A solution to the problem of planning as inference was previously proposed in the active inference framework in the form of an approximate inference scheme based on variational free energy. However, this approximate scheme was based on the mean-field approximation, which assumes statistical independence of hidden variables and is known to show overconfidence and may converge to local minima of the free energy. To better capture the spatiotemporal properties of an environment, we reformulated the approximate inference process using the so-called Bethe approximation. Importantly, the Bethe approximation allows for representation of pairwise statistical dependencies. Under these assumptions, the minimizer of the variational free energy corresponds to the belief propagation algorithm, commonly used in machine learning. To illustrate the differences between the mean-field approximation and the Bethe approximation, we have simulated agent behavior in a simple goal-reaching task with different types of uncertainties. Overall, the Bethe agent achieves higher success rates in reaching goal states. We relate the better performance of the Bethe agent to more accurate predictions about the consequences of its own actions. Consequently, active inference based on the Bethe approximation extends the application range of active inference to more complex behavioral tasks.

scholarly journals Whence the Expected Free Energy?

2021 ◽  
pp. 1-36
Author(s):  
Beren Millidge ◽  
Alexander Tschantz ◽  
Christopher L. Buckley
Keyword(s):  
Free Energy ◽  
Exploratory Behavior ◽  
Energy Minimization ◽  
Intrinsic Value ◽  
Natural Extension ◽  
Active Inference ◽  
Exploration And Exploitation ◽  
Free Energy Minimization ◽  
Variational Free Energy ◽  
The Future

The expected free energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the variational free energy (VFE) remain unclear. In this letter, we investigate the origins of the EFE in detail and show that it is not simply ”the free energy in the future.” We present a functional that we argue is the natural extension of the VFE but actively discourages exploratory behavior, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the free energy of the expected future (FEEF), which possesses both the epistemic component of the EFE and an intuitive mathematical grounding as the divergence between predicted and desired futures.

scholarly journals Active Inference: A Process Theory

2017 ◽  
Vol 29 (1) ◽  
pp. 1-49 ◽  
Author(s):  
Karl Friston ◽  
Thomas FitzGerald ◽  
Francesco Rigoli ◽  
Philipp Schwartenbeck ◽  
Giovanni Pezzulo
Keyword(s):  
Free Energy ◽  
Gradient Descent ◽  
Cell Activity ◽  
Face Validity ◽  
Process Theory ◽  
Active Inference ◽  
Neuronal Dynamics ◽  
Context Learning ◽  
Repetition Suppression ◽  
Variational Free Energy

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence—or minimizing variational free energy—we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes’ optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton’s principle of least action.

scholarly journals Generalised free energy and active inference: can the future cause the past?

2018 ◽  
Author(s):  
Thomas Parr ◽  
Karl J Friston
Keyword(s):  
Free Energy ◽  
Energy Functional ◽  
Generative Model ◽  
Free Energy Function ◽  
Active Inference ◽  
Future Expectations ◽  
Energy Functionals ◽  
The Future ◽  
Markov Decision ◽  
Future Outcomes

AbstractWe compare two free energy functionals for active inference under Markov decision processes. One of these is a functional of beliefs about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy); priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to counterfactual (i.e., future) observations, we equip the generative model with a prior over policies that ensure preferred (i.e., priors over) outcomes are realised. In other words, selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional – that effectively treats counterfactual observations as hidden states – we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.

scholarly journals The Emperor's New Markov Blankets

2021 ◽  
pp. 1-63
Author(s):  
Jelle Bruineberg ◽  
Krzysztof Dolega ◽  
Joe Dewhurst ◽  
Manuel Baltieri
Keyword(s):  
Free Energy ◽  
Mathematical Framework ◽  
Active Inference ◽  
Energy Principle ◽  
Free Energy Principle ◽  
Variational Free Energy ◽  
Standard Application ◽  
The Mind ◽  
Physical Boundary ◽  
Markov Blankets

Abstract The free energy principle, an influential framework in computational neuroscience and theoretical neurobiology, starts from the assumption that living systems ensure adaptive exchanges with their environment by minimizing the objective function of variational free energy. Following this premise, it claims to deliver a promising integration of the life sciences. In recent work, Markov Blankets, one of the central constructs of the free energy principle, have been applied to resolve debates central to philosophy (such as demarcating the boundaries of the mind). The aim of this paper is twofold. First, we trace the development of Markov blankets starting from their standard application in Bayesian networks, via variational inference, to their use in the literature on active inference. We then identify a persistent confusion in the literature between the formal use of Markov blankets as an epistemic tool for Bayesian inference, and their novel metaphysical use in the free energy framework to demarcate the physical boundary between an agent and its environment. Consequently, we propose to distinguish between ‘Pearl blankets’ to refer to the original epistemic use of Markov blankets and ‘Friston blankets’ to refer to the new metaphysical construct. Second, we use this distinction to critically assess claims resting on the application of Markov blankets to philosophical problems. We suggest that this literature would do well in differentiating between two different research programs: ‘inference with a model’ and ‘inference within a model’. Only the latter is capable of doing metaphysical work with Markov blankets, but requires additional philosophical premises and cannot be justified by an appeal to the success of the mathematical framework alone.

scholarly journals On the Kirchhoff-Love Hypothesis (Revised and Vindicated)

2021 ◽  
Author(s):  
Olivier Ozenda ◽  
Epifanio G. Virga
Keyword(s):  
Free Energy ◽  
Dimension Reduction ◽  
Three Dimensional ◽  
Energy Functional ◽  
The Other ◽  
Variational Model ◽  
Two Dimensional ◽  
Elastic Plates ◽  
Kinematic Constraint ◽  
Free Energy Functional

AbstractThe Kirchhoff-Love hypothesis expresses a kinematic constraint that is assumed to be valid for the deformations of a three-dimensional body when one of its dimensions is much smaller than the other two, as is the case for plates. This hypothesis has a long history checkered with the vicissitudes of life: even its paternity has been questioned, and recent rigorous dimension-reduction tools (based on standard $\varGamma $ Γ -convergence) have proven to be incompatible with it. We find that an appropriately revised version of the Kirchhoff-Love hypothesis is a valuable means to derive a two-dimensional variational model for elastic plates from a three-dimensional nonlinear free-energy functional. The bending energies thus obtained for a number of materials also show to contain measures of stretching of the plate’s mid surface (alongside the expected measures of bending). The incompatibility with standard $\varGamma $ Γ -convergence also appears to be removed in the cases where contact with that method and ours can be made.

scholarly journals Droplet minimizers for the Gates–Lebowitz–Penrose free energy functional

Nonlinearity ◽  
2009 ◽  
Vol 22 (12) ◽  
pp. 2919-2952 ◽  
Author(s):  
E A Carlen ◽  
M C Carvalho ◽  
R Esposito ◽  
J L Lebowitz ◽  
R Marra
Keyword(s):  
Free Energy ◽  
Energy Functional ◽  
Free Energy Functional

scholarly journals Droplet minimizers for the Cahn-Hilliard free energy functional

2006 ◽  
Vol 16 (2) ◽  
pp. 233-264 ◽  
Author(s):  
E. A. Carlen ◽  
M. C. Carvalho ◽  
R. Esposito ◽  
J. L. Lebowitz ◽  
R. Marra
Keyword(s):  
Free Energy ◽  
Energy Functional ◽  
Free Energy Functional
Sign in / Sign up

Export Citation Format

Share Document