scholarly journals Statistical Physics and Information Theory

2010 ◽  
Vol 6 (1-2) ◽  
pp. 1-212 ◽  
Author(s):  
Neri Merhav
Keyword(s):  
Information Theory ◽  
Statistical Physics

Inferring the Logic of Collective Information Processors

2020 ◽  
pp. 81-112
Author(s):  
Bryan C. Daniels
Keyword(s):  
Information Theory ◽  
Information Processing ◽  
Distributed Systems ◽  
Collective Behavior ◽  
Statistical Physics ◽  
Collective Information

From neurons to insects to societies, across biology we see impressive feats of collective information processing. What strategies do these systems use to perform useful computations? Moving toward an answer to this question, this chapter focuses on common challenges in inferring models of complicated distributed systems and how the perspective of information theory and statistical physics is useful for understanding collective behavior.

scholarly journals Criticality in Pareto Optimal Grammars?

Entropy ◽  
2020 ◽  
Vol 22 (2) ◽  
pp. 165 ◽  
Author(s):  
Luís F Seoane ◽  
Ricard Solé
Keyword(s):  
Information Theory ◽  
Phase Transitions ◽  
Critical Point ◽  
Statistical Physics ◽  
Pareto Optimal ◽  
Human Language ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Linguistic Description ◽  
Different Levels

What are relevant levels of description when investigating human language? How are these levels connected to each other? Does one description yield smoothly into the next one such that different models lie naturally along a hierarchy containing each other? Or, instead, are there sharp transitions between one description and the next, such that to gain a little bit accuracy it is necessary to change our framework radically? Do different levels describe the same linguistic aspects with increasing (or decreasing) accuracy? Historically, answers to these questions were guided by intuition and resulted in subfields of study, from phonetics to syntax and semantics. Need for research at each level is acknowledged, but seldom are these different aspects brought together (with notable exceptions). Here, we propose a methodology to inspect empirical corpora systematically, and to extract from them, blindly, relevant phenomenological scales and interactions between them. Our methodology is rigorously grounded in information theory, multi-objective optimization, and statistical physics. Salient levels of linguistic description are readily interpretable in terms of energies, entropies, phase transitions, or criticality. Our results suggest a critical point in the description of human language, indicating that several complementary models are simultaneously necessary (and unavoidable) to describe it.

scholarly journals Criticality in Pareto Optimal Grammars?

2020 ◽  
Author(s):  
Luis F Seoane ◽  
Ricard Solé
Keyword(s):  
Information Theory ◽  
Phase Transitions ◽  
Critical Point ◽  
Statistical Physics ◽  
Pareto Optimal ◽  
Human Language ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Linguistic Description ◽  
Different Levels

What are relevant levels of description when investigating human language? How are these levels connected to each other? Does one description yield smoothly into the next one such that different models lie naturally along a hierarchy containing each other? Or, instead, are there sharp transitions between one description and the next, such that to gain a little bit accuracy it is necessary to change our framework radically? Do different levels describe the same linguistic aspects with increasing (or decreasing) accuracy? Historically, answers to these questions were guided by intuition and resulted in subfields of study, from phonetics to syntax and semantics. Need for research at each level is acknowledged, but seldom are these different aspects brought together (with notable exceptions). Here we propose a methodology to inspect empirical corpora systematically, and to extract from them, blindly, relevant phenomenological scales and interactions between them. Our methodology is rigorously grounded in information theory, multi-objective optimization, and statistical physics. Salient levels of linguistic description are readily interpretable in terms of energies, entropies, phase transitions, or criticality. Our results suggest a critical point in the description of human language, indicating that several complementary models are simultaneously necessary (and unavoidable) to describe it.

Sign in / Sign up

Export Citation Format

Share Document