scholarly journals Information-Theoretic Bound on the Entropy Production to Maintain a Classical Nonequilibrium Distribution Using Ancillary Control

Entropy ◽  
2017 ◽  
Vol 19 (7) ◽  
pp. 333 ◽  
Author(s):  
◽  
Keyword(s):  
Entropy Production ◽  
Nonequilibrium Distribution ◽  
Information Theoretic

Entropy production in bulk isothermal relaxation

1977 ◽  
Vol 55 (9) ◽  
pp. 1588-1591 ◽  
Author(s):  
Andrew W. Yau ◽  
Huw O. Pritchard
Keyword(s):  
Entropy Production ◽  
Time Constant ◽  
Maximum Entropy Principle ◽  
Complete Monotonicity ◽  
Theoretic Approach ◽  
Total Entropy ◽  
Information Theoretic ◽  
Entropy Principle ◽  
Bulk Relaxation ◽  
Information Theoretic Approach

Four theorems relating to complete monotonicity of entropy production in isothermal bulk relaxation have been proved, and in another two cases, it is shown by numerical experiment that the entropy production is probably monotonic also. The total entropy and the total energy do not in general relax with the same rate constant, and under many conditions the time constant for entropy relaxation approaches twice the time constant for energy relaxation. We speculate that the maximum entropy principle used in the information-theoretic approach to bulk relaxation may be synonymous with the principle of complete monotonicity of entropy production.

Large-sample confidence intervals of information-theoretic measures in linguistics

2020 ◽  
Vol 6 (1) ◽  
pp. 19-54
Author(s):  
Ryan Ka Yau Lai ◽  
Youngah Do
Keyword(s):  
Maximum Likelihood ◽  
Corpus Linguistics ◽  
Delta Method ◽  
Confidence Bounds ◽  
Likelihood Estimator ◽  
Information Theoretic ◽  
Leibler Divergence ◽  
Information Theoretic Measures ◽  
Data Points ◽  
Measure Of Uncertainty

This article explores a method of creating confidence bounds for information-theoretic measures in linguistics, such as entropy, Kullback-Leibler Divergence (KLD), and mutual information. We show that a useful measure of uncertainty can be derived from simple statistical principles, namely the asymptotic distribution of the maximum likelihood estimator (MLE) and the delta method. Three case studies from phonology and corpus linguistics are used to demonstrate how to apply it and examine its robustness against common violations of its assumptions in linguistics, such as insufficient sample size and non-independence of data points.

Sign in / Sign up

Export Citation Format

Share Document