The Information Content of Accounting Reports: An Information Theory Perspective

2013 ◽  
Author(s):  
Jonathan F. Ross
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Accounting Reports

scholarly journals What’s Worth Talking About? Information Theory Reveals How Children Balance Informativeness and Ease of Production

2017 ◽  
Vol 28 (7) ◽  
pp. 954-966 ◽  
Author(s):  
Colin Bannard ◽  
Marla Rosner ◽  
Danielle Matthews
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Low Frequency ◽  
Initial Experiment ◽  
Information Theoretic ◽  
Information Theoretic Measures

Of all the things a person could say in a given situation, what determines what is worth saying? Greenfield’s principle of informativeness states that right from the onset of language, humans selectively comment on whatever they find unexpected. In this article, we quantify this tendency using information-theoretic measures and report on a study in which we tested the counterintuitive prediction that children will produce words that have a low frequency given the context, because these will be most informative. Using corpora of child-directed speech, we identified adjectives that varied in how informative (i.e., unexpected) they were given the noun they modified. In an initial experiment ( N = 31) and in a replication ( N = 13), 3-year-olds heard an experimenter use these adjectives to describe pictures. The children’s task was then to describe the pictures to another person. As the information content of the experimenter’s adjective increased, so did children’s tendency to comment on the feature that adjective had encoded. Furthermore, our analyses suggest that children balance informativeness with a competing drive to ease production.

scholarly journals Quantifying Legal Entropy

2021 ◽  
Vol 9 ◽  
Author(s):  
Ted Sichelman
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Legal System ◽  
Legal Systems ◽  
Information Costs ◽  
Precise Quantification ◽  
The Law ◽  
Legal Contexts ◽  
Definition Of ◽  
Quantitative Definition

Many scholars have employed the term “entropy” in the context of law and legal systems to roughly refer to the amount of “uncertainty” present in a given law, doctrine, or legal system. Just a few of these scholars have attempted to formulate a quantitative definition of legal entropy, and none have provided a precise formula usable across a variety of legal contexts. Here, relying upon Claude Shannon's definition of entropy in the context of information theory, I provide a quantitative formalization of entropy in delineating, interpreting, and applying the law. In addition to offering a precise quantification of uncertainty and the information content of the law, the approach offered here provides other benefits. For example, it offers a more comprehensive account of the uses and limits of “modularity” in the law—namely, using the terminology of Henry Smith, the use of legal “boundaries” (be they spatial or intangible) that “economize on information costs” by “hiding” classes of information “behind” those boundaries. In general, much of the “work” performed by the legal system is to reduce legal entropy by delineating, interpreting, and applying the law, a process that can in principle be quantified.

scholarly journals Zipf’s Law, unbounded complexity and open-ended evolution

2018 ◽  
Vol 15 (149) ◽  
pp. 20180395 ◽  
Author(s):  
Bernat Corominas-Murtra ◽  
Luís F. Seoane ◽  
Ricard Solé
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Multiple Scales ◽  
Evolutionary Process ◽  
Zipf’S Law ◽  
Shannon Information ◽  
Zipf's Law ◽  
Paradoxical Situation ◽  
Algorithmic Information ◽  
Standard Information

A major problem for evolutionary theory is understanding the so-called open-ended nature of evolutionary change, from its definition to its origins. Open-ended evolution (OEE) refers to the unbounded increase in complexity that seems to characterize evolution on multiple scales. This property seems to be a characteristic feature of biological and technological evolution and is strongly tied to the generative potential associated with combinatorics, which allows the system to grow and expand their available state spaces. Interestingly, many complex systems presumably displaying OEE, from language to proteins, share a common statistical property: the presence of Zipf’s Law. Given an inventory of basic items (such as words or protein domains) required to build more complex structures (sentences or proteins) Zipf’s Law tells us that most of these elements are rare whereas a few of them are extremely common. Using algorithmic information theory, in this paper we provide a fundamental definition for open-endedness, which can be understood as postulates . Its statistical counterpart, based on standard Shannon information theory, has the structure of a variational problem which is shown to lead to Zipf’s Law as the expected consequence of an evolutionary process displaying OEE. We further explore the problem of information conservation through an OEE process and we conclude that statistical information (standard Shannon information) is not conserved, resulting in the paradoxical situation in which the increase of information content has the effect of erasing itself. We prove that this paradox is solved if we consider non-statistical forms of information. This last result implies that standard information theory may not be a suitable theoretical framework to explore the persistence and increase of the information content in OEE systems.

scholarly journals Quantifying Information Content in Multispectral Remote-Sensing Images Based on Image Transforms and Geostatistical Modelling

2020 ◽  
Vol 12 (5) ◽  
pp. 880
Author(s):  
Ying Zhang ◽  
Jingxiong Zhang ◽  
Wenjing Yang
Keyword(s):  
Remote Sensing ◽  
Information Theory ◽  
Information Content ◽  
Spatial Dependence ◽  
Spectral Dependence ◽  
Systematic Research ◽  
Multispectral Images ◽  
Remote Sensing Images ◽  
Information Theoretic ◽  
Information Processes

Quantifying information content in remote-sensing images is fundamental for information-theoretic characterization of remote sensing information processes, with the images being usually information sources. Information-theoretic methods, being complementary to conventional statistical methods, enable images and their derivatives to be described and analyzed in terms of information as defined in information theory rather than data per se. However, accurately quantifying images’ information content is nontrivial, as information redundancy due to spectral and spatial dependence needs to be properly handled. There has been little systematic research on this, hampering wide applications of information theory. This paper seeks to fill this important research niche by proposing a strategy for quantifying information content in multispectral images based on information theory, geostatistics, and image transformations, by which interband spectral dependence, intraband spatial dependence, and additive noise inherent to multispectral images are effectively dealt with. Specifically, to handle spectral dependence, independent component analysis (ICA) is performed to transform a multispectral image into one with statistically independent image bands (not spectral bands of the original image). The ICA-transformed image is further normal-transformed to facilitate computation of information content based on entropy formulas for Gaussian distributions. Normal transform facilitates straightforward incorporation of spatial dependence in entropy computation for the aforementioned double-transformed image bands with inter-pixel spatial correlation modeled via variograms. Experiments were undertaken using Landsat ETM+ and TM image subsets featuring different dominant land cover types (i.e., built-up, agricultural, and hilly). The experimental results confirm that the proposed methods provide more objective estimates of information content than otherwise when spectral dependence, spatial dependence, or non-normality is not accommodated properly. The differences in information content between image subsets obtained with ETM+ and TM were found to be about 3.6 bits/pixel, indicating the former’s greater information content. The proposed methods can be adapted for information-theoretic analyses of remote sensing information processes.

scholarly journals A NEW GENERALIZED VARENTROPY AND ITS PROPERTIES

2020 ◽  
Vol 6 (1) ◽  
pp. 114
Author(s):  
Saeid Maadani ◽  
Gholam Reza Mohtashami Borzadaran ◽  
Abdol Hamid Rezaei Roknabadi
Keyword(s):  
Information Theory ◽  
Order Statistics ◽  
Information Content ◽  
Random Variable ◽  
Tsallis Entropy ◽  
Shannon Information ◽  
Lifetime Distributions

The variance of Shannon information related to the random variable \(X\), which is called varentropy, is a measurement that indicates, how the information content of \(X\) is scattered around its entropy and explains its various applications in information theory, computer sciences, and statistics. In this paper, we introduce a new generalized varentropy based on the Tsallis entropy and also obtain some results and bounds for it. We compare the varentropy with the Tsallis varentropy. Moreover, we explain the Tsallis varentropy of the order statistics and analyse this concept in residual (past) lifetime distributions and then introduce two new classes of distributions by them.

scholarly journals Improvements to Information Entropy for Raster Spatial Data: A Thermodynamic-based Evaluation

2019 ◽  
Vol 1 ◽  
pp. 1-1
Author(s):  
Hong Zhang ◽  
Peichao Gao ◽  
Zhilin Li
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Information Entropy ◽  
Spatial Data ◽  
Spatial Information ◽  
Comprehensive Evaluation ◽  
Statistical Information ◽  
Late 20Th Century ◽  
Spatial Datasets ◽  
Spatial Dataset

<p><strong>Abstract.</strong> Spatial information is fundamentally important to our daily life. It has been estimated by many scholars that almost 80 percent or more of all information in this world are spatially referenced and can be regarded as spatial information. Given such importance, a discipline called spatial information theory has been formed since the late 20th century. In addition, international conferences on spatial information have been frequently held. For example, COSIT (Conference on Spatial Information Theory) was established in 1993 and are held every two years all over the world.</p><p>In spatial information theory, one fundamental question is how to measure the amount of information (i.e., information content) of a spatial dataset. A widely used method is to employ entropy, which is proposed by the American mathematician Claude Shannon in 1948 and usually referred to as Shannon entropy or information entropy. This information entropy was originally designed to measure the statistical information content of a telegraph message. However, a spatial dataset such as a map or a remote sensing image contains not only statistical information but also spatial information, which cannot be measured by using the information entropy.</p><p>As a consequence, considerable efforts have been made to improve the information entropy for spatial datasets in either a vector format of a raster format. There are two basic lines of thought. The first is to improve the information entropy by defining how to calculate its probability parameters, and the other is to introduce new parameters into the formula of the information entropy. The former results in a number of improved information entropies, while the latter leads to a series of variants of the information entropy. Both seem to be capable of distinguishing different spatial datasets, but there is a lack of comprehensive evaluation of their performance in measuring spatial information.</p><p>This study first presents a state-of-the-art review of the improvements to the information entropy for the information content of spatial datasets in a raster format (i.e., raster spatial data, such as a grey image and a digital elevation model). Then, it presents a comprehensive evaluation of the resultant measures (either improved information entropies or variants of the information entropy) according to the Second Law of Thermodynamics. A set of evaluation criteria were proposed, as well as corresponding measures. All resultant measures were ranked accordingly.</p><p>The results reported in this study should be useful for entropic spatial data analysis. For example, in image fusion, a crucial question is how to evaluate the performance of a fusion algorithm. This evaluation is usually achieved by using the information entropy to measure the increase in the information content during the fusion. It can now be performed by the best-improved information entropy reported in this study.</p>

scholarly journals Data compression to define information content of hydrological time series

2013 ◽  
Vol 10 (2) ◽  
pp. 2029-2065 ◽  
Author(s):  
S. V. Weijs ◽  
N. van de Giesen ◽  
M. B. Parlange
Keyword(s):  
Information Theory ◽  
Time Series ◽  
Data Compression ◽  
Information Content ◽  
Epistemic Uncertainty ◽  
Model Performance ◽  
Analysis Tool ◽  
Algorithmic Information Theory ◽  
Hydrological Data ◽  
Algorithmic Information

Abstract. When inferring models from hydrological data or calibrating hydrological models, we might be interested in the information content of those data to quantify how much can potentially be learned from them. In this work we take a perspective from (algorithmic) information theory (AIT) to discuss some underlying issues regarding this question. In the information-theoretical framework, there is a strong link between information content and data compression. We exploit this by using data compression performance as a time series analysis tool and highlight the analogy to information content, prediction, and learning (understanding is compression). The analysis is performed on time series of a set of catchments, searching for the mechanisms behind compressibility. We discuss both the deeper foundation from algorithmic information theory, some practical results and the inherent difficulties in answering the question: "How much information is contained in this data?". The conclusion is that the answer to this question can only be given once the following counter-questions have been answered: (1) Information about which unknown quantities? (2) What is your current state of knowledge/beliefs about those quantities? Quantifying information content of hydrological data is closely linked to the question of separating aleatoric and epistemic uncertainty and quantifying maximum possible model performance, as addressed in current hydrological literature. The AIT perspective teaches us that it is impossible to answer this question objectively, without specifying prior beliefs. These beliefs are related to the maximum complexity one is willing to accept as a law and what is considered as random.

scholarly journals Interpretation of multi-scale permeability data through an information theory perspective

2020 ◽  
Vol 24 (6) ◽  
pp. 3097-3109
Author(s):  
Aronne Dell'Oca ◽  
Alberto Guadagnini ◽  
Monica Riva
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Information Content ◽  
Gas Permeability ◽  
Measurement Scale ◽  
Measurement Scales ◽  
Multi Scale ◽  
Multivariate Mutual Information ◽  
Permeability Data

Abstract. We employ elements of information theory to quantify (i) the information content related to data collected at given measurement scales within the same porous medium domain and (ii) the relationships among information contents of datasets associated with differing scales. We focus on gas permeability data collected over Berea Sandstone and Topopah Spring Tuff blocks, considering four measurement scales. We quantify the way information is shared across these scales through (i) the Shannon entropy of the data associated with each support scale, (ii) mutual information shared between data taken at increasing support scales, and (iii) multivariate mutual information shared within triplets of datasets, each associated with a given scale. We also assess the level of uniqueness, redundancy and synergy (rendering, i.e., information partitioning) of information content that the data associated with the intermediate and largest scales provide with respect to the information embedded in the data collected at the smallest support scale in a triplet. Highlights. Information theory allows characterization of the information content of permeability data related to differing measurement scales. An increase in the measurement scale is associated with quantifiable loss of information about permeability. Redundant, unique and synergetic contributions of information are evaluated for triplets of permeability datasets, each taken at a given scale.

Sign in / Sign up

Export Citation Format

Share Document