Text compression by context tree weighting

2002 ◽  
Author(s):  
J. Aberg ◽  
Yu.M. Shtarkov
Keyword(s):  
Text Compression ◽  
Context Tree

Text compression using FI sequences

1993 ◽  
Author(s):  
Cleopas Angaye ◽  
Paul Fisher
Keyword(s):  
Text Compression

Erratum: Text compression as rule based pattern recognition

1993 ◽  
Vol 29 (24) ◽  
pp. 2155
Author(s):  
H.U. Khan ◽  
J. Ahmad ◽  
A. Mahmood ◽  
H.A. Fatmi
Keyword(s):  
Pattern Recognition ◽  
Text Compression ◽  
Rule Based

scholarly journals Text compression via alphabet re-representation

1999 ◽  
Vol 12 (4-5) ◽  
pp. 755-765 ◽  
Author(s):  
P.M. Long ◽  
A.I. Natsev ◽  
J.S. Vitter
Keyword(s):  
Text Compression

scholarly journals Adaptive Context Tree Weighting

2012 ◽  
Author(s):  
Alexander O'Neill ◽  
Marcus Hutter ◽  
Wen Shao ◽  
Peter Sunehag
Keyword(s):  
Context Tree

Poisson mixtures

1995 ◽  
Vol 1 (2) ◽  
pp. 163-190 ◽  
Author(s):  
Kenneth W. Church ◽  
William A. Gale
Keyword(s):  
Negative Binomial ◽  
Probability Distributions ◽  
Hidden Variables ◽  
Heterogeneous Structure ◽  
Text Compression ◽  
Inverse Document Frequency ◽  
Poisson Mixtures ◽  
Document Frequency ◽  
Wide Range ◽  
Better Than

AbstractShannon (1948) showed that a wide range of practical problems can be reduced to the problem of estimating probability distributions of words and ngrams in text. It has become standard practice in text compression, speech recognition, information retrieval and many other applications of Shannon's theory to introduce a “bag-of-words” assumption. But obviously, word rates vary from genre to genre, author to author, topic to topic, document to document, section to section, and paragraph to paragraph. The proposed Poisson mixture captures much of this heterogeneous structure by allowing the Poisson parameter θ to vary over documents subject to a density function φ. φ is intended to capture dependencies on hidden variables such genre, author, topic, etc. (The Negative Binomial is a well-known special case where φ is a Г distribution.) Poisson mixtures fit the data better than standard Poissons, producing more accurate estimates of the variance over documents (σ2), entropy (H), inverse document frequency (IDF), and adaptation (Pr(x ≥ 2/x ≥ 1)).

Sign in / Sign up

Export Citation Format

Share Document