scholarly journals Towards a Unified Theory of Learning and Information

Entropy ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. 438
Author(s):  
Ibrahim Alabdulmohsin
Keyword(s):  
Information Theory ◽  
Statistical Learning ◽  
Learning Theory ◽  
Communication Systems ◽  
Differential Privacy ◽  
Statistical Learning Theory ◽  
Information Leakage ◽  
Risk Minimization ◽  
Open Problems ◽  
Learning Capacity

In this paper, we introduce the notion of “learning capacity” for algorithms that learn from data, which is analogous to the Shannon channel capacity for communication systems. We show how “learning capacity” bridges the gap between statistical learning theory and information theory, and we will use it to derive generalization bounds for finite hypothesis spaces, differential privacy, and countable domains, among others. Moreover, we prove that under the Axiom of Choice, the existence of an empirical risk minimization (ERM) rule that has a vanishing learning capacity is equivalent to the assertion that the hypothesis space has a finite Vapnik–Chervonenkis (VC) dimension, thus establishing an equivalence relation between two of the most fundamental concepts in statistical learning theory and information theory. In addition, we show how the learning capacity of an algorithm provides important qualitative results, such as on the relation between generalization and algorithmic stability, information leakage, and data processing. Finally, we conclude by listing some open problems and suggesting future directions of research.

Leak Detection Method Based on Support Vector Machine

2008 ◽  
Author(s):  
XiaoJing Fan ◽  
LaiBin Zhang ◽  
Wei Liang ◽  
ZhaoHui Wang
Keyword(s):  
Support Vector Machine ◽  
Statistical Learning ◽  
Learning Theory ◽  
Statistical Learning Theory ◽  
Leak Detection ◽  
Small Sample ◽  
Support Vector ◽  
Svm Classifier ◽  
Risk Minimization ◽  
Sample Set

Assumptive and uncertain factors, few leak samples, complex non-linear pipeline systems are the problems often involved in the process of pipeline leak detection. Furthermore, the pressure wave changes of leakage are similar to these of valve regulation and pump closure. Thus it is difficult to establish a reliable model and to distinguish the leak signal pattern from others in pipeline leak detection. The veracity of leak detection system is limited. This paper presents a novel technique based on the statistical learning theory, support vector machine (SVM) for pipeline leak detection. Support Vector Machine (SVM) is learning system that uses a hypothesis space of linear functions in a high dimensional feature space, trained with a learning algorithm from optimization theory that implements a learning bias derived from statistical learning theory. SVM is based on the principle of structure risk minimization as opposed to the principle of empirical risk minimization in conventional techniques. Thus, SVM has good performance for classification over small sample set. In this paper, an overview of the limitations of traditional statistics and the advantage of statistical learning theory will be introduced. In this paper, an SVM classifier is used to classify the signal pattern with few samples. Firstly, the algorithm of the SVM classifier and steps of using the model to identify leakage signals are studied. Secondly, the classification results of the experiment show that SVM classifier has high recognition accuracy. In addition, SVM is compared with neural network method. Then the paper concludes that in terms of classification ability and generalization performance, SVM has clearly advantages than neural network method over small sample set, so SVM is more applicable to pipeline leak detection.

Sign in / Sign up

Export Citation Format

Share Document