scholarly journals Probabilistic Ensemble of Deep Information Networks

Entropy ◽  
2020 ◽  
Vol 22 (1) ◽  
pp. 100 ◽  
Author(s):  
Giulio Franzese ◽  
Monica Visintin
Keyword(s):  
Information Theory ◽  
Cost Function ◽  
Decision Trees ◽  
Information Networks ◽  
Information Bottleneck ◽  
Tree Classifier ◽  
Reduced Complexity ◽  
Function Information

We describe a classifier made of an ensemble of decision trees, designed using information theory concepts. In contrast to algorithms C4.5 or ID3, the tree is built from the leaves instead of the root. Each tree is made of nodes trained independently of the others, to minimize a local cost function (information bottleneck). The trained tree outputs the estimated probabilities of the classes given the input datum, and the outputs of many trees are combined to decide the class. We show that the system is able to provide results comparable to those of the tree classifier in terms of accuracy, while it shows many advantages in terms of modularity, reduced complexity, and memory requirements.

scholarly journals Counter-Interception and Counter-Exploitation Features of Noise Radar Technology

2021 ◽  
Vol 13 (22) ◽  
pp. 4509
Author(s):  
Gaspare Galati ◽  
Gabriele Pavan ◽  
Kubilay Savci ◽  
Christoph Wasserzier
Keyword(s):  
Information Theory ◽  
Technological Progress ◽  
A Priori ◽  
A Priori Knowledge ◽  
Noise Radar ◽  
Low Probability ◽  
Increasing Demand ◽  
Signal Processors ◽  
Function Information ◽  
Selection Of

In defense applications, the main features of radars are the Low Probability of Intercept (LPI) and the Low Probability of Exploitation (LPE). The counterpart uses more and more capable intercept receivers and signal processors thanks to the ongoing technological progress. Noise Radar Technology (NRT) is probably a very effective answer to the increasing demand for operational LPI/LPE radars. The design and selection of the radiated waveforms, while respecting the prescribed spectrum occupancy, has to comply with the contrasting requirements of LPI/LPE and of a favorable shape of the ambiguity function. Information theory seems to be a “technologically agnostic” tool to attempt to quantify the LPI/LPE capability of noise waveforms with little, or absent, a priori knowledge of the means and the strategies used by the counterpart. An information theoretical analysis can lead to practical results in the design and selection of NRT waveforms.

scholarly journals A Comparison of Variational Bounds for the Information Bottleneck Functional

Entropy ◽  
2020 ◽  
Vol 22 (11) ◽  
pp. 1229 ◽  
Author(s):  
Bernhard C. Geiger ◽  
Ian S. Fischer
Keyword(s):  
Cost Function ◽  
Short Note ◽  
General Setting ◽  
Conditional Entropy ◽  
Variational Bounds ◽  
Information Bottleneck ◽  
Feasible Sets ◽  
Shed Light ◽  
Relevant Cost ◽  
General Setup

In this short note, we relate the variational bounds proposed in Alemi et al. (2017) and Fischer (2020) for the information bottleneck (IB) and the conditional entropy bottleneck (CEB) functional, respectively. Although the two functionals were shown to be equivalent, it was empirically observed that optimizing bounds on the CEB functional achieves better generalization performance and adversarial robustness than optimizing those on the IB functional. This work tries to shed light on this issue by showing that, in the most general setting, no ordering can be established between these variational bounds, while such an ordering can be enforced by restricting the feasible sets over which the optimizations take place. The absence of such an ordering in the general setup suggests that the variational bound on the CEB functional is either more amenable to optimization or a relevant cost function for optimization in its own regard, i.e., without justification from the IB or CEB functionals.

scholarly journals Comparing the utility of decision trees and support vector machines when planning inspections of linear sewer infrastructure

2014 ◽  
Vol 16 (6) ◽  
pp. 1265-1279 ◽  
Author(s):  
Robert Richard Harvey ◽  
Edward Arthur McBean
Keyword(s):  
Support Vector Machines ◽  
Decision Tree ◽  
Decision Trees ◽  
Predictive Analytics ◽  
Negative Impact ◽  
Support Vector ◽  
Pipe Inspection ◽  
Essential Information ◽  
Tree Classifier ◽  
Vector Machines

Closed-circuit television inspection technology is traditionally used to identify aging sewer pipes requiring rehabilitation. While these inspections provide essential information on the condition of pipes hidden from day-to-day view, they are expensive and often limited to small portions of an entire sewer system. Municipalities may benefit from utilizing predictive analytics to leverage existing inspection datasets so that reliable predictions of condition are available for pipes that have not yet been inspected. The predictive capabilities of data mining systems, namely support vector machines (SVMs) and decision tree classifiers, are demonstrated using a case study of sanitary sewer pipe inspection data collected by the municipality of Guelph, Ontario, Canada. The modeling algorithms are implemented using open-source software and are tuned to counteract the negative impact on predictive performance resulting from class imbalance common within pipe inspection datasets. The decision tree classifier outperforms SVM for this classification task – achieving an acceptable area under the receiver operating characteristic curve of 0.77 and an overall accuracy of 76% on a stratified test set. Although predicting individual pipe condition is a notoriously difficult task, decision trees are found to be a useful screening tool for planning future inspection-related activities.

Conceptual Approach to Predict Loan Defaults Using Decision Trees

2019 ◽  
pp. 148-161 ◽  
Author(s):  
Syed Muzamil Basha ◽  
Dharmendra Singh Rajput ◽  
N. Ch. S. N. Iyengar
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Prediction Algorithm ◽  
Classification Error ◽  
Selection Algorithm ◽  
Decision Tree Classifier ◽  
Time Data ◽  
Conceptual Approach ◽  
Tree Classifier ◽  
Loan Defaults

In this chapter, the authors show how to build a decision tree from given real-time data. They interpret the output of decision tree by learning decision tree classifier using really recursive greedy algorithm. Feature selection is made based on classification error using the algorithm called feature split selection algorithm (FSSA), with all different possible stopping conditions for splitting. The authors perform prediction with decision trees using decision tree prediction algorithm (DTPA), followed by multiclass predictions and their probabilities. Finally, they perform splitting procedure on real continuous value input using threshold split selection algorithm (TSSA).

Reduced-Complexity Greedy Distributed Information Bottleneck Algorithm

2021 ◽  
Author(s):  
Steffen Steiner ◽  
Volker Kuehn ◽  
Maximilian Stark ◽  
Gerhard Bauch
Keyword(s):  
Distributed Information ◽  
Information Bottleneck ◽  
Reduced Complexity

scholarly journals Reduced-Complexity Model Predictive Control with Online Parameter Assessment for a Grid-Connected Single-Phase Multilevel Inverter

2020 ◽  
Vol 12 (19) ◽  
pp. 7997
Author(s):  
Ibrahim Harbi ◽  
Mohamed Abdelrahem ◽  
Mostafa Ahmed ◽  
Ralph Kennel
Keyword(s):  
Model Predictive Control ◽  
Cost Function ◽  
Predictive Control ◽  
Single Phase ◽  
Operating Conditions ◽  
Reference Voltage ◽  
Multilevel Inverter ◽  
Switching Frequency ◽  
Computational Load ◽  
Reduced Complexity

This paper proposes a finite control set model predictive control (FCS-MPC) with a reduced computational burden for a single-phase grid-connected modified packed U-cell multilevel inverter (MPUC-MLI) with two control objectives: reference current tracking and switching frequency minimization. The considered competitive topology consists of two units with six active switches and two DC sources in each unit, allowing the generation of 49 levels in the output voltage, which is considered a significant reduction in the active and passive components compared to the conventional and recently developed topologies of multilevel inverters (MLIs). This topology has 49 different switching states, which means that 49 predictions of the future current and 49 calculations of the cost function are required for each evaluation of the conventional FCS-MPC. Accordingly, the computational load is heavy. Thus, this paper presents two reduced-complexity FCS-MPC methods to reduce the calculation burden. The first technique reduces the computational load almost to half by computing the reference voltage and dividing the states of the MLI into two sets. Based on the reference voltage polarity, one set is defined and evaluated to specify the optimal state, which has a minimal cost function. However, in the second proposed method, only three states of the 49 states are evaluated each iteration, achieving a significant reduction in the execution time and superior control performance compared to the conventional FCS-MPC. A mathematical analysis is conducted based on the reference voltage value to locate the three vectors under evaluation. In the second part of the paper, the sensitivity to parameter variations for the proposed simplified FCS-MPC is investigated and tackled by employing an extended Kalman filter (EKF). In addition, noise related to variable measurement is filtered in the proposed system with the EKF. The simulation investigation was performed using MATLAB/Simulink to validate the system under different operating conditions.

scholarly journals Application of information theory to the construction of efficient decision trees

1982 ◽  
Vol 28 (4) ◽  
pp. 565-577 ◽  
Author(s):  
C. Hartmann ◽  
P. Varshney ◽  
K. Mehrotra ◽  
C. Gerberich
Keyword(s):  
Information Theory ◽  
Decision Trees

scholarly journals Beyond IIT: (how) can we model the evolution of consciousness?

2021 ◽  
Author(s):  
Adrian Kent
Keyword(s):  
Information Theory ◽  
Network Dynamics ◽  
Information Networks ◽  
Darwinian Evolution ◽  
Physical Systems ◽  
Evolution Of Consciousness ◽  
Classical Information ◽  
Key Features ◽  
High Level ◽  
Integrated Information Theory

Tononi et al.'s "integrated information theory" (IIT) postulates rules for assigning measures Phi and qualia types Q of consciousness to classical information networks. We consider whether IIT is compatible with Darwinian evolution. We argue that an IIT-like theory that assigns consciousness to physical systems by relatively simple mathematical rules poses extraordinary ?ne-tuning problems.For example, why, among all possible lawlike theories of consciousness, do we have one that makes us conscious of a high-level narrative of our environment and actions, so accurate that it appears to us to cause our behaviour?We introduce IIT+, a class of extensions of IIT in which Phi and/or Q influence the network dynamics. We argue that IIT+-like theories, unlike IIT-like theories, offer at least partial explanations of how some key features of consciousness evolved. We conclude that if one takes seriously Darwinian evolution and the case for an IIT-like theory, one has to take seriously the case for an IIT+-like theory.

Sign in / Sign up

Export Citation Format

Share Document