scholarly journals Computational Techniques for Investigating Information Theoretic Limits of Information Systems

Information ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 82
Author(s):  
Chao Tian ◽  
James S. Plank ◽  
Brent Hurst ◽  
Ruida Zhou
Keyword(s):  
Information Systems ◽  
Optimal Solution ◽  
Linear Program ◽  
Computational Techniques ◽  
Fundamental Limits ◽  
Information Theoretic ◽  
Information Measures ◽  
Computation Efficiency ◽  
Optimal Value ◽  
Outer Bound

Computer-aided methods, based on the entropic linear program framework, have been shown to be effective in assisting the study of information theoretic fundamental limits of information systems. One key element that significantly impacts their computation efficiency and applicability is the reduction of variables, based on problem-specific symmetry and dependence relations. In this work, we propose using the disjoint-set data structure to algorithmically identify the reduction mapping, instead of relying on exhaustive enumeration in the equivalence classification. Based on this reduced linear program, we consider four techniques to investigate the fundamental limits of information systems: (1) computing an outer bound for a given linear combination of information measures and providing the values of information measures at the optimal solution; (2) efficiently computing a polytope tradeoff outer bound between two information quantities; (3) producing a proof (as a weighted sum of known information inequalities) for a computed outer bound; and (4) providing the range for information quantities between which the optimal value does not change, i.e., sensitivity analysis. A toolbox, with an efficient JSON format input frontend, and either Gurobi or Cplex as the linear program solving engine, is implemented and open-sourced.

scholarly journals The Expected Total Cost Criterion for Markov Decision Processes under Constraints

2013 ◽  
Vol 45 (3) ◽  
pp. 837-859 ◽  
Author(s):  
François Dufour ◽  
A. B. Piunovskiy
Keyword(s):  
Markov Decision Processes ◽  
Optimal Solution ◽  
Decision Processes ◽  
Linear Program ◽  
Programming Approach ◽  
Stationary Policy ◽  
Total Cost ◽  
Optimal Value ◽  
Markov Decision ◽  
Expected Total Cost

In this work, we study discrete-time Markov decision processes (MDPs) with constraints when all the objectives have the same form of expected total cost over the infinite time horizon. Our objective is to analyze this problem by using the linear programming approach. Under some technical hypotheses, it is shown that if there exists an optimal solution for the associated linear program then there exists a randomized stationary policy which is optimal for the MDP, and that the optimal value of the linear program coincides with the optimal value of the constrained control problem. A second important result states that the set of randomized stationary policies provides a sufficient set for solving this MDP. It is important to note that, in contrast with the classical results of the literature, we do not assume the MDP to be transient or absorbing. More importantly, we do not impose the cost functions to be nonnegative or to be bounded below. Several examples are presented to illustrate our results.

scholarly journals The Expected Total Cost Criterion for Markov Decision Processes under Constraints

2013 ◽  
Vol 45 (03) ◽  
pp. 837-859 ◽  
Author(s):  
François Dufour ◽  
A. B. Piunovskiy
Keyword(s):  
Markov Decision Processes ◽  
Optimal Solution ◽  
Decision Processes ◽  
Linear Program ◽  
Programming Approach ◽  
Stationary Policy ◽  
Total Cost ◽  
Optimal Value ◽  
Markov Decision ◽  
Expected Total Cost

In this work, we study discrete-time Markov decision processes (MDPs) with constraints when all the objectives have the same form of expected total cost over the infinite time horizon. Our objective is to analyze this problem by using the linear programming approach. Under some technical hypotheses, it is shown that if there exists an optimal solution for the associated linear program then there exists a randomized stationary policy which is optimal for the MDP, and that the optimal value of the linear program coincides with the optimal value of the constrained control problem. A second important result states that the set of randomized stationary policies provides a sufficient set for solving this MDP. It is important to note that, in contrast with the classical results of the literature, we do not assume the MDP to be transient or absorbing. More importantly, we do not impose the cost functions to be nonnegative or to be bounded below. Several examples are presented to illustrate our results.

scholarly journals Approaching the Optimal Solution of the Maximal α-quasi-clique Local Community Problem

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1438
Author(s):  
Patricia Conde-Cespedes
Keyword(s):  
Graph Theory ◽  
Community Detection ◽  
Local Community ◽  
Optimal Solution ◽  
Local Communities ◽  
Optimal Value ◽  
Interesting Task ◽  
Local Community Detection ◽  
Networks Analysis ◽  
Node Problem

Complex networks analysis (CNA) has attracted so much attention in the last few years. An interesting task in CNA complex network analysis is community detection. In this paper, we focus on Local Community Detection, which is the problem of detecting the community of a given node of interest in the whole network. Moreover, we study the problem of finding local communities of high density, known as α-quasi-cliques in graph theory (for high values of α in the interval ]0,1[). Unfortunately, the higher α is, the smaller the communities become. This led to the maximal α-quasi-clique community of a given node problem, which is, the problem of finding local communities that are α-quasi-cliques of maximal size. This problem is NP-hard, then, to approach the optimal solution, some heuristics exist. When α is high (>0.5) the diameter of a maximal α-quasi-clique is at most 2. Based on this property, we propose an algorithm to calculate an upper bound to approach the optimal solution. We evaluate our method in real networks and conclude that, in most cases, the bound is very accurate. Furthermore, for a real small network, the optimal value is exactly achieved in more than 80% of cases.

scholarly journals Distributed Constrained Stochastic Subgradient Algorithms Based on Random Projection and Asynchronous Broadcast over Networks

2017 ◽  
Vol 2017 ◽  
pp. 1-13
Author(s):  
Junlong Zhu ◽  
Ping Xie ◽  
Qingtao Wu ◽  
Mingchuan Zhang ◽  
Ruijuan Zheng ◽  
...  
Keyword(s):  
Information Exchange ◽  
Optimal Solution ◽  
Random Projection ◽  
Learning Rate ◽  
Time Varying ◽  
Optimal Value ◽  
Constraint Set ◽  
Time Varying Networks ◽  
Subgradient Algorithms ◽  
Asymptotic Upper Bound

We consider a distributed constrained optimization problem over a time-varying network, where each agent only knows its own cost functions and its constraint set. However, the local constraint set may not be known in advance or consists of huge number of components in some applications. To deal with such cases, we propose a distributed stochastic subgradient algorithm over time-varying networks, where the estimate of each agent projects onto its constraint set by using random projection technique and the implement of information exchange between agents by employing asynchronous broadcast communication protocol. We show that our proposed algorithm is convergent with probability 1 by choosing suitable learning rate. For constant learning rate, we obtain an error bound, which is defined as the expected distance between the estimates of agent and the optimal solution. We also establish an asymptotic upper bound between the global objective function value at the average of the estimates and the optimal value.

scholarly journals Information Guided Exploration of Scalar Values and Isocontours in Ensemble Datasets

Entropy ◽  
2018 ◽  
Vol 20 (7) ◽  
pp. 540 ◽  
Author(s):  
Subhashis Hazarika ◽  
Ayan Biswas ◽  
Soumya Dutta ◽  
Han-Wei Shen
Keyword(s):  
Weather Forecasting ◽  
Conditional Entropy ◽  
Specific Information ◽  
Theoretic Approach ◽  
Information Theoretic ◽  
Information Measures ◽  
New Information ◽  
Real World Datasets ◽  
The Individual ◽  
Information Theoretic Approach

Uncertainty of scalar values in an ensemble dataset is often represented by the collection of their corresponding isocontours. Various techniques such as contour-boxplot, contour variability plot, glyphs and probabilistic marching-cubes have been proposed to analyze and visualize ensemble isocontours. All these techniques assume that a scalar value of interest is already known to the user. Not much work has been done in guiding users to select the scalar values for such uncertainty analysis. Moreover, analyzing and visualizing a large collection of ensemble isocontours for a selected scalar value has its own challenges. Interpreting the visualizations of such large collections of isocontours is also a difficult task. In this work, we propose a new information-theoretic approach towards addressing these issues. Using specific information measures that estimate the predictability and surprise of specific scalar values, we evaluate the overall uncertainty associated with all the scalar values in an ensemble system. This helps the scientist to understand the effects of uncertainty on different data features. To understand in finer details the contribution of individual members towards the uncertainty of the ensemble isocontours of a selected scalar value, we propose a conditional entropy based algorithm to quantify the individual contributions. This can help simplify analysis and visualization for systems with more members by identifying the members contributing the most towards overall uncertainty. We demonstrate the efficacy of our method by applying it on real-world datasets from material sciences, weather forecasting and ocean simulation experiments.

Semidefinite Programming-Based Method for Implementing Linear Fitting to Interval-Valued Data

2011 ◽  
Vol 1 (3) ◽  
pp. 32-46 ◽  
Author(s):  
Minghuang Li ◽  
Fusheng Yu
Keyword(s):  
Lower Bound ◽  
Semidefinite Programming ◽  
Large Scale ◽  
Experimental Studies ◽  
Optimal Solution ◽  
Data Set ◽  
Linear Fitting ◽  
Optimal Value ◽  
Fitting Model ◽  
Interval Valued

Building a linear fitting model for a given interval-valued data set is challenging since the minimization of the residue function leads to a huge combinatorial problem. To overcome such a difficulty, this article proposes a new semidefinite programming-based method for implementing linear fitting to interval-valued data. First, the fitting model is cast to a problem of quadratically constrained quadratic programming (QCQP), and then two formulae are derived to develop the lower bound on the optimal value of the nonconvex QCQP by semidefinite relaxation and Lagrangian relaxation. In many cases, this method can solve the fitting problem by giving the exact optimal solution. Even though the lower bound is not the optimal value, it is still a good approximation of the global optimal solution. Experimental studies on different fitting problems of different scales demonstrate the good performance and stability of our method. Furthermore, the proposed method performs very well in solving relatively large-scale interval-fitting problems.

Theoretical Analysis on Powers-of-Two Applied to JSP

2011 ◽  
Vol 2 (2) ◽  
pp. 1-20
Author(s):  
V. Mahesh ◽  
L. Siva Rama Krishna ◽  
Sandeep Dulluri ◽  
C. S. P. Rao
Keyword(s):  
Job Shop ◽  
Job Shop Scheduling ◽  
Optimal Schedule ◽  
Optimal Solution ◽  
Mathematical Induction ◽  
Release Dates ◽  
Scheduling Problems ◽  
Job Shop Scheduling Problem ◽  
Shop Scheduling ◽  
Optimal Value

This paper discusses the scheduling of precedence-related jobs non-preemptively in a job shop environment with an objective of minimizing the makespan. Due to the NP-hard nature of the scheduling problems, it is usually difficult to find an exact optimal schedule and hence one should rely on finding a near to optimal solution. This paper proposes a computationally effective powers-of-two heuristic for solving job shop scheduling problem. The authors prove that the makespan obtained through powers-of-two release dates lies within 6% of the optimal value. The authors also prove the efficacy of powers-of-two approach through mathematical induction.

Sign in / Sign up

Export Citation Format

Share Document