scholarly journals Sparse covariance estimation in logit mixture models

2021 ◽  
Author(s):  
Youssef M Aboutaleb ◽  
Mazen Danaf ◽  
Yifei Xie ◽  
Moshe E Ben-Akiva
Keyword(s):  
Covariance Matrix ◽  
Mixture Models ◽  
Stated Preference ◽  
Synthetic Data ◽  
Covariance Matrices ◽  
Random Coefficients ◽  
Mixed Integer ◽  
Integer Optimization ◽  
Sparsity Level ◽  
Full Covariance Matrix

Abstract This paper introduces a new data-driven methodology for estimating sparse covariance matrices of the random coefficients in logit mixture models. Researchers typically specify covariance matrices in logit mixture models under one of two extreme assumptions: either an unrestricted full covariance matrix (allowing correlations between all random coefficients), or a restricted diagonal matrix (allowing no correlations at all). Our objective is to find optimal subsets of correlated coefficients for which we estimate covariances. We propose a new estimator, called MISC, that uses a mixed-integer optimization (MIO) program to find an optimal block diagonal structure specification for the covariance matrix, corresponding to subsets of correlated coefficients, for any desired sparsity level using Markov Chain Monte Carlo (MCMC) posterior draws from the unrestricted full covariance matrix. The optimal sparsity level of the covariance matrix is determined using out-of-sample validation. We demonstrate the ability of MISC to correctly recover the true covariance structure from synthetic data. In an empirical illustration using a stated preference survey on modes of transportation, we use MISC to obtain a sparse covariance matrix indicating how preferences for attributes are related to one another.

scholarly journals Learning a Mixture of Gaussians via Mixed-Integer Optimization

2019 ◽  
Vol 1 (3) ◽  
pp. 221-240
Author(s):  
Hari Bandi ◽  
Dimitris Bertsimas ◽  
Rahul Mazumder
Keyword(s):  
Distribution Function ◽  
Em Algorithm ◽  
Covariance Matrices ◽  
Mixed Integer ◽  
Multidimensional Data ◽  
Mixture Component ◽  
Data Sets ◽  
Integer Optimization ◽  
Mixed Integer Optimization ◽  
Average Improvement

We consider the problem of estimating the parameters of a multivariate Gaussian mixture model (GMM) given access to n samples that are believed to have come from a mixture of multiple subpopulations. State-of-the-art algorithms used to recover these parameters use heuristics to either maximize the log-likelihood of the sample or try to fit first few moments of the GMM to the sample moments. In contrast, we present here a novel mixed-integer optimization (MIO) formulation that optimally recovers the parameters of the GMM by minimizing a discrepancy measure (either the Kolmogorov–Smirnov or the total variation distance) between the empirical distribution function and the distribution function of the GMM whenever the mixture component weights are known. We also present an algorithm for multidimensional data that optimally recovers corresponding means and covariance matrices. We show that the MIO approaches are practically solvable for data sets with n in the tens of thousands in minutes and achieve an average improvement of 60%–70% and 50%–60% on mean absolute percentage error in estimating the means and the covariance matrices, respectively, over the expectation–maximization (EM) algorithm independent of the sample size n. As the separation of the Gaussians decreases and, correspondingly, the problem becomes more difficult, the edge in performance in favor of the MIO methods widens. Finally, we also show that the MIO methods outperform the EM algorithm with an average improvement of 4%–5% on the out-of-sample accuracy for real-world data sets.

Nonlinear and Mixed-Integer Optimization

1995 ◽  
Author(s):  
Christodoulos A. Floudas
Keyword(s):  
Nonlinear Optimization ◽  
Systems Engineering ◽  
Chemical Engineering ◽  
Applied Mathematics ◽  
Linear Optimization ◽  
Mixed Integer ◽  
Process Synthesis ◽  
Synthesis Process ◽  
Integer Optimization ◽  
Mixed Integer Optimization

Filling a void in chemical engineering and optimization literature, this book presents the theory and methods for nonlinear and mixed-integer optimization, and their applications in the important area of process synthesis. Other topics include modeling issues in process synthesis, and optimization-based approaches in the synthesis of heat recovery systems, distillation-based systems, and reactor-based systems. The basics of convex analysis and nonlinear optimization are also covered and the elementary concepts of mixed-integer linear optimization are introduced. All chapters have several illustrations and geometrical interpretations of the material as well as suggested problems. Nonlinear and Mixed-Integer Optimization will prove to be an invaluable source--either as a textbook or a reference--for researchers and graduate students interested in continuous and discrete nonlinear optimization issues in engineering design, process synthesis, process operations, applied mathematics, operations research, industrial management, and systems engineering.

A mixed-integer optimization approach for homogeneous magnet design

TECHNOLOGY ◽  
2018 ◽  
Vol 06 (02) ◽  
pp. 49-58
Author(s):  
Iman Dayarian ◽  
Timothy C.Y. Chan ◽  
David Jaffray ◽  
Teo Stanescu
Keyword(s):  
Imaging Modality ◽  
Spatial Location ◽  
Mixed Integer ◽  
Optimization Approach ◽  
Integer Optimization ◽  
Magnet Design ◽  
Field Homogeneity ◽  
Feasibility Constraints ◽  
Design Generation ◽  
Magnetic Resonance Imaging Mri

Magnetic resonance imaging (MRI) is a powerful diagnostic tool that has become the imaging modality of choice for soft-tissue visualization in radiation therapy. Emerging technologies aim to integrate MRI with a medical linear accelerator to form novel cancer therapy systems (MR-linac), but the design of these systems to date relies on heuristic procedures. This paper develops an exact, optimization-based approach for magnet design that 1) incorporates the most accurate physics calculations to date, 2) determines precisely the relative spatial location, size, and current magnitude of the magnetic coils, 3) guarantees field homogeneity inside the imaging volume, 4) produces configurations that satisfy, for the first time, small-footprint feasibility constraints required for MR-linacs. Our approach leverages modern mixed-integer programming (MIP), enabling significant flexibility in magnet design generation, e.g., controlling the number of coils and enforcing symmetry between magnet poles. Our numerical results demonstrate the superiority of our method versus current mainstream methods.

Split cuts for robust mixed-integer optimization

2012 ◽  
Vol 40 (3) ◽  
pp. 165-171
Author(s):  
Utz-Uwe Haus ◽  
Frank Pfeuffer
Keyword(s):  
Mixed Integer ◽  
Integer Optimization ◽  
Mixed Integer Optimization ◽  
Split Cuts

scholarly journals Partially distributed outer approximation

2021 ◽  
Author(s):  
Alexander Murray ◽  
Timm Faulwasser ◽  
Veit Hagenmeyer ◽  
Mario E. Villanueva ◽  
Boris Houska
Keyword(s):  
Large Scale ◽  
Optimization Problems ◽  
Outer Approximation ◽  
Mixed Integer ◽  
Global Optimality ◽  
Integer Optimization ◽  
Global Minimizers ◽  
Outer Approximation Method ◽  
Coupling Constraints ◽  
Outer Approximation Algorithm

AbstractThis paper presents a novel partially distributed outer approximation algorithm, named PaDOA, for solving a class of structured mixed integer convex programming problems to global optimality. The proposed scheme uses an iterative outer approximation method for coupled mixed integer optimization problems with separable convex objective functions, affine coupling constraints, and compact domain. PaDOA proceeds by alternating between solving large-scale structured mixed-integer linear programming problems and partially decoupled mixed-integer nonlinear programming subproblems that comprise much fewer integer variables. We establish conditions under which PaDOA converges to global minimizers after a finite number of iterations and verify these properties with an application to thermostatically controlled loads and to mixed-integer regression.

Sign in / Sign up

Export Citation Format

Share Document