Fast exact conformalization of the lasso using piecewise linear homotopy

Summary Conformal prediction is a general method that converts almost any point predictor to a prediction set. The resulting set retains the good statistical properties of the original estimator under standard assumptions, and guarantees valid average coverage even when the model is mis-specified. A main challenge in applying conformal prediction in modern applications is efficient computation, as it generally requires an exhaustive search over the entire output space. In this paper we develop an exact and computationally efficient conformalization of the lasso and elastic net. The method makes use of a novel piecewise linear homotopy of the lasso solution under perturbation of a single input sample point. As a by-product, we provide a simpler and better-justified online lasso algorithm, which may be of independent interest. Our derivation also reveals an interesting accuracy-stability trade-off in conformal inference, which is analogous to the bias-variance trade-off in traditional parameter estimation. The practical performance of the new algorithm is demonstrated in both synthetic and real data examples.

A comparative study of the PL homotopy and BFGS methods for some nonsmooth optimization problems

We consider some non-smooth functions and investigate the numerical behavior of the Piecewise Linear Hompotopy (PLH) method ([Bozântan, A., An implementation of the piecewise-linear homotopy algorithm for the computation of fixed points, Creat. Math. Inform., 19 (2010), No.~2, 140–148] and [Bozântan, A. and Berinde, V., Applications of the PL homotopy algorithm for the computation of fixed points to unconstrained optimization problems, Creat. Math. Inform., 22 (2013), No. 1, 41–46]). We compare the PLH method with the BFGS with inexact line search, a quasi-Newton method, having some results reported in [Lewis, A. S. and Overton, M. L., Nonsmooth optimization via BFGS, submitted to SIAM J. Optimiz, (2009)]. For most of the considered cases, the characteristics of the PLH method are quite similar to the BFGS method, that is, the PLH method converges to local minimum values and the convergence rate seems to be linear with respect to the number of function evaluations, but we also identify some issues with the PLH method.

Applications of the PL homotopy algorithm for the computation of fixed points to unconstrained optimization problems

This paper describes the main aspects of the ”piecewise-linear homotopy method” for fixed point approximation proposed by Eaves and Saigal [Eaves, C. B. and Saigal, R., Homotopies for computation of fixed points on unbounded regions, Mathematical Programming, 3 (1972), No. 1, 225–237]. The implementation of the method is developed using the modern programming language C# and then is used for solving some unconstrained optimization problems. The PL homotopy algorithm appears to be more reliable than the classical Newton method in the case of the problem of finding a local minima for Schwefel’s function and other optimization problems.