scholarly journals A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions

2016 ◽  
Author(s):  
Eric Schulz ◽  
Maarten Speekenbrink ◽  
Andreas Krause
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Gaussian Process Regression ◽  
Risk Averse ◽  
Exploration And Exploitation ◽  
Regression Modelling ◽  
Regression Problems ◽  
Exploration Exploitation ◽  
Psychological Experiments ◽  
Non Parametric

AbstractThis tutorial introduces the reader to Gaussian process regression as an expressive tool to model, actively explore and exploit unknown functions. Gaussian process regression is a powerful, non-parametric Bayesian approach towards regression problems that can be utilized in exploration and exploitation scenarios. This tutorial aims to provide an accessible introduction to these techniques. We will introduce Gaussian processes which generate distributions over functions used for Bayesian non-parametric regression, and demonstrate their use in applications and didactic examples including simple regression problems, a demonstration of kernel-encoded prior assumptions and compositions, a pure exploration scenario within an optimal design framework, and a bandit-like exploration-exploitation scenario where the goal is to recommend movies. Beyond that, we describe a situation modelling risk-averse exploration in which an additional constraint (not to sample below a certain threshold) needs to be accounted for. Lastly, we summarize recent psychological experiments utilizing Gaussian processes. Software and literature pointers are also provided.

scholarly journals Detecting periodicities with Gaussian processes

2016 ◽  
Vol 2 ◽  
pp. e50 ◽  
Author(s):  
Nicolas Durrande ◽  
James Hensman ◽  
Magnus Rattray ◽  
Neil D. Lawrence
Keyword(s):  
Hilbert Space ◽  
Gaussian Process ◽  
Gaussian Processes ◽  
Covariance Function ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Gaussian Process Regression ◽  
Inner Product ◽  
Input Output ◽  
Periodic Data

We consider the problem of detecting and quantifying the periodic component of a function given noise-corrupted observations of a limited number of input/output tuples. Our approach is based on Gaussian process regression, which provides a flexible non-parametric framework for modelling periodic data. We introduce a novel decomposition of the covariance function as the sum of periodic and aperiodic kernels. This decomposition allows for the creation of sub-models which capture the periodic nature of the signal and its complement. To quantify the periodicity of the signal, we derive a periodicity ratio which reflects the uncertainty in the fitted sub-models. Although the method can be applied to many kernels, we give a special emphasis to the Matérn family, from the expression of the reproducing kernel Hilbert space inner product to the implementation of the associated periodic kernels in a Gaussian process toolkit. The proposed method is illustrated by considering the detection of periodically expressed genes in thearabidopsisgenome.

scholarly journals Gaussian Processes for Regression and Optimisation

2021 ◽  
Author(s):  
◽  
Phillip Boyle
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Model Comparison ◽  
Classical Method ◽  
Gaussian Process Regression ◽  
Training Data ◽  
Linear Filter ◽  
Efficient Computation ◽  
Linear Filters ◽  
Reduced Rank

<p>Gaussian processes have proved to be useful and powerful constructs for the purposes of regression. The classical method proceeds by parameterising a covariance function, and then infers the parameters given the training data. In this thesis, the classical approach is augmented by interpreting Gaussian processes as the outputs of linear filters excited by white noise. This enables a straightforward definition of dependent Gaussian processes as the outputs of a multiple output linear filter excited by multiple noise sources. We show how dependent Gaussian processes defined in this way can also be used for the purposes of system identification. Onewell known problem with Gaussian process regression is that the computational complexity scales poorly with the amount of training data. We review one approximate solution that alleviates this problem, namely reduced rank Gaussian processes. We then show how the reduced rank approximation can be applied to allow for the efficient computation of dependent Gaussian processes. We then examine the application of Gaussian processes to the solution of other machine learning problems. To do so, we review methods for the parameterisation of full covariance matrices. Furthermore, we discuss how improvements can be made by marginalising over alternative models, and introduce methods to perform these computations efficiently. In particular, we introduce sequential annealed importance sampling as a method for calculating model evidence in an on-line fashion as new data arrives. Gaussian process regression can also be applied to optimisation. An algorithm is described that uses model comparison between multiple models to find the optimum of a function while taking as few samples as possible. This algorithm shows impressive performance on the standard control problem of double pole balancing. Finally, we describe how Gaussian processes can be used to efficiently estimate gradients of noisy functions, and numerically estimate integrals.</p>

Approximation Bounds for Some Sparse Kernel Regression Algorithms

2002 ◽  
Vol 14 (12) ◽  
pp. 3013-3042 ◽  
Author(s):  
Tong Zhang
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Computational Cost ◽  
Kernel Regression ◽  
Sparse Regression ◽  
Regression Algorithms ◽  
Sparse Approximations ◽  
Regression Problems ◽  
Computationally Expensive ◽  
Sparse Kernel

Gaussian processes have been widely applied to regression problems with good performance. However, they can be computationally expensive. In order to reduce the computational cost, there have been recent studies on using sparse approximations in gaussian processes. In this article, we investigate properties of certain sparse regression algorithms that approximately solve a gaussian process. We obtain approximation bounds and compare our results with related methods.

scholarly journals An additive Gaussian process regression model for interpretable non-parametric analysis of longitudinal data

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Lu Cheng ◽  
Siddharth Ramchandran ◽  
Tommi Vatanen ◽  
Niina Lietzén ◽  
Riitta Lahesmaa ◽  
...  
Keyword(s):  
Longitudinal Data ◽  
Regression Model ◽  
Gaussian Process ◽  
Parametric Analysis ◽  
Gaussian Process Regression ◽  
Non Parametric

scholarly journals Detecting periodicities with Gaussian processes

2016 ◽  
Author(s):  
Nicolas Durrande ◽  
James Hensman ◽  
Magnus Rattray ◽  
Neil D Lawrence
Keyword(s):  
Hilbert Space ◽  
Gaussian Process ◽  
Gaussian Processes ◽  
Covariance Function ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Gaussian Process Regression ◽  
Inner Product ◽  
Input Output ◽  
Periodic Data

We consider the problem of detecting and quantifying the periodic component of a function given noise-corrupted observations of a limited number of input/output tuples. Our approach is based on Gaussian process regression which provides a flexible non-parametric framework for modelling periodic data. We introduce a novel decomposition of the covariance function as the sum of periodic and aperiodic kernels. This decomposition allows for the creation of sub-models which capture the periodic nature of the signal and its complement. To quantify the periodicity of the signal, we derive a periodicity ratio which reflects the uncertainty in the fitted sub-models. Although the method can be applied to many kernels, we give a special emphasis to the Matérn family, from the expression of the reproducing kernel Hilbert space inner product to the implementation of the associated periodic kernels in a Gaussian process toolkit. The proposed method is illustrated by considering the detection of periodically expressed genes in the arabidopsis genome.

scholarly journals Detecting periodicities with Gaussian processes

2016 ◽  
Author(s):  
Nicolas Durrande ◽  
James Hensman ◽  
Magnus Rattray ◽  
Neil D Lawrence
Keyword(s):  
Hilbert Space ◽  
Gaussian Process ◽  
Gaussian Processes ◽  
Covariance Function ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Gaussian Process Regression ◽  
Inner Product ◽  
Input Output ◽  
Periodic Data

We consider the problem of detecting and quantifying the periodic component of a function given noise-corrupted observations of a limited number of input/output tuples. Our approach is based on Gaussian process regression which provides a flexible non-parametric framework for modelling periodic data. We introduce a novel decomposition of the covariance function as the sum of periodic and aperiodic kernels. This decomposition allows for the creation of sub-models which capture the periodic nature of the signal and its complement. To quantify the periodicity of the signal, we derive a periodicity ratio which reflects the uncertainty in the fitted sub-models. Although the method can be applied to many kernels, we give a special emphasis to the Matérn family, from the expression of the reproducing kernel Hilbert space inner product to the implementation of the associated periodic kernels in a Gaussian process toolkit. The proposed method is illustrated by considering the detection of periodically expressed genes in the arabidopsis genome.

scholarly journals The pitfalls of using Gaussian Process Regression for normative modeling

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0252108
Author(s):  
Bohan Xu ◽  
Rayus Kuplicki ◽  
Sandip Sen ◽  
Martin P. Paulus
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Gaussian Process Regression ◽  
Input Domain

Normative modeling, a group of methods used to quantify an individual’s deviation from some expected trajectory relative to observed variability around that trajectory, has been used to characterize subject heterogeneity. Gaussian Processes Regression includes an estimate of variable uncertainty across the input domain, which at face value makes it an attractive method to normalize the cohort heterogeneity where the deviation between predicted value and true observation is divided by the derived uncertainty directly from Gaussian Processes Regression. However, we show that the uncertainty directly from Gaussian Processes Regression is irrelevant to the cohort heterogeneity in general.

scholarly journals Non-parametric synergy modeling of chemical compounds with Gaussian processes

2022 ◽  
Vol 23 (1) ◽  
Author(s):  
Yuliya Shapovalova ◽  
Tom Heskes ◽  
Tjeerd Dijkstra
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Simulated Data ◽  
Parametric Model ◽  
Null Models ◽  
Data Sets ◽  
Hand Model ◽  
The Difference ◽  
Gp Model ◽  
Non Parametric

Abstract Background Understanding the synergetic and antagonistic effects of combinations of drugs and toxins is vital for many applications, including treatment of multifactorial diseases and ecotoxicological monitoring. Synergy is usually assessed by comparing the response of drug combinations to a predicted non-interactive response from reference (null) models. Possible choices of null models are Loewe additivity, Bliss independence and the recently rediscovered Hand model. A different approach is taken by the MuSyC model, which directly fits a generalization of the Hill model to the data. All of these models, however, fit the dose–response relationship with a parametric model. Results We propose the Hand-GP model, a non-parametric model based on the combination of the Hand model with Gaussian processes. We introduce a new logarithmic squared exponential kernel for the Gaussian process which captures the logarithmic dependence of response on dose. From the monotherapeutic response and the Hand principle, we construct a null reference response and synergy is assessed from the difference between this null reference and the Gaussian process fitted response. Statistical significance of the difference is assessed from the confidence intervals of the Gaussian process fits. We evaluate performance of our model on a simulated data set from Greco, two simulated data sets of our own design and two benchmark data sets from Chou and Talalay. We compare the Hand-GP model to standard synergy models and show that our model performs better on these data sets. We also compare our model to the MuSyC model as an example of a recent method on these five data sets and on two-drug combination screens: Mott et al. anti-malarial screen and O’Neil et al. anti-cancer screen. We identify cases in which the HandGP model is preferred and cases in which the MuSyC model is preferred. Conclusion The Hand-GP model is a flexible model to capture synergy. Its non-parametric and probabilistic nature allows it to model a wide variety of response patterns.

scholarly journals Gaussian Processes for Regression and Optimisation

2021 ◽  
Author(s):  
◽  
Phillip Boyle
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Model Comparison ◽  
Classical Method ◽  
Gaussian Process Regression ◽  
Training Data ◽  
Linear Filter ◽  
Efficient Computation ◽  
Linear Filters ◽  
Reduced Rank

<p>Gaussian processes have proved to be useful and powerful constructs for the purposes of regression. The classical method proceeds by parameterising a covariance function, and then infers the parameters given the training data. In this thesis, the classical approach is augmented by interpreting Gaussian processes as the outputs of linear filters excited by white noise. This enables a straightforward definition of dependent Gaussian processes as the outputs of a multiple output linear filter excited by multiple noise sources. We show how dependent Gaussian processes defined in this way can also be used for the purposes of system identification. Onewell known problem with Gaussian process regression is that the computational complexity scales poorly with the amount of training data. We review one approximate solution that alleviates this problem, namely reduced rank Gaussian processes. We then show how the reduced rank approximation can be applied to allow for the efficient computation of dependent Gaussian processes. We then examine the application of Gaussian processes to the solution of other machine learning problems. To do so, we review methods for the parameterisation of full covariance matrices. Furthermore, we discuss how improvements can be made by marginalising over alternative models, and introduce methods to perform these computations efficiently. In particular, we introduce sequential annealed importance sampling as a method for calculating model evidence in an on-line fashion as new data arrives. Gaussian process regression can also be applied to optimisation. An algorithm is described that uses model comparison between multiple models to find the optimum of a function while taking as few samples as possible. This algorithm shows impressive performance on the standard control problem of double pole balancing. Finally, we describe how Gaussian processes can be used to efficiently estimate gradients of noisy functions, and numerically estimate integrals.</p>

Sign in / Sign up

Export Citation Format

Share Document