scholarly journals Bayesian machine learning for financial modeling

2021 ◽  
Author(s):  
◽  
Rajbir Singh Nirwan
Keyword(s):  
Machine Learning ◽  
Order Statistics ◽  
Gaussian Processes ◽  
Asset Allocation ◽  
Latent Variable ◽  
Nonlinear Models ◽  
Rotational Symmetry ◽  
Variable Model ◽  
Advantages And Disadvantages ◽  
Latent Space

Machine Learning (ML) is so pervasive in our todays life that we don't even realise that, more often than expected, we are using systems based on it. It is also evolving faster than ever before. When deploying ML systems that make decisions on their own, we need to think about their ignorance of our uncertain world. The uncertainty might arise due to scarcity of the data, the bias of the data or even a mismatch between the real world and the ML-model. Given all these uncertainties, we need to think about how to build systems that are not totally ignorant thereof. Bayesian ML can to some extent deal with these problems. The specification of the model using probabilities provides a convenient way to quantify uncertainties, which can then be included in the decision making process. In this thesis, we introduce the Bayesian ansatz to modeling and apply Bayesian ML models in finance and economics. Especially, we will dig deeper into Gaussian processes (GP) and Gaussian process latent variable model (GPLVM). Applied to the returns of several assets, GPLVM provides the covariance structure and also a latent space embedding thereof. Several financial applications can be build upon the output of the GPLVM. To demonstrate this, we build an automated asset allocation system, a predictor for missing asset prices and identify other structure in financial data. It turns out that the GPLVM exhibits a rotational symmetry in the latent space, which makes it harder to fit. Our second publication reports, how to deal with that symmetry. We propose another parameterization of the model using Householder transformations, by which the symmetry is broken. Bayesian models are changed by reparameterization, if the prior is not changed accordingly. We provide the correct prior distribution of the new parameters, such that the model, i.e. the data density, is not changed under the reparameterization. After applying the reparametrization on Bayesian PCA, we show that the symmetry of nonlinear models can also be broken in the same way. In our last project, we propose a new method for matching quantile observations, which uses order statistics. The use of order statistics as the likelihood, instead of a Gaussian likelihood, has several advantages. We compare these two models and highlight their advantages and disadvantages. To demonstrate our method, we fit quantiled salary data of several European countries. Given several candidate models for the fit, our method also provides a metric to choose the best option. We hope that this thesis illustrates some benefits of Bayesian modeling (especially Gaussian processes) in finance and economics and its usage when uncertainties are to be quantified.

scholarly journals Gaussian Processes Proxy Model with Latent Variable Models and Variogram-Based Sensitivity Analysis for Assisted History Matching

Energies ◽  
2020 ◽  
Vol 13 (17) ◽  
pp. 4290
Author(s):  
Dongmei Zhang ◽  
Yuyang Zhang ◽  
Bohou Jiang ◽  
Xinwei Jiang ◽  
Zhijiang Kang
Keyword(s):  
Sensitivity Analysis ◽  
Gaussian Processes ◽  
Latent Variable ◽  
History Matching ◽  
Latent Variable Models ◽  
High Dimensional ◽  
Model Parameters ◽  
Variable Model ◽  
Assisted History Matching ◽  
Proxy Models

Reservoir history matching is a well-known inverse problem for production prediction where enormous uncertain reservoir parameters of a reservoir numerical model are optimized by minimizing the misfit between the simulated and history production data. Gaussian Process (GP) has shown promising performance for assisted history matching due to the efficient nonparametric and nonlinear model with few model parameters to be tuned automatically. Recently introduced Gaussian Processes proxy models and Variogram Analysis of Response Surface-based sensitivity analysis (GP-VARS) uses forward and inverse Gaussian Processes (GP) based proxy models with the VARS-based sensitivity analysis to optimize the high-dimensional reservoir parameters. However, the inverse GP solution (GPIS) in GP-VARS are unsatisfactory especially for enormous reservoir parameters where the mapping from low-dimensional misfits to high-dimensional uncertain reservoir parameters could be poorly modeled by GP. To improve the performance of GP-VARS, in this paper we propose the Gaussian Processes proxy models with Latent Variable Models and VARS-based sensitivity analysis (GPLVM-VARS) where Gaussian Processes Latent Variable Model (GPLVM)-based inverse solution (GPLVMIS) instead of GP-based GPIS is provided with the inputs and outputs of GPIS reversed. The experimental results demonstrate the effectiveness of the proposed GPLVM-VARS in terms of accuracy and complexity. The source code of the proposed GPLVM-VARS is available at https://github.com/XinweiJiang/GPLVM-VARS.

scholarly journals Machine Learning for Non-Intrusive Speech Quality Assessment

2021 ◽  
Author(s):  
◽  
Mouna Hakami
Keyword(s):  
Machine Learning ◽  
Quality Assessment ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Latent Variable ◽  
Generative Models ◽  
Speech Quality ◽  
Speech Signals ◽  
Latent Space ◽  
Speech Quality Assessment

<p><b>This thesis presents two studies on non-intrusive speech quality assessment methods. The first applies supervised learning methods to speech quality assessment, which is a common approach in machine learning based quality assessment. To outperform existing methods, we concentrate on enhancing the feature set. In the second study, we analyse quality assessment from a different point of view inspired by the biological brain and present the first unsupervised learning based non-intrusive quality assessment that removes the need for labelled training data.</b></p> <p>Supervised learning based, non-intrusive quality predictors generally involve the development of a regressor that maps signal features to a representation of perceived quality. The performance of the predictor largely depends on 1) how sensitive the features are to the different types of distortion, and 2) how well the model learns the relation between the features and the quality score. We improve the performance of the quality estimation by enhancing the feature set and using a contemporary machine learning model that fits this objective. We propose an augmented feature set that includes raw features that are presumably redundant. The speech quality assessment system benefits from this redundancy as it results in reducing the impact of unwanted noise in the input. Feature set augmentation generally leads to the inclusion of features that have non-smooth distributions. We introduce a new pre-processing method and re-distribute the features to facilitate the training. The evaluation of the system on the ITU-T Supplement23 database illustrates that the proposed system outperforms the popular standards and contemporary methods in the literature.</p> <p>The unsupervised learning quality assessment approach presented in this thesis is based on a model that is learnt from clean speech signals. Consequently, it does not need to learn the statistics of any corruption that exists in the degraded speech signals and is trained only with unlabelled clean speech samples. The quality has a new definition, which is based on the divergence between 1) the distribution of the spectrograms of test signals, and 2) the pre-existing model that represents the distribution of the spectrograms of good quality speech. The distribution of the spectrogram of the speech is complex, and hence comparing them is not trivial. To tackle this problem, we propose to map the spectrograms of speech signals to a simple latent space.</p> <p>Generative models that map simple latent distributions into complex distributions are excellent platforms for our work. Generative models that are trained on the spectrograms of clean speech signals learned to map the latent variable $Z$ from a simple distribution $P_Z$ into a spectrogram $X$ from the distribution of good quality speech.</p> <p>Consequently, an inference model is developed by inverting the pre-trained generator, which maps spectrograms of the signal under the test, $X_t$, into its relevant latent variable, $Z_t$, in the latent space. We postulate the divergence between the distribution of the latent variable and the prior distribution $P_Z$ is a good measure of the quality of speech.</p> <p>Generative adversarial nets (GAN) are an effective training method and work well in this application. The proposed system is a novel application for a GAN. The experimental results with the TIMIT and NOIZEUS databases show that the proposed measure correlates positively with the objective quality scores.</p>

scholarly journals Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors (IDETC2021-71570)

2021 ◽  
pp. 1-36
Author(s):  
Liwei Wang ◽  
Suraj Yerramilli ◽  
Akshay Iyer ◽  
Daniel Apley ◽  
Ping Zhu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Gaussian Processes ◽  
Building Blocks ◽  
Variational Inference ◽  
Data Driven ◽  
Ternary Oxide ◽  
Latent Space ◽  
Multiple Materials ◽  
Qualitative Factors ◽  
Gp Model

Abstract Scientific and engineering problems often require the use of artificial intelligence to aid understanding and the search for promising designs. While Gaussian processes (GP) stand out as easy-to-use and interpretable learners, they have difficulties in accommodating big datasets, qualitative inputs, and multi-type responses obtained from different simulators, which has become a common challenge for data-driven design applications. In this paper, we propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. The method is built upon the latent variable Gaussian process (LVGP) model where qualitative factors are mapped into a continuous latent space to enable GP modeling of mixed-variable datasets. By extending variational inference to LVGP models, the large training dataset is replaced by a small set of inducing points to address the scalability issue. Output response vectors are represented by a linear combination of independent latent functions, forming a flexible kernel structure to handle multi-type responses. Comparative studies demonstrate that the proposed method scales well for large datasets, while outperforming state-of-the-art machine learning methods without requiring much hyperparameter tuning. In addition, an interpretable latent space is obtained to draw insights into the effect of qualitative factors, such as those associated with “building blocks” of architectures and element choices in metamaterial and materials design. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism with aperiodic microstructures and multiple materials.

scholarly journals Extracting Lexically Divergent Paraphrases from Twitter

2014 ◽  
Vol 2 ◽  
pp. 435-448 ◽  
Author(s):  
Wei Xu ◽  
Alan Ritter ◽  
Chris Callison-Burch ◽  
William B. Dolan ◽  
Yangfeng Ji
Keyword(s):  
Latent Variable ◽  
State Of The Art ◽  
Research Community ◽  
Latent Variable Model ◽  
Variable Model ◽  
New Model ◽  
Space Model ◽  
Sentence Level ◽  
Latent Space ◽  
Feature Based

We present MultiP (Multi-instance Learning Paraphrase Model), a new model suited to identify paraphrases within the short messages on Twitter. We jointly model paraphrase relations between word and sentence pairs and assume only sentence-level annotations during learning. Using this principled latent variable model alone, we achieve the performance competitive with a state-of-the-art method which combines a latent space model with a feature-based supervised classifier. Our model also captures lexically divergent paraphrases that differ from yet complement previous methods; combining our model with previous work significantly outperforms the state-of-the-art. In addition, we present a novel annotation methodology that has allowed us to crowdsource a paraphrase corpus from Twitter. We make this new dataset available to the research community.

scholarly journals Machine Learning for Non-Intrusive Speech Quality Assessment

2021 ◽  
Author(s):  
◽  
Mouna Hakami
Keyword(s):  
Machine Learning ◽  
Quality Assessment ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Latent Variable ◽  
Generative Models ◽  
Speech Quality ◽  
Speech Signals ◽  
Latent Space ◽  
Speech Quality Assessment

<p><b>This thesis presents two studies on non-intrusive speech quality assessment methods. The first applies supervised learning methods to speech quality assessment, which is a common approach in machine learning based quality assessment. To outperform existing methods, we concentrate on enhancing the feature set. In the second study, we analyse quality assessment from a different point of view inspired by the biological brain and present the first unsupervised learning based non-intrusive quality assessment that removes the need for labelled training data.</b></p> <p>Supervised learning based, non-intrusive quality predictors generally involve the development of a regressor that maps signal features to a representation of perceived quality. The performance of the predictor largely depends on 1) how sensitive the features are to the different types of distortion, and 2) how well the model learns the relation between the features and the quality score. We improve the performance of the quality estimation by enhancing the feature set and using a contemporary machine learning model that fits this objective. We propose an augmented feature set that includes raw features that are presumably redundant. The speech quality assessment system benefits from this redundancy as it results in reducing the impact of unwanted noise in the input. Feature set augmentation generally leads to the inclusion of features that have non-smooth distributions. We introduce a new pre-processing method and re-distribute the features to facilitate the training. The evaluation of the system on the ITU-T Supplement23 database illustrates that the proposed system outperforms the popular standards and contemporary methods in the literature.</p> <p>The unsupervised learning quality assessment approach presented in this thesis is based on a model that is learnt from clean speech signals. Consequently, it does not need to learn the statistics of any corruption that exists in the degraded speech signals and is trained only with unlabelled clean speech samples. The quality has a new definition, which is based on the divergence between 1) the distribution of the spectrograms of test signals, and 2) the pre-existing model that represents the distribution of the spectrograms of good quality speech. The distribution of the spectrogram of the speech is complex, and hence comparing them is not trivial. To tackle this problem, we propose to map the spectrograms of speech signals to a simple latent space.</p> <p>Generative models that map simple latent distributions into complex distributions are excellent platforms for our work. Generative models that are trained on the spectrograms of clean speech signals learned to map the latent variable $Z$ from a simple distribution $P_Z$ into a spectrogram $X$ from the distribution of good quality speech.</p> <p>Consequently, an inference model is developed by inverting the pre-trained generator, which maps spectrograms of the signal under the test, $X_t$, into its relevant latent variable, $Z_t$, in the latent space. We postulate the divergence between the distribution of the latent variable and the prior distribution $P_Z$ is a good measure of the quality of speech.</p> <p>Generative adversarial nets (GAN) are an effective training method and work well in this application. The proposed system is a novel application for a GAN. The experimental results with the TIMIT and NOIZEUS databases show that the proposed measure correlates positively with the objective quality scores.</p>

scholarly journals Estimating Party Positions across Countries and Time—A Dynamic Latent Variable Model for Manifesto Data

2013 ◽  
Vol 21 (4) ◽  
pp. 468-491 ◽  
Author(s):  
Thomas König ◽  
Moritz Marbach ◽  
Moritz Osnabrügge
Keyword(s):  
Political Parties ◽  
Latent Variable ◽  
Latent Variable Model ◽  
Systematic Bias ◽  
Variable Model ◽  
Party Positions ◽  
Ex Post ◽  
Latent Space ◽  
Ex Post Facto ◽  
Time Specific

This article presents a new method for estimating positions of political parties across country- and time-specific contexts by introducing a latent variable model for manifesto data. We estimate latent positions and exploit bridge observations to make the scales comparable. We also incorporate expert survey data as prior information in the estimation process to avoid ex post facto interpretation of the latent space. To illustrate the empirical contribution of our method, we estimate the left-right positions of 388 parties competing in 238 elections across twenty-five countries and over sixty years. Compared to the puzzling volatility of existing estimates, we find that parties more modestly change their left-right positions over time. We also show that estimates without country- and time-specific bias parameters risk serious, systematic bias in about two-thirds of our data. This suggests that researchers should carefully consider the comparability of party positions across countries and/or time.

scholarly journals GTM: The Generative Topographic Mapping

1998 ◽  
Vol 10 (1) ◽  
pp. 215-234 ◽  
Author(s):  
Christopher M. Bishop ◽  
Markus Svensén ◽  
Christopher K. I. Williams
Keyword(s):  
Latent Variable ◽  
Hidden Variables ◽  
Expectation Maximization Algorithm ◽  
Simulated Data ◽  
Topographic Mapping ◽  
Self Organizing Map ◽  
Generative Topographic Mapping ◽  
Variable Model ◽  
Oil Pipeline ◽  
Latent Space

Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis, which is based on a linear transformation between the latent space and the data space. In this article, we introduce a form of nonlinear latent variable model called the generative topographic mapping, for which the parameters of the model can be determined using the expectation-maximization algorithm. GTM provides a principled alternative to the widely used self-organizing map (SOM) of Kohonen (1982) and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multiphase oil pipeline.

scholarly journals A Latent Variable Model Approach to PMI-based Word Embeddings

2016 ◽  
Vol 4 ◽  
pp. 385-399 ◽  
Author(s):  
Sanjeev Arora ◽  
Yuanzhi Li ◽  
Yingyu Liang ◽  
Tengyu Ma ◽  
Andrej Risteski
Keyword(s):  
Latent Variable ◽  
Topic Model ◽  
Nonlinear Models ◽  
Generative Model ◽  
Word Embeddings ◽  
Variable Model ◽  
Dynamic Version ◽  
Low Dimensional ◽  
Log Linear ◽  
Model Approach

Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods. Many use nonlinear operations on co-occurrence statistics, and have hand-tuned hyperparameters and reweighting methods. This paper proposes a new generative model, a dynamic version of the log-linear topic model of Mnih and Hinton (2007). The methodological novelty is to use the prior to compute closed form expressions for word statistics. This provides a theoretical justification for nonlinear models like PMI, word2vec, and GloVe, as well as some hyperparameter choices. It also helps explain why low-dimensional semantic embeddings contain linear algebraic structure that allows solution of word analogies, as shown by Mikolov et al. (2013a) and many subsequent papers. Experimental support is provided for the generative model assumptions, the most important of which is that latent word vectors are fairly uniformly dispersed in space.

scholarly journals Student ability assessment based on two IRT models

2005 ◽  
Vol 2 (2) ◽  
Author(s):  
Silvia Cagnone ◽  
Roberto Ricci
Keyword(s):  
Latent Variable ◽  
Real Data ◽  
Partial Credit Model ◽  
Theoretical Frameworks ◽  
Variable Model ◽  
Data Set ◽  
Advantages And Disadvantages ◽  
Graded Response ◽  
Student Ability ◽  
Problem Solving Process

The aim of this work is to analyze a part of the data collected in the Computer Science Department during the Informatics exams in the year 2003. Two different Item Response Theory models for ordered polytomous variables are considered in order to get an evaluation of student ability. Ordered polytomous variables are used for a problem solving process that contains a finite number of steps so that the ability of a student can be evaluated on the basis of the step achieved, namely, higher steps achieved are related to higher ability. The models considered are the Partial Credit Model and the Graded Response Model. The choice of these models has been dictated by the fact that although they are defined into different theoretical frameworks, the former belongs to the Rasch family (Masters, 1982) and the latter can be viewed as a Generalized Linear Latent Variable Model (Bartholomew and Knott, 1999), and hence they present different properties, both of them allow to treat ordinal observed variables. The analysis of the real data set through the two approaches allows to highlight their advantages and disadvantages.

Sign in / Sign up

Export Citation Format

Share Document