Feature space maximum a posteriori linear regression for adaptation of deep neural networks

2014 ◽  
Author(s):  
Zhen Huang ◽  
Jinyu Li ◽  
Sabato Marco Siniscalchi ◽  
I-Fan Chen ◽  
Chao Weng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Linear Regression ◽  
Deep Neural Networks ◽  
Feature Space ◽  
Maximum A Posteriori ◽  
A Posteriori

scholarly journals Inter-Class Angular Loss for Convolutional Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 3894-3901 ◽  
Author(s):  
Le Hui ◽  
Xiang Li ◽  
Chen Gong ◽  
Meng Fang ◽  
Joey Tianyi Zhou ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Learning Difficulties ◽  
Feature Space ◽  
Superior Performance ◽  
Strongly Correlated ◽  
Discriminative Ability ◽  
Practical Applications ◽  
Classification Tasks

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.

scholarly journals Effects of Food Contamination on Gastrointestinal Morbidity: Comparison of Different Machine-Learning Methods

2019 ◽  
Vol 16 (5) ◽  
pp. 838 ◽  
Author(s):  
Qin Song ◽  
Yu-Jun Zheng ◽  
Jun Yang
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Linear Regression ◽  
Deep Neural Networks ◽  
Gastrointestinal Disease ◽  
Food Contamination ◽  
Gastrointestinal Diseases ◽  
Central China ◽  
Learning Models

Morbidity prediction can be useful in improving the effectiveness and efficiency of medical services, but accurate morbidity prediction is often difficult because of the complex relationships between diseases and their influencing factors. This study investigates the effects of food contamination on gastrointestinal-disease morbidities using eight different machine-learning models, including multiple linear regression, a shallow neural network, and three deep neural networks and their improved versions trained by an evolutionary algorithm. Experiments on the datasets from ten cities/counties in central China demonstrate that deep neural networks achieve significantly higher accuracy than classical linear-regression and shallow neural-network models, and the deep denoising autoencoder model with evolutionary learning exhibits the best prediction performance. The results also indicate that the prediction accuracies on acute gastrointestinal diseases are generally higher than those on other diseases, but the models are difficult to predict the morbidities of gastrointestinal tumors. This study demonstrates that evolutionary deep-learning models can be utilized to accurately predict the morbidities of most gastrointestinal diseases from food contamination, and this approach can be extended for the morbidity prediction of many other diseases.

scholarly journals New approach to Bayesian high-dimensional linear regression

2018 ◽  
Vol 7 (4) ◽  
pp. 605-655 ◽  
Author(s):  
Shirin Jalali ◽  
Arian Maleki
Keyword(s):  
Linear Regression ◽  
Sampling Rate ◽  
High Dimensional ◽  
Maximum A Posteriori ◽  
Map Estimation ◽  
A Posteriori ◽  
Fundamental Limits ◽  
New Approach ◽  
Estimation Scheme ◽  
Response Variables

Abstract Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. Lack of computationally feasible algorithms that employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. To address this challenge, in this article, a new estimation scheme named quantized maximum a posteriori (Q-MAP) is proposed. The new method has the following properties: (i) In the noiseless setting, it has similarities to maximum a posteriori (MAP) estimation. (ii) In the noiseless setting, when $X_1,\ldots,X_n$ are independent and identically distributed, asymptotically, as $n$ grows to infinity, its required sampling rate ($m/n$) for an almost zero-distortion recovery approaches the fundamental limits. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm that is provably robust to error (noise) in response variables.

Maximum A Posteriori Linear Regression for language recognition

2012 ◽  
Vol 39 (4) ◽  
pp. 4287-4291 ◽  
Author(s):  
Jinchao Yang ◽  
Xiang Zhang ◽  
Hongbin Suo ◽  
Li Lu ◽  
Jianping Zhang ◽  
...  
Keyword(s):  
Linear Regression ◽  
Maximum A Posteriori ◽  
A Posteriori ◽  
Language Recognition
Sign in / Sign up

Export Citation Format

Share Document