scholarly journals Learning Transferable Convolutional Proxy by SMI-Based Matching Technique

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Wei Jin ◽  
Nan Jia
Keyword(s):  
Density Ratio ◽  
Domain Model ◽  
Abnormal Behavior ◽  
Classification Error ◽  
Product Reviews ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
Abnormal Behavior Detection ◽  
Domain Transfer

Domain-transfer learning is a machine learning task to explore a source domain data set to help the learning problem in a target domain. Usually, the source domain has sufficient labeled data, while the target domain does not. In this paper, we propose a novel domain-transfer convolutional model by mapping a target domain data sample to a proxy in the source domain and applying a source domain model to the proxy for the purpose of prediction. In our framework, we firstly represent both source and target domains to feature vectors by two convolutional neural networks and then construct a proxy for each target domain sample in the source domain space. The proxy is supposed to be matched to the corresponding target domain sample convolutional representation vector well. To measure the matching quality, we proposed to maximize their squared-loss mutual information (SMI) between the proxy and target domain samples. We further develop a novel neural SMI estimator based on a parametric density ratio estimation function. Moreover, we also propose to minimize the classification error of both source domain samples and target domain proxies. The classification responses are also smoothened by manifolds of both the source domain and proxy space. By minimizing an objective function of SMI, classification error, and manifold regularization, we learn the convolutional networks of both source and target domains. In this way, the proxy of a target domain sample can be matched to the source domain data and thus benefits from the rich supervision information of the source domain. We design an iterative algorithm to update the parameters alternately and test it over benchmark data sets of abnormal behavior detection in video, Amazon product reviews sentiment analysis, etc.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals Multi-Source Deep Transfer Neural Network Algorithm

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3992 ◽  
Author(s):  
Jingmei Li ◽  
Weifei Wu ◽  
Di Xue ◽  
Peng Gao
Keyword(s):  
Neural Network ◽  
Probability Distribution ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Classification Performance ◽  
Classification Error ◽  
Target Domain ◽  
Source Domain ◽  
Network Algorithm ◽  
Neural Network Algorithm

Transfer learning can enhance classification performance of a target domain with insufficient training data by utilizing knowledge relating to the target domain from source domain. Nowadays, it is common to see two or more source domains available for knowledge transfer, which can improve performance of learning tasks in the target domain. However, the classification performance of the target domain decreases due to mismatching of probability distribution. Recent studies have shown that deep learning can build deep structures by extracting more effective features to resist the mismatching. In this paper, we propose a new multi-source deep transfer neural network algorithm, MultiDTNN, based on convolutional neural network and multi-source transfer learning. In MultiDTNN, joint probability distribution adaptation (JPDA) is used for reducing the mismatching between source and target domains to enhance features transferability of the source domain in deep neural networks. Then, the convolutional neural network is trained by utilizing the datasets of each source and target domain to obtain a set of classifiers. Finally, the designed selection strategy selects classifier with the smallest classification error on the target domain from the set to assemble the MultiDTNN framework. The effectiveness of the proposed MultiDTNN is verified by comparing it with other state-of-the-art deep transfer learning on three datasets.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals Deep Transfer Learning for Biology Cross-Domain Image Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Chunfeng Guo ◽  
Bin Wei ◽  
Kun Yu
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Poor Performance ◽  
Target Domain ◽  
Deep Convolutional Neural Networks ◽  
Specific Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.

scholarly journals A Framework for Self-Supervised Federated Domain Adaptation

2021 ◽  
Author(s):  
bin wang ◽  
Gang Li ◽  
Chao Wu ◽  
WeiShan Zhang ◽  
Jiehan Zhou ◽  
...  
Keyword(s):  
Privacy Protection ◽  
Domain Adaptation ◽  
Domain Model ◽  
Multiple Source ◽  
Target Domain ◽  
Adaptation Algorithm ◽  
Source Domain ◽  
Supervised Training ◽  
Model Generalization ◽  
Limited Effectiveness

Abstract Unsupervised federated domain adaptation uses the knowledge from several distributed unlabelled source domains to complete the learning on the unlabelled target domain. Some of the existing methods have limited effectiveness and involve frequent communication. This paper proposes a framework to solve the distributed multi-source domain adaptation problem, referred as self-supervised federated domain adaptation (SFDA). Specifically, a multi-domain model generalization balance (MDMGB) is proposed to aggregate the models from multiple source domains in each round of communication. A weighted strategy based on centroid similarity is also designed for SFDA. SFDA conducts self-supervised training on the target domain to tackle domain shift. Compared with the classical federated adversarial domain adaptation algorithm, SFDA is not only strong in communication cost and privacy protection but also improves in the accuracy of the model.

A working human abnormal operation recognition approach based on the deep multi-instance sorting model

2020 ◽  
pp. 095440622091521
Author(s):  
Liang Chen ◽  
Lu Du ◽  
Qiang Liu
Keyword(s):  
Monitoring System ◽  
Safety Management ◽  
Abnormal Behavior ◽  
Data Set ◽  
Behavior Detection ◽  
Feature Vectors ◽  
Video Feature ◽  
Abnormal Behavior Detection ◽  
Deep Convolution Network ◽  
Abnormal Operation

Modeling abnormal behavior detection in the intelligent video monitoring system for recognition or detection of special event has attracted significant research interest in recent years. In order to achieve more effective recognition of abnormal behavior detection of the intelligent video surveillance system, this paper proposes a working human abnormal operation recognition approach based on deep multi-instance sorting model. Firstly, the uncut long video is sparse to obtain normal and abnormal behavior video segments, and the RGB and optical flow features in the segment are extracted by the deep convolution network. Video feature vectors are obtained by the consensus function and feature extractor. Then, multi-instance sorting learning is used to assign abnormality scores between 0 and 1 to feature vectors. When the abnormal values of abnormal packets are higher than those of normal packets, the abnormality score is returned and the high abnormality score is determined as an abnormal behavior. The experiments on the open THUMOS14 data set and our own XAGCWD data set using CUDA GPU accelerated computing to demonstrate that our approach improves the recognition accuracy about 10.8% and high accuracy of abnormal detection. The main purpose of this research is to apply the model proposed in this paper to the intelligent workshop behavior monitoring system to effectively realize the safety management of workshop personnel.

A novel scheme of domain transfer in document-level cross-domain sentiment classification

2021 ◽  
pp. 016555152110123
Author(s):  
Yueting Lei ◽  
Yanting Li
Keyword(s):  
Language Model ◽  
Sentiment Classification ◽  
Language Models ◽  
Product Reviews ◽  
Data Set ◽  
Emotional Words ◽  
Cross Domain ◽  
Text Ranking ◽  
Domain Transfer

The sentiment classification aims to learn sentiment features from the annotated corpus and automatically predict the sentiment polarity of new sentiment text. However, people have different ways of expressing feelings in different domains. Thus, there are important differences in the characteristics of sentimental distribution across different domains. At the same time, in certain specific domains, due to the high cost of corpus collection, there is no annotated corpus available for the classification of sentiment. Therefore, it is necessary to leverage or reuse existing annotated corpus for training. In this article, we proposed a new algorithm for extracting central sentiment sentences in product reviews, and improved the pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) to achieve the domain transfer for cross-domain sentiment classification. We used various pre-training language models to prove the effectiveness of the newly proposed joint algorithm for text-ranking and emotional words extraction, and utilised Amazon product reviews data set to demonstrate the effectiveness of our proposed domain-transfer framework. The experimental results of 12 different cross-domain pairs showed that the new cross-domain classification method was significantly better than several popular cross-domain sentiment classification methods.

scholarly journals Application of Offshore Visibility Forecast Based on Temporal Convolutional Network and Transfer Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Zhenyu Lu ◽  
Cheng Zheng ◽  
Tingya Yang
Keyword(s):  
Transfer Learning ◽  
Forecast Error ◽  
Model Performance ◽  
Prediction Method ◽  
Small Data ◽  
Data Sets ◽  
Target Domain ◽  
Convolutional Network ◽  
Data Set ◽  
Source Domain

Visibility forecasting in offshore areas faces the problems of low observational data and complex weather. This paper proposes an intelligent prediction method of offshore visibility based on temporal convolutional network (TCN) and transfer learning to solve the problem. First, preprocess the visibility data sets of the source and target domains to improve the quality of the data. Then, build a model based on temporal convolutional network and transfer learning (TCN_TL) to learn the visibility data of the source domain. Finally, after transferring the knowledge learned from a large amount of data in the source domain, the model learns the small data set in the target domain. After completing the training, the model data of the European Mid-Range Weather Forecast Center (ECMWF) meteorological field were selected to test the model performance. The method proposed in this paper has achieved relatively good results in the visibility forecast of Qiongzhou Strait. Taking Haikou Station in the spring and winter of 2018 as an example, the forecast error is significantly lower than that before the transfer learning, and the forecast score is increased by 0.11 within the 0-1 km level and the 24 h forecast period. Compared with the CUACE forecast results, the forecast error of TCN_TL is smaller than that of the former, and the TS score is improved by 0.16. The results show that under the condition of small data sets, transfer learning improves the prediction performance of the model, and TCN_TL performs better than other deep learning methods and CUACE.

scholarly journals Cross-Domain Text Sentiment Analysis Based on CNN_FT Method

Information ◽  
2019 ◽  
Vol 10 (5) ◽  
pp. 162 ◽  
Author(s):  
Jiana Meng ◽  
Yingchun Long ◽  
Yuhai Yu ◽  
Dandan Zhao ◽  
Shuang Liu
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
Cross Domain ◽  
The Cross ◽  
Fully Connected ◽  
Text Sentiment Analysis

Transfer learning is one of the popular methods for solving the problem that the models built on the source domain cannot be directly applied to the target domain in the cross-domain sentiment classification. This paper proposes a transfer learning method based on the multi-layer convolutional neural network (CNN). Interestingly, we construct a convolutional neural network model to extract features from the source domain and share the weights in the convolutional layer and the pooling layer between the source and target domain samples. Next, we fine-tune the weights in the last layer, named the fully connected layer, and transfer the models from the source domain to the target domain. Comparing with the classical transfer learning methods, the method proposed in this paper does not need to retrain the network for the target domain. The experimental evaluation of the cross-domain data set shows that the proposed method achieves a relatively good performance.

scholarly journals Learning Domain-Independent Deep Representations by Mutual Information Minimization

2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Ke Wang ◽  
Jiayong Liu ◽  
Jing-Yan Wang
Keyword(s):  
Mutual Information ◽  
Transfer Learning ◽  
Representation Learning ◽  
Target Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Data Points ◽  
Data Point ◽  
Domain Transfer ◽  
Domain Transfer Learning

Domain transfer learning aims to learn common data representations from a source domain and a target domain so that the source domain data can help the classification of the target domain. Conventional transfer representation learning imposes the distributions of source and target domain representations to be similar, which heavily relies on the characterization of the distributions of domains and the distribution matching criteria. In this paper, we proposed a novel framework for domain transfer representation learning. Our motive is to make the learned representations of data points independent from the domains which they belong to. In other words, from an optimal cross-domain representation of a data point, it is difficult to tell which domain it is from. In this way, the learned representations can be generalized to different domains. To measure the dependency between the representations and the corresponding domain which the data points belong to, we propose to use the mutual information between the representations and the domain-belonging indicators. By minimizing such mutual information, we learn the representations which are independent from domains. We build a classwise deep convolutional network model as a representation model and maximize the margin of each data point of the corresponding class, which is defined over the intraclass and interclass neighborhood. To learn the parameters of the model, we construct a unified minimization problem where the margins are maximized while the representation-domain mutual information is minimized. In this way, we learn representations which are not only discriminate but also independent from domains. An iterative algorithm based on the Adam optimization method is proposed to solve the minimization to learn the classwise deep model parameters and the cross-domain representations simultaneously. Extensive experiments over benchmark datasets show its effectiveness and advantage over existing domain transfer learning methods.

Sign in / Sign up

Export Citation Format

Share Document