Generalization of Construction Object Segmentation Models using Self-Supervised Learning and Copy-Paste Data Augmentation

2021 ◽  
Author(s):  
Yeji Hong ◽  
Wei Chih Chern ◽  
Tam Nguyen ◽  
Hongjo Kim
Keyword(s):  
Supervised Learning ◽  
Data Augmentation ◽  
Object Segmentation ◽  
Segmentation Models

scholarly journals Simplifying the Supervised Learning of Kerr Nonlinearity Compensation Algorithms by Data Augmentation

2020 ◽  
Author(s):  
Vladislav Neskorniuk ◽  
Pedro J. Freire ◽  
Antonio Napoli ◽  
Bernhard Spinnler ◽  
Wolfgang Schairer ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Data Augmentation ◽  
Kerr Nonlinearity ◽  
Nonlinearity Compensation

scholarly journals A review: preprocessing techniques and data augmentation for sentiment analysis

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Huu-Thanh Duong ◽  
Tram-Anh Nguyen-Thi
Keyword(s):  
Machine Learning ◽  
Sentiment Analysis ◽  
Supervised Learning ◽  
Data Augmentation ◽  
Original Data ◽  
Training Data ◽  
Unseen Data ◽  
Augmentation Techniques ◽  
User Intervention

AbstractIn literature, the machine learning-based studies of sentiment analysis are usually supervised learning which must have pre-labeled datasets to be large enough in certain domains. Obviously, this task is tedious, expensive and time-consuming to build, and hard to handle unseen data. This paper has approached semi-supervised learning for Vietnamese sentiment analysis which has limited datasets. We have summarized many preprocessing techniques which were performed to clean and normalize data, negation handling, intensification handling to improve the performances. Moreover, data augmentation techniques, which generate new data from the original data to enrich training data without user intervention, have also been presented. In experiments, we have performed various aspects and obtained competitive results which may motivate the next propositions.

Impact of data augmentation on supervised learning for a moving mid-frequency source

2021 ◽  
Vol 150 (5) ◽  
pp. 3914-3928
Author(s):  
J. A. Castro-Correa ◽  
M. Badiey ◽  
T. B. Neilsen ◽  
D. P. Knobles ◽  
W. S. Hodgkiss
Keyword(s):  
Supervised Learning ◽  
Data Augmentation ◽  
Frequency Source

scholarly journals Self-Supervised Contextual Data Augmentation for Natural Language Processing

Symmetry ◽  
2019 ◽  
Vol 11 (11) ◽  
pp. 1393
Author(s):  
Dongju Park ◽  
Chang Wook Ahn
Keyword(s):  
Supervised Learning ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Question Answering ◽  
Data Augmentation ◽  
Language Model ◽  
Contextual Data ◽  
External Data ◽  
Label Information ◽  
Benchmark Datasets

In this paper, we propose a novel data augmentation method with respect to the target context of the data via self-supervised learning. Instead of looking for the exact synonyms of masked words, the proposed method finds words that can replace the original words considering the context. For self-supervised learning, we can employ the masked language model (MLM), which masks a specific word within a sentence and obtains the original word. The MLM learns the context of a sentence through asymmetrical inputs and outputs. However, without using the existing MLM, we propose a label-masked language model (LMLM) that can include label information for the mask tokens used in the MLM to effectively use the MLM in data with label information. The augmentation method performs self-supervised learning using LMLM and then implements data augmentation through the trained model. We demonstrate that our proposed method improves the classification accuracy of recurrent neural networks and convolutional neural network-based classifiers through several experiments for text classification benchmark datasets, including the Stanford Sentiment Treebank-5 (SST5), the Stanford Sentiment Treebank-2 (SST2), the subjectivity (Subj), the Multi-Perspective Question Answering (MPQA), the Movie Reviews (MR), and the Text Retrieval Conference (TREC) datasets. In addition, since the proposed method does not use external data, it can eliminate the time spent collecting external data, or pre-training using external data.

scholarly journals Graph Self Supervised Learning: the BT, the HSIC, and the VICReg

2021 ◽  
Author(s):  
Sayan Nag
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Loss Function ◽  
Data Augmentation ◽  
Learning Strategy ◽  
Loss Functions ◽  
Augmentation Strategies ◽  
Batch Sizes ◽  
Graph Neural Networks ◽  
The Impact

Self-supervised learning and pre-training strategies have developed over the last few years especially for Convolutional Neural Networks (CNNs). Recently application of such methods can also be noticed for Graph Neural Networks (GNNs). In this paper, we have used a graph based self-supervised learning strategy with different loss functions (Barlow Twins[? ], HSIC[? ], VICReg[? ]) which have shown promising results when applied with CNNs previously. We have also proposed a hybrid loss function combining the advantages of VICReg and HSIC and called it as VICRegHSIC. The performance of these aforementioned methods have been compared when applied to two different datasets namely MUTAG and PROTEINS. Moreover, the impact of different batch sizes, projector dimensions and data augmentation strategies have also been explored. The results are preliminary and we will be continuing to explore with other datasets.

scholarly journals Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Sebastian Otálora ◽  
Niccolò Marini ◽  
Henning Müller ◽  
Manfredo Atzori
Keyword(s):  
Supervised Learning ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Model Performance ◽  
Classification Problem ◽  
Heterogeneous Data ◽  
Fine Tuning ◽  
Data Sources ◽  
Gleason Pattern ◽  
Heterogeneous Data Sources

Abstract Background One challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels. Results As expected, the model performance on strongly annotated data steadily increases as the percentage of strong annotations that are used increases, reaching a performance comparable to pathologists ($$\kappa = 0.691 \pm 0.02$$ κ = 0.691 ± 0.02 ). Nevertheless, the performance sharply decreases when applied for the WSI classification scenario with $$\kappa = 0.307 \pm 0.133$$ κ = 0.307 ± 0.133 . Moreover, it only provides a lower performance regardless of the number of annotations used. The model performance increases when fine-tuning the model for the task of Gleason scoring with the weak WSI labels $$\kappa = 0.528 \pm 0.05$$ κ = 0.528 ± 0.05 . Conclusion Combining weak and strong supervision improves strong supervision in classification of Gleason patterns using tissue microarrays (TMA) and WSI regions. Our results contribute very good strategies for training CNN models combining few annotated data and heterogeneous data sources. The performance increases in the controlled TMA scenario with the number of annotations used to train the model. Nevertheless, the performance is hindered when the trained TMA model is applied directly to the more challenging WSI classification problem. This demonstrates that a good pre-trained model for prostate cancer TMA image classification may lead to the best downstream model if fine-tuned on the WSI target dataset. We have made available the source code repository for reproducing the experiments in the paper: https://github.com/ilmaro8/Digital_Pathology_Transfer_Learning

scholarly journals Dual Parallel Reverse Attention Edge Network : DPRA-EdgeNet

2021 ◽  
Vol 1 (1) ◽  
pp. 8-10
Author(s):  
Debayan Bhattacharya ◽  
Christian Betz ◽  
Dennis Eggert ◽  
Alexander Schlaefer
Keyword(s):  
Cross Validation ◽  
Multiple Scales ◽  
Object Segmentation ◽  
Edge Network ◽  
Validation Experiment ◽  
Segmentation Accuracy ◽  
Fold Cross Validation ◽  
Segmentation Models

In this paper, we propose Dual Parallel Reverse Attention Edge Network (DPRA-EdgeNet), an architecture that jointly learns to segment an object and its edge. Specifically, the model uses two cascaded partial decoders to form two initial estimates of the object segmentation map and its corresponding edge map. This is followed by a series of object decoders and edge decoders which work in conjunction with dual parallel reverse attention modules. The dual parallel reverse attention (DPRA) modules repeatedly prunes the features at multiple scales to put emphasis on the object segmentation and the edge segmentation respectively. Furthermore, we propose a novel decoder block that uses spatial and channel attention to combine features from the preceding decoder block and reverse attention (RA) modules for object and edge segmentation. We compare our model against popular segmentation models such as U-Net, SegNet and PraNet and demonstrate through a five fold cross validation experiment that our model improves the segmentation accuracy significantly on the Kvasir-SEG dataset and Kvasir-Instrument dataset.

Sign in / Sign up

Export Citation Format

Share Document