scholarly journals Learning Nonlinear Brain Dynamics: van der Pol Meets LSTM

2018 ◽  
Author(s):  
Germán Abrevaya ◽  
Aleksandr Aravkin ◽  
Guillermo Cecchi ◽  
Irina Rish ◽  
Pablo Polosecki ◽  
...  
Keyword(s):  
Large Scale ◽  
Data Augmentation ◽  
Predictive Accuracy ◽  
Training Data ◽  
Van Der Pol Oscillator ◽  
Brain Dynamics ◽  
Optimization Approach ◽  
Imaging Data ◽  
Van Der Pol ◽  
Temporal Models

AbstractMany real-world data sets, especially in biology, are produced by highly multivariate and nonlinear complex dynamical systems. In this paper, we focus on brain imaging data, including both calcium imaging and functional MRI data. Standard vector-autoregressive models are limited by their linearity assumptions, while nonlinear general-purpose, large-scale temporal models, such as LSTM networks, typically require large amounts of training data, not always readily available in biological applications; furthermore, such models have limited interpretability. We introduce here a novel approach for learning a nonlinear differential equation model aimed at capturing brain dynamics. Specifically, we propose a variable-projection optimization approach to estimate the parameters of the multivariate (coupled) van der Pol oscillator, and demonstrate that such a model can accurately represent nonlinear dynamics of the brain data. Furthermore, in order to improve the predictive accuracy when forecasting future brain-activity time series, we use this analytical model as an unlimited source of simulated data for pretraining LSTM; such model-specific data augmentation approach consistently improves LSTM performance on both calcium and fMRI imaging data.

scholarly journals Gravity Control-Based Data Augmentation Technique for Improving VR User Activity Recognition

Symmetry ◽  
2021 ◽  
Vol 13 (5) ◽  
pp. 845
Author(s):  
Dongheun Han ◽  
Chulwoo Lee ◽  
Hyeongyeop Kang
Keyword(s):  
Activity Recognition ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Data ◽  
Measurement Unit ◽  
Gravitational Acceleration ◽  
The Neural Network ◽  
Typical Data ◽  
Robust Recognition ◽  
Gravity Acceleration

The neural-network-based human activity recognition (HAR) technique is being increasingly used for activity recognition in virtual reality (VR) users. The major issue of a such technique is the collection large-scale training datasets which are key for deriving a robust recognition model. However, collecting large-scale data is a costly and time-consuming process. Furthermore, increasing the number of activities to be classified will require a much larger number of training datasets. Since training the model with a sparse dataset can only provide limited features to recognition models, it can cause problems such as overfitting and suboptimal results. In this paper, we present a data augmentation technique named gravity control-based augmentation (GCDA) to alleviate the sparse data problem by generating new training data based on the existing data. The benefits of the symmetrical structure of the data are that it increased the number of data while preserving the properties of the data. The core concept of GCDA is two-fold: (1) decomposing the acceleration data obtained from the inertial measurement unit (IMU) into zero-gravity acceleration and gravitational acceleration, and augmenting them separately, and (2) exploiting gravity as a directional feature and controlling it to augment training datasets. Through the comparative evaluations, we validated that the application of GCDA to training datasets showed a larger improvement in classification accuracy (96.39%) compared to the typical data augmentation methods (92.29%) applied and those that did not apply the augmentation method (85.21%).

scholarly journals Attention-Aware Adversarial Network for Person Re-Identification

2019 ◽  
Vol 9 (8) ◽  
pp. 1550 ◽  
Author(s):  
Aihong Shen ◽  
Huasheng Wang ◽  
Junjie Wang ◽  
Hongchen Tan ◽  
Xiuping Liu ◽  
...  
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Data Augmentation ◽  
Fundamental Problem ◽  
Training Data ◽  
Specific Data ◽  
Training Strategy ◽  
Adversarial Network ◽  
Benchmark Datasets ◽  
Adversarial Training

Person re-identification (re-ID) is a fundamental problem in the field of computer vision. The performance of deep learning-based person re-ID models suffers from a lack of training data. In this work, we introduce a novel image-specific data augmentation method on the feature map level to enforce feature diversity in the network. Furthermore, an attention assignment mechanism is proposed to enforce that the person re-ID classifier focuses on nearly all important regions of the input person image. To achieve this, a three-stage framework is proposed. First, a baseline classification network is trained for person re-ID. Second, an attention assignment network is proposed based on the baseline network, in which the attention module learns to suppress the response of the current detected regions and re-assign attentions to other important locations. By this means, multiple important regions for classification are highlighted by the attention map. Finally, the attention map is integrated in the attention-aware adversarial network (AAA-Net), which generates high-performance classification results with an adversarial training strategy. We evaluate the proposed method on two large-scale benchmark datasets, including Market1501 and DukeMTMC-reID. Experimental results show that our algorithm performs favorably against the state-of-the-art methods.

Data set entity recognition based on distant supervision

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Pengcheng Li ◽  
Qikai Liu ◽  
Qikai Cheng ◽  
Wei Lu
Keyword(s):  
Supervised Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Scientific Literature ◽  
Neural Model ◽  
Training Data ◽  
Entity Recognition ◽  
Data Set ◽  
Content Type ◽  
Augmentation Techniques

Purpose This paper aims to identify data set entities in scientific literature. To address poor recognition caused by a lack of training corpora in existing studies, a distant supervised learning-based approach is proposed to identify data set entities automatically from large-scale scientific literature in an open domain. Design/methodology/approach Firstly, the authors use a dictionary combined with a bootstrapping strategy to create a labelled corpus to apply supervised learning. Secondly, a bidirectional encoder representation from transformers (BERT)-based neural model was applied to identify data set entities in the scientific literature automatically. Finally, two data augmentation techniques, entity replacement and entity masking, were introduced to enhance the model generalisability and improve the recognition of data set entities. Findings In the absence of training data, the proposed method can effectively identify data set entities in large-scale scientific papers. The BERT-based vectorised representation and data augmentation techniques enable significant improvements in the generality and robustness of named entity recognition models, especially in long-tailed data set entity recognition. Originality/value This paper provides a practical research method for automatically recognising data set entities in scientific literature. To the best of the authors’ knowledge, this is the first attempt to apply distant learning to the study of data set entity recognition. The authors introduce a robust vectorised representation and two data augmentation strategies (entity replacement and entity masking) to address the problem inherent in distant supervised learning methods, which the existing research has mostly ignored. The experimental results demonstrate that our approach effectively improves the recognition of data set entities, especially long-tailed data set entities.

scholarly journals Current Trends and Future Directions of Large Scale Image and Video Annotation: Observations From Four Years of BIIGLE 2.0

2021 ◽  
Vol 8 ◽  
Author(s):  
Martin Zurowietz ◽  
Tim W. Nattkemper
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Image Annotation ◽  
Training Data ◽  
Video Annotation ◽  
Computer Assisted ◽  
Marine Science ◽  
Imaging Data ◽  
Video Frames ◽  
Increasing Demand

Marine imaging has evolved from small, narrowly focussed applications to large-scale applications covering areas of several hundred square kilometers or time series covering observation periods of several months. The analysis and interpretation of the accumulating large volume of digital images or videos will continue to challenge the marine science community to keep this process efficient and effective. It is safe to say that any strategy will rely on some software platform supporting manual image and video annotation, either for a direct manual annotation-based analysis or for collecting training data to deploy a machine learning–based approach for (semi-)automatic annotation. This paper describes how computer-assisted manual full-frame image and video annotation is currently performed in marine science and how it can evolve to keep up with the increasing demand for image and video annotation and the growing volume of imaging data. As an example, observations are presented how the image and video annotation tool BIIGLE 2.0 has been used by an international community of more than one thousand users in the last 4 years. In addition, new features and tools are presented to show how BIIGLE 2.0 has evolved over the same time period: video annotation, support for large images in the gigapixel range, machine learning assisted image annotation, improved mobility and affordability, application instance federation and enhanced label tree collaboration. The observations indicate that, despite novel concepts and tools introduced by BIIGLE 2.0, full-frame image and video annotation is still mostly done in the same way as two decades ago, where single users annotated subsets of image collections or single video frames with limited computational support. We encourage researchers to review their protocols for education and annotation, making use of newer technologies and tools to improve the efficiency and effectivity of image and video annotation in marine science.

scholarly journals Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network

Sensors ◽  
2020 ◽  
Vol 20 (6) ◽  
pp. 1562 ◽  
Author(s):  
Xiaoming Lv ◽  
Fajie Duan ◽  
Jia-jia Jiang ◽  
Xiao Fu ◽  
Lin Gan
Keyword(s):  
Defect Detection ◽  
Large Scale ◽  
Surface Defect ◽  
Data Augmentation ◽  
Training Data ◽  
Metallic Surface ◽  
Single Shot ◽  
Limited Data ◽  
Detection Model ◽  
Surface Defect Detection

Metallic surface defect detection is an essential and necessary process to control the qualities of industrial products. However, due to the limited data scale and defect categories, existing defect datasets are generally unavailable for the deployment of the detection model. To address this problem, we contribute a new dataset called GC10-DET for large-scale metallic surface defect detection. The GC10-DET dataset has great challenges on defect categories, image number, and data scale. Besides, traditional detection approaches are poor in both efficiency and accuracy for the complex real-world environment. Thus, we also propose a novel end-to-end defect detection network (EDDN) based on the Single Shot MultiBox Detector. The EDDN model can deal with defects with different scales. Furthermore, a hard negative mining method is designed to alleviate the problem of data imbalance, while some data augmentation methods are adopted to enrich the training data for the expensive data collection problem. Finally, the extensive experiments on two datasets demonstrate that the proposed method is robust and can meet accuracy requirements for metallic defect detection.

scholarly journals Improving Object Tracking by Added Noise and Channel Attention

Sensors ◽  
2020 ◽  
Vol 20 (13) ◽  
pp. 3780 ◽  
Author(s):  
Mustansar Fiaz ◽  
Arif Mahmood ◽  
Ki Yeol Baek ◽  
Sehar Shahzad Farooq ◽  
Soon Ki Jung
Keyword(s):  
Large Scale ◽  
Data Augmentation ◽  
Feature Fusion ◽  
State Of The Art ◽  
Computational Cost ◽  
Training Data ◽  
Superior Performance ◽  
Input Noise ◽  
Offline Learning ◽  
Benchmark Datasets

CNN-based trackers, especially those based on Siamese networks, have recently attracted considerable attention because of their relatively good performance and low computational cost. For many Siamese trackers, learning a generic object model from a large-scale dataset is still a challenging task. In the current study, we introduce input noise as regularization in the training data to improve generalization of the learned model. We propose an Input-Regularized Channel Attentional Siamese (IRCA-Siam) tracker which exhibits improved generalization compared to the current state-of-the-art trackers. In particular, we exploit offline learning by introducing additive noise for input data augmentation to mitigate the overfitting problem. We propose feature fusion from noisy and clean input channels which improves the target localization. Channel attention integrated with our framework helps finding more useful target features resulting in further performance improvement. Our proposed IRCA-Siam enhances the discrimination of the tracker/background and improves fault tolerance and generalization. An extensive experimental evaluation on six benchmark datasets including OTB2013, OTB2015, TC128, UAV123, VOT2016 and VOT2017 demonstrate superior performance of the proposed IRCA-Siam tracker compared to the 30 existing state-of-the-art trackers.

Crystallographic Symmetry for Data Augmentation in Detecting Dendrite Cores

2020 ◽  
Vol 2020 (14) ◽  
pp. 248-1-248-7
Author(s):  
Lan Fu ◽  
Hongkai Yu ◽  
Megna Shah ◽  
Jeff Simmons ◽  
Song Wang
Keyword(s):  
Cross Sections ◽  
Large Scale ◽  
Data Augmentation ◽  
Ground Truth ◽  
Training Data ◽  
Control Procedure ◽  
Detection Accuracy ◽  
Crystallographic Symmetry ◽  
Augmentation Strategy ◽  
Series Of Experiments

Accurately and rapidly detecting the locations of the cores of large-scale dendrites from 2D sectioned microscopic images helps quantify the microstructure of material components. This provides a critical link between the processing and properties of the material. Such a tool could be a critical part of a quality control procedure for manufacturing these components. In this paper, we propose to use Faster R-CNN, a convolutional neural network (CNN) model that considers both the detection accuracy and computational efficiency, to detect the dendrite cores with complex shapes. However, training CNN models usually requires a large number of images annotated with ground-truth locations of dendrite cores, which are usually obtained by highly laborintensive manual annotations. In this paper, we leverage the crystallographic symmetry of dendrite cores for data augmentation – the cross sections of dendrite cores show, not perfect, but near four-fold rotation symmetry and we can rotate the image around the center of dendrite cores by specified angles to construct new training data without additional manual annotations. We conduct a series of experiments and the results show the effectiveness of the Faster R-CNN method with the proposed data augmentation strategy. Particularly, we find that we can reduce the number of the manually annotated training images by 75% while still maintaining the same detection accuracy of dendrite cores.

scholarly journals A robust speaker recognition based on Data augmentation

2020 ◽  
Author(s):  
Chenqi Li ◽  
Haoyuan Lu ◽  
Wei Wang
Keyword(s):  
Performance Improvement ◽  
Speaker Recognition ◽  
Large Scale ◽  
Data Augmentation ◽  
Environmental Noise ◽  
Experimental Results ◽  
Training Data ◽  
Robust Speaker Recognition

Abstract It is known that large-scale training data can get the better effect of recognition. However, it is difficult to collect a lot of labeled training data for speaker recognition. At the same time, the performance of speaker recognition is greatly influenced by environmental noise. In this paper, we use data augmentation by adding noise to get much training data and improve the robustness of speaker recognition. The experimental results demonstrate that data augmentation have the better performance improvement on Chinese-863 database.

scholarly journals Deep Learning-Aided Diagnosis of Autoimmune Blistering Diseases

2021 ◽  
Author(s):  
Daniel Cai ◽  
Abbas Roayaei Ardakany ◽  
Ferhat Ay
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Skin Diseases ◽  
Diagnostic Delay ◽  
Clinical Manifestations ◽  
Training Data ◽  
Fine Tuning ◽  
Imaging Data ◽  
Improve Performance ◽  
Autoimmune Blistering Diseases

Autoimmune blistering diseases (AIBDs) are rare, chronic disorders of the skin and mucous membranes, with a broad spectrum of clinical manifestations and morphological lesions. Considering that 1) diagnosis of AIBDs is a challenging task, owing to their rarity and heterogeneous clinical features, and 2) misdiagnoses are common, and the resulting diagnostic delay is a major factor in their high mortality rate, patient prognosis stands to benefit greatly from the development of a computer-aided diagnostic (CAD) tool for AIBDs. Artificial intelligence (AI) research into rare skin diseases like AIBDs is severely underrepresented, due to a variety of factors, foremost a lack of large-scale, uniformly curated imaging data. A study by Julia S. et al. finds that, as of 2020, there exists no machine learning studies on rare skin diseases [1], despite the demonstrated success of AI in the field of dermatology. Whereas previous research has primarily looked to improve performance through extensive data collection and preprocessing, this approach remains tedious and impractical for rarer, under-documented skin diseases. This study proposes a novel approach in the development of a deep learning based diagnostic aid for AIBDs. Leveraging the visual similarities between our imaging data with pre-existing repositories, we demonstrate automated classification of AIBDs using techniques such as transfer learning and data augmentation over a convolutional neural network (CNN). A three-loop process for training is used, combining feature extraction and fine-tuning to improve performance on our classification task. Our final model retains an accuracy nearly on par with dermatologists' diagnostic accuracy on more common skin cancers. Given the efficacy of our predictive model despite low amounts of training data, this approach holds the potential to benefit clinical diagnoses of AIBDs. Furthermore, our approach can be extrapolated to the diagnosis of other clinically similar rare diseases.

scholarly journals Named Entity Recognition in Chinese Medical Literature Using Pretraining Models

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Yu Wang ◽  
Yining Sun ◽  
Zuchang Ma ◽  
Lisheng Gao ◽  
Yang Xu
Keyword(s):  
Large Scale ◽  
Data Augmentation ◽  
Medical Literature ◽  
Named Entity Recognition ◽  
Semantic Knowledge ◽  
Training Data ◽  
Fine Tuning ◽  
Entity Recognition ◽  
Small Scale ◽  
Named Entity

The medical literature contains valuable knowledge, such as the clinical symptoms, diagnosis, and treatments of a particular disease. Named Entity Recognition (NER) is the initial step in extracting this knowledge from unstructured text and presenting it as a Knowledge Graph (KG). However, the previous approaches of NER have often suffered from small-scale human-labelled training data. Furthermore, extracting knowledge from Chinese medical literature is a more complex task because there is no segmentation between Chinese characters. Recently, the pretraining models, which obtain representations with the prior semantic knowledge on large-scale unlabelled corpora, have achieved state-of-the-art results for a wide variety of Natural Language Processing (NLP) tasks. However, the capabilities of pretraining models have not been fully exploited, and applications of other pretraining models except BERT in specific domains, such as NER in Chinese medical literature, are also of interest. In this paper, we enhance the performance of NER in Chinese medical literature using pretraining models. First, we propose a method of data augmentation by replacing the words in the training set with synonyms through the Mask Language Model (MLM), which is a pretraining task. Then, we consider NER as the downstream task of the pretraining model and transfer the prior semantic knowledge obtained during pretraining to it. Finally, we conduct experiments to compare the performances of six pretraining models (BERT, BERT-WWM, BERT-WWM-EXT, ERNIE, ERNIE-tiny, and RoBERTa) in recognizing named entities from Chinese medical literature. The effects of feature extraction and fine-tuning, as well as different downstream model structures, are also explored. Experimental results demonstrate that the method of data augmentation we proposed can obtain meaningful improvements in the performance of recognition. Besides, RoBERTa-CRF achieves the highest F1-score compared with the previous methods and other pretraining models.

Sign in / Sign up

Export Citation Format

Share Document