Generic Multi-label Annotation via Adaptive Graph and Marginalized Augmentation

2021 ◽  
Vol 16 (1) ◽  
pp. 1-20
Author(s):  
Lichen Wang ◽  
Zhengming Ding ◽  
Yun Fu
Keyword(s):  
Large Scale ◽  
Learning Performance ◽  
Learning Approaches ◽  
Intrinsic Structure ◽  
Learning Framework ◽  
Model Generalization ◽  
Single Instance ◽  
Augmentation Strategy ◽  
Accurate Performance ◽  
Learning Scenarios

Multi-label learning recovers multiple labels from a single instance. It is a more challenging task compared with single-label manner. Most multi-label learning approaches need large-scale well-labeled samples to achieve high accurate performance. However, it is expensive to build such a dataset. In this work, we propose a generic multi-label learning framework based on Adaptive Graph and Marginalized Augmentation (AGMA) in a semi-supervised scenario. Generally speaking, AGMA makes use of a small amount of labeled data associated with a lot of unlabeled data to boost the learning performance. First, an adaptive similarity graph is learned to effectively capture the intrinsic structure within the data. Second, marginalized augmentation strategy is explored to enhance the model generalization and robustness. Third, a feature-label autoencoder is further deployed to improve inferring efficiency. All the modules are jointly trained to benefit each other. State-of-the-art benchmarks in both traditional and zero-shot multi-label learning scenarios are evaluated. Experiments and ablation studies illustrate the accuracy and efficiency of our AGMA method.

scholarly journals Adaptive Graph Guided Embedding for Multi-label Annotation

2018 ◽  
Author(s):  
Lichen Wang ◽  
Zhengming Ding ◽  
Yun Fu
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Unlabeled Data ◽  
Label Propagation ◽  
Experimental Results ◽  
Training Data ◽  
Learning Performance ◽  
Intrinsic Structure ◽  
Latent Space ◽  
Art Methods

Multi-label annotation is challenging since a large amount of well-labeled training data are required to achieve promising performance. However, providing such data is expensive while unlabeled data are widely available. To this end, we propose a novel Adaptive Graph Guided Embedding (AG2E) approach for multi-label annotation in a semi-supervised fashion, which utilizes limited labeled data associating with large-scale unlabeled data to facilitate learning performance. Specifically, a multi-label propagation scheme and an effective embedding are jointly learned to seek a latent space where unlabeled instances tend to be well assigned multiple labels. Furthermore, a locality structure regularizer is designed to preserve the intrinsic structure and enhance the multi-label annotation. We evaluate our model in both conventional multi-label learning and zero-shot learning scenario. Experimental results demonstrate that our approach outperforms other compared state-of-the-art methods.

Unifying Heterogenous Electronic Health Records Systems via Text-Based Code Embedding: Study of Predictive Modeling (Preprint)

2021 ◽  
Author(s):  
Kyunghoon Hur ◽  
Jiyoung Lee ◽  
Jungwoo Oh ◽  
Wesley Price ◽  
Young-Hak Kim ◽  
...  
Keyword(s):  
Electronic Health Records ◽  
Predictive Modeling ◽  
Large Scale ◽  
Representation Learning ◽  
Superior Performance ◽  
Health Records ◽  
Learning Framework ◽  
Learning Scenarios ◽  
Electronic Health ◽  
Medical Concepts

BACKGROUND Substantial increase in the use of Electronic Health Records (EHRs) has opened new frontiers for predictive healthcare. However, while EHR systems are nearly ubiquitous, they lack a unified code system for representing medical concepts. Heterogeneous formats of EHR present a substantial barrier for the training and deployment of state-of-the-art deep learning models at scale. OBJECTIVE The aim of this study is to suggest a novel text embedding approach to overcome heterogeneity of EHR structure among different EHR systems. METHODS We introduce Description-based Embedding, DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR. DescEmb takes advantage of the flexibility of neural language understanding models while maintaining a neutral approach that can be combined with prior frameworks for task-specific representation learning or predictive modeling. RESULTS Based on five prediction tasks with two heterogeneous EHR datasets, DescEmb achieves comparable or superior performance to the traditional code-based embedding approach, especially under the zero-shot and few-shot transfer learning scenarios. We also demonstrate that DescEmb enables us to train a single model on a pooled dataset from heterogeneous EHR systems and achieve the same, if not better performance compared to training separate models for each EHR system. CONCLUSIONS Based on the promising results, we believe the description-based embedding approach on EHR will open a new direction for large-scale predictive modeling in healthcare.

Recent Progress in Machine Learning-based Prediction of Peptide Activity for Drug Discovery

2019 ◽  
Vol 19 (1) ◽  
pp. 4-16 ◽  
Author(s):  
Qihui Wu ◽  
Hanzhong Ke ◽  
Dongli Li ◽  
Qi Wang ◽  
Jiansong Fang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Drug Discovery ◽  
Large Scale ◽  
Recent Progress ◽  
High Specificity ◽  
Learning Approaches ◽  
Anticancer Peptides ◽  
The Past ◽  
Traditional Approaches ◽  
Large Scale Screening

Over the past decades, peptide as a therapeutic candidate has received increasing attention in drug discovery, especially for antimicrobial peptides (AMPs), anticancer peptides (ACPs) and antiinflammatory peptides (AIPs). It is considered that the peptides can regulate various complex diseases which are previously untouchable. In recent years, the critical problem of antimicrobial resistance drives the pharmaceutical industry to look for new therapeutic agents. Compared to organic small drugs, peptide- based therapy exhibits high specificity and minimal toxicity. Thus, peptides are widely recruited in the design and discovery of new potent drugs. Currently, large-scale screening of peptide activity with traditional approaches is costly, time-consuming and labor-intensive. Hence, in silico methods, mainly machine learning approaches, for their accuracy and effectiveness, have been introduced to predict the peptide activity. In this review, we document the recent progress in machine learning-based prediction of peptides which will be of great benefit to the discovery of potential active AMPs, ACPs and AIPs.

scholarly journals Multi-task learning based Encoder-Decoder: A comprehensive detection and diagnosis system for multi-sensor data

2021 ◽  
Vol 13 (5) ◽  
pp. 168781402110131
Author(s):  
Junfeng Wu ◽  
Li Yao ◽  
Bin Liu ◽  
Zheyuan Ding ◽  
Lei Zhang
Keyword(s):  
Anomaly Detection ◽  
Event Detection ◽  
Large Scale ◽  
Multivariate Time Series ◽  
Sensor Data ◽  
Unified Framework ◽  
Diagnosis System ◽  
Learning Framework ◽  
Task Learning ◽  
Detection And Diagnosis

As more and more sensor data have been collected, automated detection, and diagnosis systems are urgently needed to lessen the increasing monitoring burden and reduce the risk of system faults. A plethora of researches have been done on anomaly detection, event detection, anomaly diagnosis respectively. However, none of current approaches can explore all these respects in one unified framework. In this work, a Multi-Task Learning based Encoder-Decoder (MTLED) which can simultaneously detect anomalies, diagnose anomalies, and detect events is proposed. In MTLED, feature matrix is introduced so that features are extracted for each time point and point-wise anomaly detection can be realized in an end-to-end way. Anomaly diagnosis and event detection share the same feature matrix with anomaly detection in the multi-task learning framework and also provide important information for system monitoring. To train such a comprehensive detection and diagnosis system, a large-scale multivariate time series dataset which contains anomalies of multiple types is generated with simulation tools. Extensive experiments on the synthetic dataset verify the effectiveness of MTLED and its multi-task learning framework, and the evaluation on a real-world dataset demonstrates that MTLED can be used in other application scenarios through transfer learning.

scholarly journals Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

2021 ◽  
Vol 13 (16) ◽  
pp. 3065
Author(s):  
Libo Wang ◽  
Rui Li ◽  
Dongzhi Wang ◽  
Chenxi Duan ◽  
Teng Wang ◽  
...  
Keyword(s):  
Large Scale ◽  
Texture Features ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Research Field ◽  
Learning Approaches ◽  
Fine Grained ◽  
Urban Scene ◽  
Fine Resolution ◽  
With Memory

Semantic segmentation from very fine resolution (VFR) urban scene images plays a significant role in several application scenarios including autonomous driving, land cover classification, urban planning, etc. However, the tremendous details contained in the VFR image, especially the considerable variations in scale and appearance of objects, severely limit the potential of the existing deep learning approaches. Addressing such issues represents a promising research field in the remote sensing community, which paves the way for scene-level landscape pattern analysis and decision making. In this paper, we propose a Bilateral Awareness Network which contains a dependency path and a texture path to fully capture the long-range relationships and fine-grained details in VFR images. Specifically, the dependency path is conducted based on the ResT, a novel Transformer backbone with memory-efficient multi-head self-attention, while the texture path is built on the stacked convolution operation. In addition, using the linear attention mechanism, a feature aggregation module is designed to effectively fuse the dependency features and texture features. Extensive experiments conducted on the three large-scale urban scene image segmentation datasets, i.e., ISPRS Vaihingen dataset, ISPRS Potsdam dataset, and UAVid dataset, demonstrate the effectiveness of our BANet. Specifically, a 64.6% mIoU is achieved on the UAVid dataset.

Machine learning identifies an immunological pattern associated with multiple juvenile idiopathic arthritis subtypes

2019 ◽  
Vol 78 (5) ◽  
pp. 617-628 ◽  
Author(s):  
Erika Van Nieuwenhove ◽  
Vasiliki Lagou ◽  
Lien Van Eyck ◽  
James Dooley ◽  
Ulrich Bodenhofer ◽  
...  
Keyword(s):  
Machine Learning ◽  
Juvenile Idiopathic Arthritis ◽  
Large Scale ◽  
Inflammatory Diseases ◽  
Adaptive Immune System ◽  
Healthy Children ◽  
Learning Approaches ◽  
Data Set ◽  
Immune Signature ◽  
Systemic Jia

ObjectivesJuvenile idiopathic arthritis (JIA) is the most common class of childhood rheumatic diseases, with distinct disease subsets that may have diverging pathophysiological origins. Both adaptive and innate immune processes have been proposed as primary drivers, which may account for the observed clinical heterogeneity, but few high-depth studies have been performed.MethodsHere we profiled the adaptive immune system of 85 patients with JIA and 43 age-matched controls with indepth flow cytometry and machine learning approaches.ResultsImmune profiling identified immunological changes in patients with JIA. This immune signature was shared across a broad spectrum of childhood inflammatory diseases. The immune signature was identified in clinically distinct subsets of JIA, but was accentuated in patients with systemic JIA and those patients with active disease. Despite the extensive overlap in the immunological spectrum exhibited by healthy children and patients with JIA, machine learning analysis of the data set proved capable of discriminating patients with JIA from healthy controls with ~90% accuracy.ConclusionsThese results pave the way for large-scale immune phenotyping longitudinal studies of JIA. The ability to discriminate between patients with JIA and healthy individuals provides proof of principle for the use of machine learning to identify immune signatures that are predictive to treatment response group.

scholarly journals Domain-Adversarial Training of Self-Attention-Based Networks for Land Cover Classification Using Multi-Temporal Sentinel-2 Satellite Imagery

2021 ◽  
Vol 13 (13) ◽  
pp. 2564
Author(s):  
Mauro Martini ◽  
Vittorio Mazzia ◽  
Aleem Khaliq ◽  
Marcello Chiaberge
Keyword(s):  
Land Cover ◽  
Large Scale ◽  
Domain Adaptation ◽  
Learning Approaches ◽  
Atmospheric Conditions ◽  
Practical Applications ◽  
Area Of Interest ◽  
Multi Temporal ◽  
Significant Performance ◽  
Adversarial Training

The increasing availability of large-scale remote sensing labeled data has prompted researchers to develop increasingly precise and accurate data-driven models for land cover and crop classification (LC&CC). Moreover, with the introduction of self-attention and introspection mechanisms, deep learning approaches have shown promising results in processing long temporal sequences in the multi-spectral domain with a contained computational request. Nevertheless, most practical applications cannot rely on labeled data, and in the field, surveys are a time-consuming solution that pose strict limitations to the number of collected samples. Moreover, atmospheric conditions and specific geographical region characteristics constitute a relevant domain gap that does not allow direct applicability of a trained model on the available dataset to the area of interest. In this paper, we investigate adversarial training of deep neural networks to bridge the domain discrepancy between distinct geographical zones. In particular, we perform a thorough analysis of domain adaptation applied to challenging multi-spectral, multi-temporal data, accurately highlighting the advantages of adapting state-of-the-art self-attention-based models for LC&CC to different target zones where labeled data are not available. Extensive experimentation demonstrated significant performance and generalization gain in applying domain-adversarial training to source and target regions with marked dissimilarities between the distribution of extracted features.

scholarly journals Enhancing graduates’ employability skills through authentic learning approaches

2019 ◽  
Vol 9 (1) ◽  
pp. 107-120 ◽  
Author(s):  
Adriana Ornellas ◽  
Kajsa Falkner ◽  
Eva Edman Stålbrandt
Keyword(s):  
Work Experience ◽  
Authentic Learning ◽  
Added Value ◽  
Employability Skills ◽  
Second Phase ◽  
Learning Approaches ◽  
Content Type ◽  
New Graduates ◽  
Design Activities ◽  
Learning Scenarios

Purpose The purpose of this paper is to introduce a theoretical framework based on authentic learning approaches that can be taken into consideration in higher education (HE) contexts to design activities that enable students to develop employability skills. Design/methodology/approach Three methods were used to develop the framework: desk research on current demand and supply of new graduate’s employability skills; interviews in four European HE institutions to identify authentic learning scenarios; and asynchronous online focus group to validate the framework. Findings The paper takes a competence-centred approach to the concept of employability skills and sets out a taxonomy of skills required to enhance new graduates’ employability. It also gives criteria and examples of authentic learning scenarios in HE settings that promote the acquisition of these skills. Research limitations/implications The framework developed remains theoretical. In a second phase, the framework will be applied to implement authentic activities in different programmes and subjects of five HE institutions, and the results will be reported in future publications. Practical implications The framework gives directions to create real and practical ways to enhance new graduates’ employability skills by improving the connection between HE curricula and the demands of the real world. Originality/value The added value of the paper lies in adopting a learner-centred, genuine and effective learning approach, such as authentic learning as a catalyst for bringing work experience to formal learning in HE institutions, in order to better develop graduates’ employability skills.

scholarly journals Survey Analysis of Robust and Real-Time Multi-Lane and Single Lane Detection in Indian Highway Scenarios

2021 ◽  
Vol 309 ◽  
pp. 01117
Author(s):  
A. Sai Hanuman ◽  
G. Prasanna Kumar
Keyword(s):  
Neural Network ◽  
System Integration ◽  
Large Scale ◽  
Feature Learning ◽  
Semantic Segmentation ◽  
Lane Detection ◽  
Learning Approaches ◽  
Survey Analysis ◽  
Lane Recognition ◽  
Continuous Frames

Studies on lane detection Lane identification methods, integration, and evaluation strategies square measure all examined. The system integration approaches for building a lot of strong detection systems are then evaluated and analyzed, taking into account the inherent limits of camera-based lane detecting systems. Present deep learning approaches to lane detection are inherently CNN's semantic segmentation network the results of the segmentation of the roadways and the segmentation of the lane markers are fused using a fusion method. By manipulating a huge number of frames from a continuous driving environment, we examine lane detection, and we propose a hybrid deep architecture that combines the convolution neural network (CNN) and the continuous neural network (CNN) (RNN). Because of the extensive information background and the high cost of camera equipment, a substantial number of existing results concentrate on vision-based lane recognition systems. Extensive tests on two large-scale datasets show that the planned technique outperforms rivals' lane detection strategies, particularly in challenging settings. A CNN block in particular isolates information from each frame before sending the CNN choices of several continuous frames with time-series qualities to the RNN block for feature learning and lane prediction.

scholarly journals Mol-BERT: An Effective Molecular Representation with BERT for Molecular Property Prediction

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Juncai Li ◽  
Xiaofei Jiang
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Large Scale ◽  
Molecular Data ◽  
Molecular Property ◽  
Property Prediction ◽  
Learning Framework ◽  
Learning Techniques ◽  
Potential Benefits ◽  
Current Sequence

Molecular property prediction is an essential task in drug discovery. Most computational approaches with deep learning techniques either focus on designing novel molecular representation or combining with some advanced models together. However, researchers pay fewer attention to the potential benefits in massive unlabeled molecular data (e.g., ZINC). This task becomes increasingly challenging owing to the limitation of the scale of labeled data. Motivated by the recent advancements of pretrained models in natural language processing, the drug molecule can be naturally viewed as language to some extent. In this paper, we investigate how to develop the pretrained model BERT to extract useful molecular substructure information for molecular property prediction. We present a novel end-to-end deep learning framework, named Mol-BERT, that combines an effective molecular representation with pretrained BERT model tailored for molecular property prediction. Specifically, a large-scale prediction BERT model is pretrained to generate the embedding of molecular substructures, by using four million unlabeled drug SMILES (i.e., ZINC 15 and ChEMBL 27). Then, the pretrained BERT model can be fine-tuned on various molecular property prediction tasks. To examine the performance of our proposed Mol-BERT, we conduct several experiments on 4 widely used molecular datasets. In comparison to the traditional and state-of-the-art baselines, the results illustrate that our proposed Mol-BERT can outperform the current sequence-based methods and achieve at least 2% improvement on ROC-AUC score on Tox21, SIDER, and ClinTox dataset.

Sign in / Sign up

Export Citation Format

Share Document