Conditional assessment of large-scale infrastructure systems using deep learning approaches (Conference Presentation)

2020 ◽  
Author(s):  
Hong Pan ◽  
Zi Zhang ◽  
Qi Cao ◽  
Xingyu Wang ◽  
Zhibin Lin
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Learning Approaches ◽  
Infrastructure Systems

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning

Database ◽  
2019 ◽  
Vol 2019 ◽  
Author(s):  
Tao Chen ◽  
Mingfen Wu ◽  
Hexi Li
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Relation Extraction ◽  
Training Model ◽  
Biomedical Literature ◽  
Training Data ◽  
Fine Tuning ◽  
Learning Approaches ◽  
Additional Time ◽  
Clinical Records

Abstract The automatic extraction of meaningful relations from biomedical literature or clinical records is crucial in various biomedical applications. Most of the current deep learning approaches for medical relation extraction require large-scale training data to prevent overfitting of the training model. We propose using a pre-trained model and a fine-tuning technique to improve these approaches without additional time-consuming human labeling. Firstly, we show the architecture of Bidirectional Encoder Representations from Transformers (BERT), an approach for pre-training a model on large-scale unstructured text. We then combine BERT with a one-dimensional convolutional neural network (1d-CNN) to fine-tune the pre-trained model for relation extraction. Extensive experiments on three datasets, namely the BioCreative V chemical disease relation corpus, traditional Chinese medicine literature corpus and i2b2 2012 temporal relation challenge corpus, show that the proposed approach achieves state-of-the-art results (giving a relative improvement of 22.2, 7.77, and 38.5% in F1 score, respectively, compared with a traditional 1d-CNN classifier). The source code is available at https://github.com/chentao1999/MedicalRelationExtraction.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals TreeVibes: Modern tools for Global Monitoring of Trees Against Borers

2020 ◽  
Author(s):  
Iraklis Rigakis ◽  
Ilyas Potamitis ◽  
Nicolas Alexander Tatlas ◽  
Stelios M. Potirakis ◽  
Stavros Ntalampiras
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Cost Effective ◽  
Automatic Monitoring ◽  
Urban Environments ◽  
Learning Approaches ◽  
Learning Techniques ◽  
Longhorn Beetles ◽  
Cloud Server ◽  
Wood Boring

Is there a wood-feeding insect inside a tree or wooden structure? We investigate several ways on how deep learning approaches can massively scan recordings of vibrations stemming from probed trees to infer their infestation state with wood-boring insects that feed and move inside wood. The recordings come from remotely controlled devices that sample the internal soundscape of trees on a 24/7 basis and wirelessly transmit brief recordings of the registered vibrations to a cloud server. We discuss the different sources of vibrations that can be picked up from trees in urban environments and how deep learning methods can focus on those originating from borers. Our goal is to match the problem of the accelerated&mdash;due to global trade and climate change&mdash; establishment of invasive xylophagus insects by increasing the capacity of inspection agencies. We aim at introducing permanent, cost-effective, automatic monitoring of trees based on deep learning techniques, in commodity entry point as well as in wild, urban and cultivated areas in order to effect large-scale, sustainable pest-risk analysis and management of wood boring insects such as those from the Cerambycidae family (longhorn beetles).

scholarly journals Fingerprint Classification Based on Deep Learning Approaches: Experimental Findings and Comparisons

Symmetry ◽  
2021 ◽  
Vol 13 (5) ◽  
pp. 750
Author(s):  
Carmelo Militello ◽  
Leonardo Rundo ◽  
Salvatore Vitabile ◽  
Vincenzo Conti
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Learning Approaches ◽  
Biometric Recognition ◽  
Fingerprint Classification ◽  
Model Efficiency ◽  
Experimental Findings ◽  
Processing Techniques

Biometric classification plays a key role in fingerprint characterization, especially in the identification process. In fact, reducing the number of comparisons in biometric recognition systems is essential when dealing with large-scale databases. The classification of fingerprints aims to achieve this target by splitting fingerprints into different categories. The general approach of fingerprint classification requires pre-processing techniques that are usually computationally expensive. Deep Learning is emerging as the leading field that has been successfully applied to many areas, such as image processing. This work shows the performance of pre-trained Convolutional Neural Networks (CNNs), tested on two fingerprint databases—namely, PolyU and NIST—and comparisons to other results presented in the literature in order to establish the type of classification that allows us to obtain the best performance in terms of precision and model efficiency, among approaches under examination, namely: AlexNet, GoogLeNet, and ResNet. We present the first study that extensively compares the most used CNN architectures by classifying the fingerprints into four, five, and eight classes. From the experimental results, the best performance was obtained in the classification of the PolyU database by all the tested CNN architectures due to the higher quality of its samples. To confirm the reliability of our study and the results obtained, a statistical analysis based on the McNemar test was performed.

scholarly journals TreeVibes: Modern Tools for Global Monitoring of Trees for Borers

Smart Cities ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 271-285
Author(s):  
Iraklis Rigakis ◽  
Ilyas Potamitis ◽  
Nicolaos-Alexandros Tatlas ◽  
Stelios M. Potirakis ◽  
Stavros Ntalampiras
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Cost Effective ◽  
Automatic Monitoring ◽  
Urban Environments ◽  
Learning Approaches ◽  
Learning Techniques ◽  
Longhorn Beetles ◽  
Entry Points ◽  
Wood Boring

Is there a wood-feeding insect inside a tree or wooden structure? We investigate several ways of how deep learning approaches can massively scan recordings of vibrations stemming from probed trees to infer their infestation state with wood-boring insects that feed and move inside wood. The recordings come from remotely controlled devices that sample the internal soundscape of trees on a 24/7 basis and wirelessly transmit brief recordings of the registered vibrations to a cloud server. We discuss the different sources of vibrations that can be picked up from trees in urban environments and how deep learning methods can focus on those originating from borers. Our goal is to match the problem of the accelerated—due to global trade and climate change— establishment of invasive xylophagus insects by increasing the capacity of inspection agencies. We aim at introducing permanent, cost-effective, automatic monitoring of trees based on deep learning techniques, in commodity entry points as well as in wild, urban and cultivated areas in order to effect large-scale, sustainable pest-risk analysis and management of wood boring insects such as those from the Cerambycidae family (longhorn beetles).

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Applying Deep Learning Technique for Depression Classification in Social Media Text

2020 ◽  
Vol 10 (10) ◽  
pp. 2446-2451
Author(s):  
Hussain Ahmad ◽  
Muhammad Zubair Asghar ◽  
Fahad M. Alotaibi ◽  
Ibrahim A. Hameed
Keyword(s):  
Social Media ◽  
Deep Learning ◽  
Large Scale ◽  
Research Area ◽  
Mental Illnesses ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Learning Approaches ◽  
Close Relationship ◽  
Social Media Platforms

In social media, depression identification could be regarded as a complex task because of the complicated nature associated with mental disorders. In recent times, there has been an evolution in this research area with growing popularity of social media platforms as these have become a fundamental part of people's day-to-day life. Social media platforms and their users share a close relationship due to which the users' personal life is reflected in these platforms on several levels. Apart from the associated complexity in recognising mental illnesses via social media platforms, implementing supervised machine learning approaches like deep neural networks is yet to be adopted in a large scale because of the inherent difficulties associated with procuring sufficient quantities of annotated training data. Because of such reasons, we have made effort to identify deep learning model that is most effective from amongst selected architectures with previous successful record in supervised learning methods. The selected model is employed to recognise online users that display depression; since there is limited unstructured text data that could be extracted from Twitter.

scholarly journals Large-scale Image Classification with Explainable deep learning scheme

2021 ◽  
Author(s):  
Bin Wu ◽  
Yuhong Fan ◽  
Li Mao
Keyword(s):  
Decision Making ◽  
Deep Learning ◽  
Spatial Attention ◽  
Large Scale ◽  
Spatial Dimension ◽  
Main Function ◽  
Learning Approaches ◽  
Attention Model ◽  
Object Decision ◽  
The Arts

Abstract For the uncertainty and complexity in object decision making and the differences of decision makers ' reliabilities, an object decision making method based on deep learning theory is proposed. However, traditional deep learning approaches optimize the parameters in an "end-to-end" mode by annotating large amounts of data to propagate the errors backwards. The learning method could be considered to be as a "black box", which is weak in explainability. Explainability refers to an algorithm that gives a clear summary of a particular task and connects it to defined principles or principles in the human world. This paper proposes an explainable attention model consisting of channel attention module and spatial attention module. The proposed module derives attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. For different channels, the higher the weight, the higher the correlation which required more attention. The main function of spatial attention is to capture the most informative part in the local feature graph, which is a supplement to channel attention. We evaluate our proposed module based on the ImageNet-1K and Cifar-100 respectively. Experimental results show that our algorithm is superior in both accuracy and robustness compared with the state of the arts.

scholarly journals An Overview on Analyzing Deep Learning and Transfer Learning Approaches for Health Monitoring

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Yiting Wang ◽  
Shah Nazir ◽  
Muhammad Shafiq
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Health Monitoring ◽  
Large Scale ◽  
Medical Information ◽  
Handheld Devices ◽  
Smart Devices ◽  
World Population ◽  
Learning Approaches ◽  
Home Based

With the rise and advancement of technology, early detection and involvement in health-associated monitoring through home control are growing with population aging. The expansion of healthy life expectations is progressively significant due to the speedy aging of the world population. The patient requires early and home-based treatment to detect and prevent disease on time and with less effort. Home-based health monitoring has been considered the need of a smart home. The services of health monitoring can facilitate the patient by collecting and analyzing the data of health for tackling diverse complex issues of health at a large scale. Health monitoring is a sustainable progression of clinical trials for ensuring that health is monitored according to the defined protocol and standard operating procedures. Various scenarios can be considered for monitoring health and are performed through experts of the field. Healthcare systems are having large-scale infrastructure of electronic devices, medical information systems, wearable and smart devices, medical records, and handheld devices. The growth in medical infrastructure, combined with the development of computational approaches in healthcare, has empowered practitioners and researchers to devise a novel solution in the innovative spectra. A detailed report of the existing literature in terms of deep learning and transfer learning is the dire need and facilitating of modern healthcare. To overcome these limitations, therefore, the proposed study presents a comprehensive review of the existing approaches, techniques, and methods associated with deep learning and transfer learning for health monitoring. This review will help researchers to formulate new ideas for facilitating healthcare based on the existing evidence.

Sign in / Sign up

Export Citation Format

Share Document