scholarly journals Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series

2019 ◽  
Vol 11 (5) ◽  
pp. 523 ◽  
Author(s):  
Charlotte Pelletier ◽  
Geoffrey Webb ◽  
François Petitjean
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Land Cover ◽  
Network Architecture ◽  
Satellite Image ◽  
Classification Systems ◽  
Batch Size ◽  
Learning Approach ◽  
Land Cover Maps

Latest remote sensing sensors are capable of acquiring high spatial and spectral Satellite Image Time Series (SITS) of the world. These image series are a key component of classification systems that aim at obtaining up-to-date and accurate land cover maps of the Earth’s surfaces. More specifically, current SITS combine high temporal, spectral and spatial resolutions, which makes it possible to closely monitor vegetation dynamics. Although traditional classification algorithms, such as Random Forest (RF), have been successfully applied to create land cover maps from SITS, these algorithms do not make the most of the temporal domain. This paper proposes a comprehensive study of Temporal Convolutional Neural Networks (TempCNNs), a deep learning approach which applies convolutions in the temporal dimension in order to automatically learn temporal (and spectral) features. The goal of this paper is to quantitatively and qualitatively evaluate the contribution of TempCNNs for SITS classification, as compared to RF and Recurrent Neural Networks (RNNs) —a standard deep learning approach that is particularly suited to temporal data. We carry out experiments on Formosat-2 scene with 46 images and one million labelled time series. The experimental results show that TempCNNs are more accurate than the current state of the art for SITS classification. We provide some general guidelines on the network architecture, common regularization mechanisms, and hyper-parameter values such as batch size; we also draw out some differences with standard results in computer vision (e.g., about pooling layers). Finally, we assess the visual quality of the land cover maps produced by TempCNNs.

scholarly journals Land Cover Maps Production with High Resolution Satellite Image Time Series and Convolutional Neural Networks: Adaptations and Limits for Operational Systems

2019 ◽  
Author(s):  
Andrei Stoian ◽  
Vincent Poulain ◽  
Jordi Inglada ◽  
Victor Poughon ◽  
Dawa Derksen
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
High Resolution ◽  
Land Cover ◽  
Convolutional Neural Networks ◽  
Satellite Image ◽  
Image Data ◽  
Design Decision ◽  
Land Cover Maps ◽  
Sentinel 2

The Sentinel-2 satellite mission offers high resolution multispectral time series image data, enabling the production of detailed land cover maps globally. At this scale, the trade-off between processing time and result quality is a central design decision. Currently, this machine learning task is usually performed using pixelwise classification methods. The radical shift of the computer vision field away from hand engineered image features and towards more automation by representation learning comes with many promises, including higher quality results and less engineering effort. In this paper we assess fully convolutional neural networks architectures as replacements for a Random Forest classifier in an operational context for the production of high resolution land cover maps with Sentinel-2 time series at the country scale. Our contributions include a framework for working with Sentinel-2 L2A time series image data, an adaptation of the U-Net model for dealing with sparse annotation data while maintaining high resolution output, and an analysis of those results in the context of operational production of land cover maps.

scholarly journals Land Cover Maps Production with High Resolution Satellite Image Time Series and Convolutional Neural Networks: Adaptations and Limits for Operational Systems

2019 ◽  
Author(s):  
Andrei Stoian ◽  
Vincent Poulain ◽  
Jordi Inglada ◽  
Victor Poughon ◽  
Dawa Derksen
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
High Resolution ◽  
Land Cover ◽  
Convolutional Neural Networks ◽  
Satellite Image ◽  
Image Data ◽  
Design Decision ◽  
Land Cover Maps ◽  
Sentinel 2

The Sentinel-2 satellite mission offers high resolution multispectral time series image data, enabling the production of detailed land cover maps globally. At this scale, the trade-off between processing time and result quality is a central design decision. Currently, this machine learning task is usually performed using pixelwise classification methods. The radical shift of the computer vision field away from hand engineered image features and towards more automation by representation learning comes with many promises, including higher quality results and less engineering effort. In this paper we assess fully convolutional neural networks architectures as replacements for a Random Forest classifier in an operational context for the production of high resolution land cover maps with Sentinel-2 time series at the country scale. Our contributions include a framework for working with Sentinel-2 L2A time series image data, an adaptation of the U-Net model for dealing with sparse annotation data while maintaining high resolution output, and an analysis of those results in the context of operational production of land cover maps.

scholarly journals BENCHMARKING OF CONVOLUTIONAL NEURAL NETWORK APPROACHES FOR VEGETATION LAND COVER MAPPING

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 915-922
Author(s):  
B. Carpentier ◽  
A. Masse ◽  
E. Lavergne ◽  
C. Sannier
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Land Cover ◽  
Convolutional Neural Network ◽  
Temporal Dynamics ◽  
Satellite Image ◽  
Classification Systems ◽  
Temporal Features ◽  
Land Cover Maps ◽  
Promising Source

Abstract. Satellite Image Time Series (SITS) are becoming available at high spatial, spectral and temporal resolutions across the globe by the latest remote sensing sensors. These series of images can be highly valuable when exploited by classification systems to produce frequently updated and accurate land cover maps. The richness of spectral, spatial and temporal features in SITS is a promising source of data for developing better classification algorithms. However, machine learning methods such as Random Forests (RF), despite their fruitful application to SITS to produce land cover maps, are structurally unable to properly handle intertwined spatial, spectral and temporal dynamics without breaking the structure of the data. Therefore, the present work proposes a comparative study of various deep learning algorithms from the Convolutional Neural Network (CNN) family and evaluate their performance on SITS classification. They are compared to the processing chain coined iota2, developed by the CESBIO and based on a RF model. Experiments are carried out in an operational context using with sparse annotations from 290 labeled polygons. Less than 80 000 pixel time series belonging to 8 land cover classes from a year of Sentinel-2 monthly syntheses are used. Results show on a test set of 131 polygons that CNNs using 3D convolutions in space and time are more accurate than 1D temporal, stacked 2D and RF approaches. Best-performing models are CNNs using spatio-temporal features, namely 3D-CNN, 2D-CNN and SpatioTempCNN, a two-stream model using both 1D and 3D convolutions.

scholarly journals INTRODUCING AUTOMATIC SATELLITE IMAGE PROCESSING INTO LAND COVER MAPPING BY PHOTO-INTERPRETATION OF AIRBORNE DATA

2020 ◽  
Vol XLII-3/W11 ◽  
pp. 29-34
Author(s):  
H. Costa ◽  
P. Benevides ◽  
F. Marcelino ◽  
M. Caetano
Keyword(s):  
Time Series ◽  
Land Cover ◽  
Change Detection ◽  
Land Use Changes ◽  
Satellite Image ◽  
Landsat 8 ◽  
Photo Interpretation ◽  
Time Production ◽  
Mainland Portugal ◽  
Land Cover Maps

Abstract. A series of five land cover maps, widely known as COS (Carta de Uso e Ocupação do Solo), have been produced since 1990 for mainland Portugal. Previous to 2015, all maps were produced through photo-interpretation of orthophotos. Land cover and land use changes were detected through comparison of previous and recent orthophotos, which were used for map updating, thereby producing a new map. The remaining areas of no change were preserved across the maps for consistency. Despite the value of the maps produced, the method is very time-consuming and limited to the single-date reference of the orthophotos. From 2015 onwards, a new approach was adopted for map production. Photo-interpretation of orthophoto maps is still the basis of mapping, but assisted by products derived from satellite data. The goals are three-fold: (i) cut time production, (ii) increase map accuracy, and (iii) further detail the nomenclature. The last map published (COS 2015) benefited from change detection and classification analyses of Landsat data, namely for guiding the photo-interpretation in forest, shrublands, and mapping annual agriculture. Time production and map error have been reduced comparing to previous maps. The new 2018 map, currently in production, further explores this approach. Landsat 8 time series of 2015–2018 are used for change detection in vegetation based on NDVI differencing, thresholding and clustering. Sentinel-2 time series of 2017–2018 are used to classify Autumn/Winter crops and Spring/Summer crops based on NDVI temporal profiles and classification rules. Benefits and pitfalls of the new mapping approach are presented and discussed.

scholarly journals Land Cover Maps Production with High Resolution Satellite Image Time Series and Convolutional Neural Networks: Adaptations and Limits for Operational Systems

2019 ◽  
Vol 11 (17) ◽  
pp. 1986 ◽  
Author(s):  
Andrei Stoian ◽  
Vincent Poulain ◽  
Jordi Inglada ◽  
Victor Poughon ◽  
Dawa Derksen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
High Resolution ◽  
Land Cover ◽  
Convolutional Neural Networks ◽  
Image Data ◽  
Operational Systems ◽  
Land Cover Maps ◽  
Sentinel 2

The Sentinel-2 satellite mission offers high resolution multispectral time-series image data, enabling the production of detailed land cover maps globally. When mapping large territories, the trade-off between processing time and result quality is a central design decision. Currently, this machine learning task is usually performed using pixel-wise classification methods. However, the radical shift of the computer vision field away from hand-engineered image features and towards more automation by representation learning comes with many promises, including higher quality results and less engineering effort. In particular, convolutional neural networks learn features which take into account the context of the pixels and, therefore, a better representation of the data can be obtained. In this paper, we assess fully convolutional neural network architectures as replacements for a Random Forest classifier in an operational context for the production of high resolution land cover maps with Sentinel-2 time-series at the country scale. Our contributions include a framework for working with Sentinel-2 L2A time-series image data, an adaptation of the U-Net model (a fully convolutional neural network) for dealing with sparse annotation data while maintaining high resolution output, and an analysis of those results in the context of operational production of land cover maps. We conclude that fully convolutional neural networks can yield improved results with respect to pixel-wise Random Forest classifiers for classes where texture and context are pertinent. However, this new approach shows higher variability in quality across different landscapes and comes with a computational cost which could be to high for operational systems.

scholarly journals Learning, Visualizing and Exploring 16S rRNA Structure Using an Attention-based Deep Neural Network

2020 ◽  
Author(s):  
Zhengqiao Zhao ◽  
Stephen Woloszynek ◽  
Felix Agbavor ◽  
Joshua Chang Mell ◽  
Bahrad A. Sokhansanj ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Microbial Community ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Community Sample ◽  
Biological Research ◽  
Marker Genes ◽  
Learning Approach

AbstractRecurrent neural networks (RNNs) with memory (e.g. LSTMs) and attention mechanisms are widely used in natural language processing because they can capture short and long term sequential information for diverse tasks. We propose an integrated deep learning model for microbial DNA sequence data, which exploits convolutional networks, recurrent neural networks, and attention mechanisms to perform sample-associated attribute prediction—phenotype prediction—and extract interesting features, such as informative taxa and predictive k-mer context. In this paper, we develop this novel deep learning approach and evaluate its application to amplicon sequences. We focus on typically short DNA reads of 16s ribosomal RNA (rRNA) marker genes, which identify the heterogeneity of a microbial community sample. Our deep learning approach enables sample-level attribute and taxonomic prediction, with the aim of aiding biological research and supporting medical diagnosis. We demonstrate that our implementation of a novel attention-based deep network architecture, Read2Pheno, achieves read-level phenotypic prediction and, in turn, that aggregating read-level information can robustly predict microbial community properties, host phenotype, and taxonomic classification, with performance comparable to conventional approaches. Most importantly, as a further result of the training process, the network architecture will encode sequences (reads) into dense, meaningful representations: learned embedded vectors output on the intermediate layer of the network model, which can provide biological insight when visualized. Finally, we demonstrate that a model with an attention layer can automatically identify informative regions in sequences/reads which are particularly informative for classification tasks. An implementation of the attention-based deep learning network is available at https://github.com/EESI/sequence_attention.

scholarly journals Synergy of ICESat-2 and Landsat for Mapping Forest Aboveground Biomass with Deep Learning

2019 ◽  
Vol 11 (12) ◽  
pp. 1503 ◽  
Author(s):  
Lana L. Narine ◽  
Sorin C. Popescu ◽  
Lonesome Malambo
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Land Cover ◽  
Aboveground Biomass ◽  
Network Architecture ◽  
Prediction Models ◽  
Landsat Imagery ◽  
Canopy Cover ◽  
Land Cover Maps ◽  
Forest Aboveground Biomass

Spatially continuous estimates of forest aboveground biomass (AGB) are essential to supporting the sustainable management of forest ecosystems and providing invaluable information for quantifying and monitoring terrestrial carbon stocks. The launch of the Ice, Cloud, and land Elevation Satellite-2 (ICESat-2) on September 15th, 2018 offers an unparalleled opportunity to assess AGB at large scales using along-track samples that will be provided during its three-year mission. The main goal of this study was to investigate deep learning (DL) neural networks for mapping AGB with ICESat-2, using simulated photon-counting lidar (PCL)-estimated AGB for daytime, nighttime, and no noise scenarios, Landsat imagery, canopy cover, and land cover maps. The study was carried out in Sam Houston National Forest located in south-east Texas, using a simulated PCL-estimated AGB along two years of planned ICESat-2 profiles. The primary tasks were to investigate and determine neural network architecture, examine the hyper-parameter settings, and subsequently generate wall-to-wall AGB maps. A first set of models were developed using vegetation indices calculated from single-date Landsat imagery, canopy cover, and land cover, and a second set of models were generated using metrics from one year of Landsat imagery with canopy cover and land cover maps. To compare the effectiveness of final models, comparisons with Random Forests (RF) models were made. The deep neural network (DNN) models achieved R2 values of 0.42, 0.49, and 0.50 for the daytime, nighttime, and no noise scenarios respectively. With the extended dataset containing metrics calculated from Landsat images acquired on different dates, substantial improvements in model performance for all data scenarios were noted. The R2 values increased to 0.64, 0.66, and 0.67 for the daytime, nighttime, and no noise scenarios. Comparisons with Random forest (RF) prediction models highlighted similar results, with the same R2 and root mean square error (RMSE) range (15–16 Mg/ha) for daytime and nighttime scenarios. Findings suggest that there is potential for mapping AGB using a combinatory approach with ICESat-2 and Landsat-derived products with DL.

Detection of Bacterial Wilt on Enset Crop Using Deep Learning Approach

2020 ◽  
Vol 51 ◽  
pp. 131-146
Author(s):  
Yidnekachew Kibru Afework ◽  
Taye Girma Debelee
Keyword(s):  
Deep Learning ◽  
Bacterial Wilt ◽  
Network Architecture ◽  
Data Augmentation ◽  
Wilt Disease ◽  
Batch Size ◽  
Learning Approach ◽  
Bacterial Wilt Disease ◽  
Proposed Model ◽  
Augmentation Techniques

Bacterial Wilt disease is the most determinant factor as it results in a serious reduction in the quality and quantity of food produced by Enset crop. Therefore, early detection of Bacterial Wilt disease is important to diagnose and fight the disease. To this end, a deep learning approach that can detect the disease by using healthy and infected leave images of the crop is proposed. In particular, a convolutional neural network architecture is designed to classify the images collected from different farms as diseased or healthy. A total of 4896 images that were captured directly from the farm with the help of experts in the field of agriculture was used to train the proposed model. The proposed model was trained using these images and data augmentation techniques was applied to generate more images. Besides training the proposed model, a pre-trained model namely VGG16 is trained by using our dataset. The proposed model achieved a mean accuracy of 98.5% and the VGG16 pre-trained model achieved a mean accuracy of 96.6% by using a mini-batch size of 32 and a learning rate of 0.001. The preliminary results demonstrated that the effectiveness of the proposed approach under challenging conditions such as illumination, complex background, different resolutions, variable scale, rotation, and orientation of the real scene images.

Sign in / Sign up

Export Citation Format

Share Document