Optimizing the Recognition and Feature Extraction of Wind Turbines through Hybrid Semantic Segmentation Architectures

Miguel-Ángel Manso-Callejo; Calimanut-Ionut Cira; Ramón Alcarria; José-Juan Arranz-Justel

doi:10.3390/rs12223743

Optimizing the Recognition and Feature Extraction of Wind Turbines through Hybrid Semantic Segmentation Architectures

Remote Sensing ◽

10.3390/rs12223743 ◽

2020 ◽

Vol 12 (22) ◽

pp. 3743

Author(s):

Miguel-Ángel Manso-Callejo ◽

Calimanut-Ionut Cira ◽

Ramón Alcarria ◽

José-Juan Arranz-Justel

Keyword(s):

Wind Turbines ◽

Large Scale ◽

Semantic Segmentation ◽

Aerial Images ◽

Training Dataset ◽

Network Architectures ◽

Positional Accuracy ◽

High Resolution Images ◽

Alternative Sources ◽

Geometric Quality

Updating the mapping of wind turbines farms—found in constant expansion—is important to predict energy production or to minimize the risk of these infrastructures during storms. This geoinformation is not usually provided by public mapping agencies, and the alternative sources are usually consortiums or individuals interested in mapping and study. However, they do not offer metadata or genealogy, and their quality is unknown. This article presents a methodology oriented to optimize the recognition and extraction of features (wind turbines) using hybrid architectures of semantic segmentation. The aim is to characterize the quality of these datasets and help to improve and update them automatically at a large-scale. To this end, we intend to evaluate the capacity of hybrid semantic segmentation networks trained to extract features representing wind turbines from high-resolution images and to characterize the positional accuracy and completeness of a dataset whose genealogy and quality are unknown. We built a training dataset composed of 5140 tiles of aerial images and their cartography to train six different neural network architectures. The networks were evaluated on five test areas (covering 520 km2 of the Spanish territory) to identify the best segmentation architecture (in our case, LinkNet as base architecture and EfficientNet-b3 as the backbone). This hybrid segmentation model allowed us to characterize the completeness—both by commission and by omission—of the available georeferenced wind turbine dataset, as well as its geometric quality.

Download Full-text

Orchard Mapping with Deep Learning Semantic Segmentation

Sensors ◽

10.3390/s21113813 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3813

Author(s):

Athanasios Anagnostis ◽

Aristotelis C. Tagarakis ◽

Dimitrios Kateris ◽

Vasileios Moysiadis ◽

Claus Grøn Sørensen ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Semantic Segmentation ◽

Automated Detection ◽

Aerial Images ◽

Training Dataset ◽

Field Boundary ◽

Different Seasons ◽

Detection And Localization ◽

Different Levels

This study aimed to propose an approach for orchard trees segmentation using aerial images based on a deep learning convolutional neural network variant, namely the U-net network. The purpose was the automated detection and localization of the canopy of orchard trees under various conditions (i.e., different seasons, different tree ages, different levels of weed coverage). The implemented dataset was composed of images from three different walnut orchards. The achieved variability of the dataset resulted in obtaining images that fell under seven different use cases. The best-trained model achieved 91%, 90%, and 87% accuracy for training, validation, and testing, respectively. The trained model was also tested on never-before-seen orthomosaic images or orchards based on two methods (oversampling and undersampling) in order to tackle issues with out-of-the-field boundary transparent pixels from the image. Even though the training dataset did not contain orthomosaic images, it achieved performance levels that reached up to 99%, demonstrating the robustness of the proposed approach.

Download Full-text

A Fortran-Keras Deep Learning Bridge for Scientific Computing

Scientific Programming ◽

10.1155/2020/8888811 ◽

2020 ◽

Vol 2020 ◽

pp. 1-13 ◽

Cited By ~ 1

Author(s):

Jordan Ott ◽

Mike Pritchard ◽

Natalie Best ◽

Erik Linstead ◽

Milan Curcic ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Large Scale ◽

Climate Model ◽

Emergent Behavior ◽

Training Dataset ◽

Climate Simulation ◽

Network Architectures ◽

Efficient Computation ◽

Radiation Physics

Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. These software libraries come preloaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model’s emergent behavior to be assessed, i.e., when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of the optimizer proves unexpectedly critical. This in turn reveals many new neural network architectures that produce considerable improvements in climate model stability including some with reduced error, for an especially challenging training dataset.

Download Full-text

Sequentially Delineation of Rooftops with Holes from VHR Aerial Images Using a Convolutional Recurrent Neural Network

Remote Sensing ◽

10.3390/rs13214271 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4271

Author(s):

Wei Huang ◽

Zeping Liu ◽

Hong Tang ◽

Jiayi Ge

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Semantic Segmentation ◽

Aerial Images ◽

Irregular Shapes ◽

Segmentation Methods ◽

High Resolution Images ◽

Novel Method ◽

Statistical Indicators ◽

Instance Segmentation

Semantic and instance segmentation methods are commonly used to build extraction from high-resolution images. The semantic segmentation method involves assigning a class label to each pixel in the image, thus ignoring the geometry of the building rooftop, which results in irregular shapes of the rooftop edges. As for instance segmentation, there is a strong assumption within this method that there exists only one outline polygon along the rooftop boundary. In this paper, we present a novel method to sequentially delineate exterior and interior contours of rooftops with holes from VHR aerial images, where most of the buildings have holes, by integrating semantic segmentation and polygon delineation. Specifically, semantic segmentation from the Mask R-CNN is used as a prior for hole detection. Then, the holes are used as objects for generating the internal contours of the rooftop. The external and internal contours of the rooftop are inferred separately using a convolutional recurrent neural network. Experimental results showed that the proposed method can effectively delineate the rooftops with both one and multiple polygons and outperform state-of-the-art methods in terms of the visual results and six statistical indicators, including IoU, OA, F1, BoundF, RE and Hd.

Download Full-text

Attention-Based Context Aware Network for Semantic Comprehension of Aerial Scenery

Sensors ◽

10.3390/s21061983 ◽

2021 ◽

Vol 21 (6) ◽

pp. 1983

Author(s):

Weipeng Shi ◽

Wenhu Qin ◽

Zhonghua Yun ◽

Peng Ping ◽

Kaiyang Wu ◽

...

Keyword(s):

High Resolution ◽

Semantic Segmentation ◽

Aerial Images ◽

Aerial Image ◽

Convolutional Network ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Semantic Labeling ◽

Autonomous Cars ◽

High Resolution Images

It is essential for researchers to have a proper interpretation of remote sensing images (RSIs) and precise semantic labeling of their component parts. Although FCN (Fully Convolutional Networks)-like deep convolutional network architectures have been widely applied in the perception of autonomous cars, there are still two challenges in the semantic segmentation of RSIs. The first is to identify details in high-resolution images with complex scenes and to solve the class-mismatch issues; the second is to capture the edge of objects finely without being confused by the surroundings. HRNET has the characteristics of maintaining high-resolution representation by fusing feature information with parallel multi-resolution convolution branches. We adopt HRNET as a backbone and propose to incorporate the Class-Oriented Region Attention Module (CRAM) and Class-Oriented Context Fusion Module (CCFM) to analyze the relationships between classes and patch regions and between classes and local or global pixels, respectively. Thus, the perception capability of the model for the detailed part in the aerial image can be enhanced. We leverage these modules to develop an end-to-end semantic segmentation model for aerial images and validate it on the ISPRS Potsdam and Vaihingen datasets. The experimental results show that our model improves the baseline accuracy and outperforms some commonly used CNN architectures.

Download Full-text

AUTOMATIC OBJECT SEGMENTATION TO SUPPORT CRISIS MANAGEMENT OF LARGE-SCALE EVENTS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b2-2021-433-2021 ◽

2021 ◽

Vol XLIII-B2-2021 ◽

pp. 433-440

Author(s):

S. M. Azimi ◽

R. Kiefl ◽

V. Gstaiger ◽

R. Bahmanyar ◽

N. Merkle ◽

...

Keyword(s):

Large Scale ◽

Remote Sensing Data ◽

Semantic Segmentation ◽

Aerial Images ◽

Proof Of Concept ◽

Timely Manner ◽

Security Forces ◽

Supply Logistics ◽

Special Challenge ◽

Situational Information

Abstract. The management of large-scale events with a widely distributed camping area is a special challenge for organisers and security forces and requires both comprehensive preparation and attentive monitoring to ensure the safety of the participants. Crucial to this is the availability of up-to-date situational information, e.g. from remote sensing data. In particular, information on the number and distribution of people is important in the event of a crisis in order to be able to react quickly and effectively manage the corresponding rescue and supply logistics. One way to estimate the number of persons especially at night is to classify the type and size of objects such as tents and vehicles on site and to distinguish between objects with and without a sleeping function. In order to make this information available in a timely manner, an automated situation assessment is required. In this work, we have prepared the first high-quality dataset in order to address the aforementioned challenge which contains aerial images over a large-scale festival of different dates. We investigate the feasibility of this task using Convolutional Neural Networks for instance-wise semantic segmentation and carry out several experiments using the Mask-RCNN algorithm and evaluate the results. Results are promising and indicate the possibility of function-based tent classification as a proof-of-concept. The results and thereof discussions can pave the way for future developments and investigations.

Download Full-text

Deep Neural Networks Capabilities for Semantic Segmentation of Noisy Aerial Images

10.51130/graphicon-2020-2-3-71 ◽

2020 ◽

pp. paper71-1-paper71-12

Author(s):

Aleksandr Markelov ◽

Ivan Krivorotov ◽

Vadim Gorbachev

Keyword(s):

Neural Network ◽

Neural Networks ◽

Semantic Segmentation ◽

Low Noise ◽

Aerial Images ◽

Training Dataset ◽

Network Algorithms ◽

Aerial Photos ◽

Aggregation Techniques ◽

Segmentation Of Images

Semantic segmentation is one of the important ways of extracting information about objects in images. State of the art neural network algorithms allow to perform highly accurate semantic segmentation of images, including aerial photos. However, in most of the works authors use high-quality low-noise images. In this work, we study the ability of neural networks to correctly segment images with intensive uncorrelated Gaussian noise. The study brings us three main conclusions. Firstly, it demonstrates that neural network algorithms are capable of working with extreme image distortions without using additional filtration or image recovery techniques. Secondly, the experiments quantitatively show that distortion intensity can be negated with increased training set size. Such process is similar to model’s quality improvement and generalization due to training dataset enlargement. Finally, we quantitatively demonstrate how image aggregation techniques affect training with noised data.

Download Full-text

Non-Blind Image Deconvolution Based on “Ringing” Removal Using Convolutional Neural Network

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.10.ipas-180 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 181-1-181-7

Author(s):

Takahiro Kudo ◽

Takanori Fujisawa ◽

Takuro Yamaguchi ◽

Masaaki Ikehara

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Network Architecture ◽

Large Scale ◽

Blind Deconvolution ◽

Training Dataset ◽

Image Deconvolution ◽

Classic Problem ◽

Key Points ◽

Blind Image

Image deconvolution has been an important issue recently. It has two kinds of approaches: non-blind and blind. Non-blind deconvolution is a classic problem of image deblurring, which assumes that the PSF is known and does not change universally in space. Recently, Convolutional Neural Network (CNN) has been used for non-blind deconvolution. Though CNNs can deal with complex changes for unknown images, some CNN-based conventional methods can only handle small PSFs and does not consider the use of large PSFs in the real world. In this paper we propose a non-blind deconvolution framework based on a CNN that can remove large scale ringing in a deblurred image. Our method has three key points. The first is that our network architecture is able to preserve both large and small features in the image. The second is that the training dataset is created to preserve the details. The third is that we extend the images to minimize the effects of large ringing on the image borders. In our experiments, we used three kinds of large PSFs and were able to observe high-precision results from our method both quantitatively and qualitatively.

Download Full-text

DeepSSPred: A Deep Learning Based Sulfenylation site predictor via a novel n-segmented optimize federated feature encoder

Protein and Peptide Letters ◽

10.2174/0929866527666201202103411 ◽

2020 ◽

Vol 27 ◽

Author(s):

Zaheer Ullah Khan ◽

Dechang Pi

Keyword(s):

Large Scale ◽

Computational Models ◽

Research Work ◽

Training Data ◽

Training Dataset ◽

Validation Dataset ◽

Cytokine Signaling ◽

Minority Class ◽

Independent Dataset ◽

Feature Encoding

Background: S-sulfenylation (S-sulphenylation, or sulfenic acid) proteins, are special kinds of post-translation modification, which plays an important role in various physiological and pathological processes such as cytokine signaling, transcriptional regulation, and apoptosis. Despite these aforementioned significances, and by complementing existing wet methods, several computational models have been developed for sulfenylation cysteine sites prediction. However, the performance of these models was not satisfactory due to inefficient feature schemes, severe imbalance issues, and lack of an intelligent learning engine. Objective: In this study, our motivation is to establish a strong and novel computational predictor for discrimination of sulfenylation and non-sulfenylation sites. Methods: In this study, we report an innovative bioinformatics feature encoding tool, named DeepSSPred, in which, resulting encoded features is obtained via n-segmented hybrid feature, and then the resampling technique called synthetic minority oversampling was employed to cope with the severe imbalance issue between SC-sites (minority class) and non-SC sites (majority class). State of the art 2DConvolutional Neural Network was employed over rigorous 10-fold jackknife cross-validation technique for model validation and authentication. Results: Following the proposed framework, with a strong discrete presentation of feature space, machine learning engine, and unbiased presentation of the underline training data yielded into an excellent model that outperforms with all existing established studies. The proposed approach is 6% higher in terms of MCC from the first best. On an independent dataset, the existing first best study failed to provide sufficient details. The model obtained an increase of 7.5% in accuracy, 1.22% in Sn, 12.91% in Sp and 13.12% in MCC on the training data and12.13% of ACC, 27.25% in Sn, 2.25% in Sp, and 30.37% in MCC on an independent dataset in comparison with 2nd best method. These empirical analyses show the superlative performance of the proposed model over both training and Independent dataset in comparison with existing literature studies. Conclusion : In this research, we have developed a novel sequence-based automated predictor for SC-sites, called DeepSSPred. The empirical simulations outcomes with a training dataset and independent validation dataset have revealed the efficacy of the proposed theoretical model. The good performance of DeepSSPred is due to several reasons, such as novel discriminative feature encoding schemes, SMOTE technique, and careful construction of the prediction model through the tuned 2D-CNN classifier. We believe that our research work will provide a potential insight into a further prediction of S-sulfenylation characteristics and functionalities. Thus, we hope that our developed predictor will significantly helpful for large scale discrimination of unknown SC-sites in particular and designing new pharmaceutical drugs in general.

Download Full-text

Optimal Design of Novel Blade Profile for Savonius Wind Turbines

Energies ◽

10.3390/en14123484 ◽

2021 ◽

Vol 14 (12) ◽

pp. 3484

Author(s):

Tai-Lin Chang ◽

Shun-Feng Tsai ◽

Chun-Lung Chen

Keyword(s):

Wind Energy ◽

Drag Force ◽

Wind Turbines ◽

Rural Areas ◽

Large Scale ◽

Design Method ◽

Angular Position ◽

Vertical Axis ◽

Blade Profile ◽

Main Concept

Since the affirming of global warming, most wind energy projects have focused on the large-scale Horizontal Axis Wind Turbines (HAWTs). In recent years, the fast-growing wind energy sector and the demand for smarter grids have led to the use of Vertical Axis Wind Turbines (VAWTs) for decentralized energy generation systems, both in urban and remote rural areas. The goals of this study are to improve the Savonius-type VAWT’s efficiency and oscillation. The main concept is to redesign a Novel Blade profile using the Taguchi Robust Design Method and the ANSYS-Fluent simulation package. The convex contour of the blade faces against the wind, creating sufficient lift force and minimizing drag force; the concave contour faces up to the wind, improving or maintaining the drag force. The result is that the Novel Blade improves blade performance by 65% over the Savonius type at the best angular position. In addition, it decreases the oscillation and noise accordingly. This study achieved its two goals.

Download Full-text