scholarly journals Lightweight End-to-End Speech Enhancement Generative Adversarial Network Using Sinc Convolutions

2021 ◽  
Vol 11 (16) ◽  
pp. 7564
Author(s):  
Lujun Li ◽  
Wudamu ◽  
Ludwig Kürzinger ◽  
Tobias Watzel ◽  
Gerhard Rigoll
Keyword(s):  
Speech Enhancement ◽  
Data Augmentation ◽  
Evaluation Criteria ◽  
Objective Evaluation ◽  
Research Direction ◽  
Generative Adversarial Networks ◽  
Speech Signal Processing ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
End To End

Generative adversarial networks (GANs) have recently garnered significant attention for their use in speech enhancement tasks, in which they generally process and reconstruct speech waveforms directly. Existing GANs for speech enhancement rely solely on the convolution operation, which may not accurately characterize the local information of speech signals—particularly high-frequency components. Sinc convolution has been proposed in order to allow the GAN to learn more meaningful filters in the input layer, and has achieved remarkable success in several speech signal processing tasks. Nevertheless, Sinc convolution for speech enhancement is still an under-explored research direction. This paper proposes Sinc–SEGAN, a novel generative adversarial architecture for speech enhancement, which usefully merges two powerful paradigms: Sinc convolution and the speech enhancement GAN (SEGAN). There are two highlights of the proposed system. First, it works in an end-to-end manner, overcoming the distortion caused by imperfect phase estimation. Second, the system derives a customized filter bank, tuned for the desired application compactly and efficiently. We empirically study the influence of different configurations of Sinc convolution, including the placement of the Sinc convolution layer, length of input signals, number of Sinc filters, and kernel size of Sinc convolution. Moreover, we employ a set of data augmentation techniques in the time domain, which further improve the system performance and its generalization abilities. Compared to competitive baseline systems, Sinc–SEGAN overtakes all of them with drastically reduced system parameters, demonstrating its effectiveness for practical usage, e.g., hearing aid design and cochlear implants. Additionally, data augmentation methods further boost Sinc–SEGAN performance across classic objective evaluation criteria for speech enhancement.

scholarly journals SharpGAN: Dynamic Scene Deblurring Method for Smart Ship Based on Receptive Field Block and Generative Adversarial Networks

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3641
Author(s):  
Hui Feng ◽  
Jundong Guo ◽  
Haixiang Xu ◽  
Shuzhi Sam Ge
Keyword(s):  
Receptive Field ◽  
Evaluation Criteria ◽  
Objective Evaluation ◽  
Detection Algorithm ◽  
Image Features ◽  
Motion Blur ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Blurred Image

Complex marine environment has an adverse effect on the object detection algorithm based on the vision sensor for the smart ship sailing at sea. In order to eliminate the motion blur in the images during the navigation of the smart ship and ensure safety, we propose SharpGAN, a new image deblurring method based on the generative adversarial network (GAN). First of all, we introduce the receptive field block net (RFBNet) to the deblurring network to enhance the network’s ability to extract blurred image features. Secondly, we propose a feature loss that combines different levels of image features to guide the network to perform higher-quality deblurring and improve the feature similarity between the restored images and the sharp images. Besides, we use the lightweight RFB-s module to significantly improve the real-time performance of the deblurring network. Compared with the existing deblurring methods, the proposed method not only has better deblurring performance in subjective visual effects and objective evaluation criteria, but also has higher deblurring efficiency. Finally, the experimental results reveal that the SharpGAN has a high correlation with the deblurring methods based on the physical model.

scholarly journals Alleviating Class Imbalance in Actuarial Applications Using Generative Adversarial Networks

Risks ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 49
Author(s):  
Kwanda Sydwell Ngwenduna ◽  
Rendani Mbuvha
Keyword(s):  
Predictive Models ◽  
Data Augmentation ◽  
Class Imbalance ◽  
Machine Learning Techniques ◽  
Quality Data ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Missing Data Imputation ◽  
Adversarial Network ◽  
Imbalanced Classes

To build adequate predictive models, a substantial amount of data is desirable. However, when expanding to new or unexplored territories, this required level of information is rarely always available. To build such models, actuaries often have to: procure data from local providers, use limited unsuitable industry and public research, or rely on extrapolations from other better-known markets. Another common pathology when applying machine learning techniques in actuarial domains is the prevalence of imbalanced classes where risk events of interest, such as mortality and fraud, are under-represented in data. In this work, we show how an implicit model using the Generative Adversarial Network (GAN) can alleviate these problems through the generation of adequate quality data from very limited or highly imbalanced samples. We provide an introduction to GANs and how they are used to synthesize data that accurately enhance the data resolution of very infrequent events and improve model robustness. Overall, we show a significant superiority of GANs for boosting predictive models when compared to competing approaches on benchmark data sets. This work offers numerous of contributions to actuaries with applications to inter alia new sample creation, data augmentation, boosting predictive models, anomaly detection, and missing data imputation.

scholarly journals Speech Emotion Recognition using Data Augmentation Method by Cycle-Generative Adversarial Networks

2021 ◽  
Author(s):  
Arash Shilandari ◽  
Hossein Marvi ◽  
Hossein Khosravi
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Speech Processing ◽  
Data Augmentation ◽  
Generative Adversarial Networks ◽  
Speech Emotion Recognition ◽  
Support Vector ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks

Nowadays, and with the mechanization of life, speech processing has become so crucial for the interaction between humans and machines. Deep neural networks require a database with enough data for training. The more features are extracted from the speech signal, the more samples are needed to train these networks. Adequate training of these networks can be ensured when there is access to sufficient and varied data in each class. If there is not enough data; it is possible to use data augmentation methods to obtain a database with enough samples. One of the obstacles to developing speech emotion recognition systems is the Data sparsity problem in each class for neural network training. The current study has focused on making a cycle generative adversarial network for data augmentation in a system for speech emotion recognition. For each of the five emotions employed, an adversarial generating network is designed to generate data that is very similar to the main data in that class, as well as differentiate the emotions of the other classes. These networks are taught in an adversarial way to produce feature vectors like each class in the space of the main feature, and then they add to the training sets existing in the database to train the classifier network. Instead of using the common cross-entropy error to train generative adversarial networks and to remove the vanishing gradient problem, Wasserstein Divergence has been used to produce high-quality artificial samples. The suggested network has been tested to be applied for speech emotion recognition using EMODB as training, testing, and evaluating sets, and the quality of artificial data evaluated using two Support Vector Machine (SVM) and Deep Neural Network (DNN) classifiers. Moreover, it has been revealed that extracting and reproducing high-level features from acoustic features, speech emotion recognition with separating five primary emotions has been done with acceptable accuracy.

scholarly journals RF-GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Yu Chen ◽  
Jun Long ◽  
Jifeng Guo
Keyword(s):  
Data Augmentation ◽  
Semantic Segmentation ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Fundus Images ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Retinal Fundus Images ◽  
Retinal Fundus ◽  
Segmentation Models

Diabetic retinopathy (DR) is a diabetic complication affecting the eyes, which is the main cause of blindness in young and middle-aged people. In order to speed up the diagnosis of DR, a mass of deep learning methods have been used for the detection of this disease, but they failed to attain excellent results due to unbalanced training data, i.e., the lack of DR fundus images. To address the problem of data imbalance, this paper proposes a method dubbed retinal fundus images generative adversarial networks (RF-GANs), which is based on generative adversarial network, to synthesize retinal fundus images. RF-GANs is composed of two generation models, RF-GAN1 and RF-GAN2. Firstly, RF-GAN1 is employed to translate retinal fundus images from source domain (the domain of semantic segmentation datasets) to target domain (the domain of EyePACS dataset connected to Kaggle (EyePACS)). Then, we train the semantic segmentation models with the translated images, and employ the trained models to extract the structural and lesion masks (hereafter, we refer to it as Masks) of EyePACS. Finally, we employ RF-GAN2 to synthesize retinal fundus images using the Masks and DR grading labels. This paper verifies the effectiveness of the method: RF-GAN1 can narrow down the domain gap between different datasets to improve the performance of the segmentation models. RF-GAN2 can synthesize realistic retinal fundus images. Adopting the synthesized images for data augmentation, the accuracy and quadratic weighted kappa of the state-of-the-art DR grading model on the testing set of EyePACS increase by 1.53% and 1.70%, respectively.

scholarly journals Improved Classification of White Blood Cells with the Generative Adversarial Network and Deep Convolutional Neural Network

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Khaled Almezhghwi ◽  
Sertan Serte
Keyword(s):  
Blood Cells ◽  
Data Augmentation ◽  
White Blood Cells ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Human Errors ◽  
Morphological Variations ◽  
Adversarial Network

White blood cells (leukocytes) are a very important component of the blood that forms the immune system, which is responsible for fighting foreign elements. The five types of white blood cells include neutrophils, eosinophils, lymphocytes, monocytes, and basophils, where each type constitutes a different proportion and performs specific functions. Being able to classify and, therefore, count these different constituents is critical for assessing the health of patients and infection risks. Generally, laboratory experiments are used for determining the type of a white blood cell. The staining process and manual evaluation of acquired images under the microscope are tedious and subject to human errors. Moreover, a major challenge is the unavailability of training data that cover the morphological variations of white blood cells so that trained classifiers can generalize well. As such, this paper investigates image transformation operations and generative adversarial networks (GAN) for data augmentation and state-of-the-art deep neural networks (i.e., VGG-16, ResNet, and DenseNet) for the classification of white blood cells into the five types. Furthermore, we explore initializing the DNNs’ weights randomly or using weights pretrained on the CIFAR-100 dataset. In contrast to other works that require advanced image preprocessing and manual feature extraction before classification, our method works directly with the acquired images. The results of extensive experiments show that the proposed method can successfully classify white blood cells. The best DNN model, DenseNet-169, yields a validation accuracy of 98.8%. Particularly, we find that the proposed approach outperforms other methods that rely on sophisticated image processing and manual feature engineering.

Colorization of fusion image of infrared and visible images based on parallel generative adversarial network approach

2021 ◽  
pp. 1-10
Author(s):  
Lei Chen ◽  
Jun Han ◽  
Feng Tian
Keyword(s):  
Ground Truth ◽  
Objective Evaluation ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Fusion Image ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Low Contrast ◽  
Adversarial Network ◽  
Visible Images

Fusing the infrared (IR) and visible images has many advantages and can be applied to applications such as target detection and recognition. Colors can give more accurate and distinct features, but the low resolution and low contrast of fused images make this a challenge task. In this paper, we proposed a method based on parallel generative adversarial networks (GANs) to address the challenge. We used IR image, visible image and fusion image as ground truth of ‘L’, ‘a’ and ‘b’ of the Lab model. Through the parallel GANs, we can gain the Lab data which can be converted to RGB image. We adopt TNO and RoadScene data sets to verify our method, and compare with five objective evaluation parameters obtained by other three methods based on deep learning (DL). It is demonstrated that the proposed approach is able to achieve better performance against state-of-arts methods.

scholarly journals Super-Resolution of Remote Sensing Images via a Dense Residual Generative Adversarial Network

2019 ◽  
Vol 11 (21) ◽  
pp. 2578 ◽  
Author(s):  
Wen Ma ◽  
Zongxu Pan ◽  
Feng Yuan ◽  
Bin Lei
Keyword(s):  
Remote Sensing ◽  
Network Architecture ◽  
Super Resolution ◽  
Objective Evaluation ◽  
Generative Adversarial Networks ◽  
Remote Sensing Images ◽  
Generative Adversarial Network ◽  
Memory Mechanism ◽  
Adversarial Network ◽  
Sensing Applications

Single image super-resolution (SISR) has been widely studied in recent years as a crucial technique for remote sensing applications. In this paper, a dense residual generative adversarial network (DRGAN)-based SISR method is proposed to promote the resolution of remote sensing images. Different from previous super-resolution (SR) approaches based on generative adversarial networks (GANs), the novelty of our method mainly lies in the following factors. First, we made a breakthrough in terms of network architecture to improve performance. We designed a dense residual network as the generative network in GAN, which can make full use of the hierarchical features from low-resolution (LR) images. We also introduced a contiguous memory mechanism into the network to take advantage of the dense residual block. Second, we modified the loss function and altered the model of the discriminative network according to the Wasserstein GAN with a gradient penalty (WGAN-GP) for stable training. Extensive experiments were performed using the NWPU-RESISC45 dataset, and the results demonstrated that the proposed method outperforms state-of-the-art methods in terms of both objective evaluation and subjective perspective.

scholarly journals Generating Scenery Images with Larger Variety According to User Descriptions

2021 ◽  
Vol 11 (21) ◽  
pp. 10224
Author(s):  
Hsu-Yung Cheng ◽  
Chih-Chang Yu
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Memory Cell ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Hidden Layer

In this paper, a framework based on generative adversarial networks is proposed to perform nature-scenery generation according to descriptions from the users. The desired place, time and seasons of the generated scenes can be specified with the help of text-to-image generation techniques. The framework improves and modifies the architecture of a generative adversarial network with attention models by adding the imagination models. The proposed attentional and imaginative generative network uses the hidden layer information to initialize the memory cell of the recurrent neural network to produce the desired photos. A data set containing different categories of scenery images is established to train the proposed system. The experiments validate that the proposed method is able to increase the quality and diversity of the generated images compared to the existing method. A possible application of road image generation for data augmentation is also demonstrated in the experimental results.

scholarly journals Data Augmentation for EEG-Based Emotion Recognition Using Generative Adversarial Networks

2021 ◽  
Vol 15 ◽  
Author(s):  
Guangcheng Bao ◽  
Bin Yan ◽  
Li Tong ◽  
Jun Shu ◽  
Linyuan Wang ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Data Augmentation ◽  
Generative Models ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Training Samples ◽  
Using Data ◽  
Effective Models

One of the greatest limitations in the field of EEG-based emotion recognition is the lack of training samples, which makes it difficult to establish effective models for emotion recognition. Inspired by the excellent achievements of generative models in image processing, we propose a data augmentation model named VAE-D2GAN for EEG-based emotion recognition using a generative adversarial network. EEG features representing different emotions are extracted as topological maps of differential entropy (DE) under five classical frequency bands. The proposed model is designed to learn the distributions of these features for real EEG signals and generate artificial samples for training. The variational auto-encoder (VAE) architecture can learn the spatial distribution of the actual data through a latent vector, and is introduced into the dual discriminator GAN to improve the diversity of the generated artificial samples. To evaluate the performance of this model, we conduct a systematic test on two public emotion EEG datasets, the SEED and the SEED-IV. The obtained recognition accuracy of the method using data augmentation shows as 92.5 and 82.3%, respectively, on the SEED and SEED-IV datasets, which is 1.5 and 3.5% higher than that of methods without using data augmentation. The experimental results show that the artificial samples generated by our model can effectively enhance the performance of the EEG-based emotion recognition.

scholarly journals Intraoral image generation by progressive growing of generative adversarial network and evaluation of generated image quality by dentists

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Kazuma Kokomoto ◽  
Rena Okawa ◽  
Kazuhiko Nakano ◽  
Kazunori Nozaki
Keyword(s):  
Private Information ◽  
Data Augmentation ◽  
Wasserstein Distance ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Pediatric Dentists ◽  
Clinical Cases

AbstractDentists need experience with clinical cases to practice specialized skills. However, the need to protect patient's private information limits their ability to utilize intraoral images obtained from clinical cases. In this study, since generating realistic images could make it possible to utilize intraoral images, progressive growing of generative adversarial networks are used to generate intraoral images. A total of 35,254 intraoral images were used as training data with resolutions of 128 × 128, 256 × 256, 512 × 512, and 1024 × 1024. The results of the training datasets with and without data augmentation were compared. The Sliced Wasserstein Distance was calculated to evaluate the generated images. Next, 50 real images and 50 generated images for each resolution were randomly selected and shuffled. 12 pediatric dentists were asked to observe these images and assess whether they were real or generated. The d prime of the 1024 × 1024 images was significantly higher than that of the other resolutions. In conclusion, generated intraoral images with resolutions of 512 × 512 or lower were so realistic that the dentists could not distinguish whether they were real or generated. This implies that the generated images can be used in dental education or data augmentation for deep learning, without privacy restrictions.

Sign in / Sign up

Export Citation Format

Share Document