scholarly journals Dual Attention Network for Pitch Estimation of Monophonic Music

Symmetry ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1296
Author(s):  
Wenfang Ma ◽  
Ying Hu ◽  
Hao Huang
Keyword(s):  
Audio Signal ◽  
Attention Mechanism ◽  
Feature Maps ◽  
Pitch Estimation ◽  
Attention Network ◽  
Coupled Mode ◽  
Tightly Coupled ◽  
Combination Modes ◽  
The Time Domain ◽  
Fully Connected

The task of pitch estimation is an essential step in many audio signal processing applications. In this paper, we propose a data-driven pitch estimation network, the Dual Attention Network (DA-Net), which processes directly on the time-domain samples of monophonic music. DA-Net includes six Dual Attention Modules (DA-Modules), and each of them includes two kinds of attention: element-wise and channel-wise attention. DA-Net is to perform element attention and channel attention operations on convolution features, which reflects the idea of "symmetry". DA-Modules can model the semantic interdependencies between element-wise and channel-wise features. In the DA-Module, the element-wise attention mechanism is realized by a Convolutional Gated Linear Unit (ConvGLU), and the channel-wise attention mechanism is realized by a Squeeze-and-Excitation (SE) block. We explored three kinds of combination modes (serial mode, parallel mode, and tightly coupled mode) of the element-wise attention and channel-wise attention. Element-wise attention selectively emphasizes useful features by re-weighting the features at all positions. Channel-wise attention can learn to use global information to selectively emphasize the informative feature maps and suppress the less useful ones. Therefore, DA-Net adaptively integrates the local features with their global dependencies. The outputs of DA-Net are fed into a fully connected layer to generate a 360-dimensional vector corresponding to 360 pitches. We trained the proposed network on the iKala and MDB-stem-synth datasets, respectively. According to the experimental results, our proposed dual attention network with tightly coupled mode achieved the best performance.

scholarly journals A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 183 ◽  
Author(s):  
Mustaqeem ◽  
Soonil Kwon
Keyword(s):  
Emotion Recognition ◽  
Audio Signal ◽  
Potential Method ◽  
Speech Emotion Recognition ◽  
Speech Signals ◽  
Human Beings ◽  
Feature Maps ◽  
Softmax Classifier ◽  
Visual Database ◽  
Fully Connected

Speech is the most significant mode of communication among human beings and a potential method for human-computer interaction (HCI) by using a microphone sensor. Quantifiable emotion recognition using these sensors from speech signals is an emerging area of research in HCI, which applies to multiple applications such as human-reboot interaction, virtual reality, behavior assessment, healthcare, and emergency call centers to determine the speaker’s emotional state from an individual’s speech. In this paper, we present major contributions for; (i) increasing the accuracy of speech emotion recognition (SER) compared to state of the art and (ii) reducing the computational complexity of the presented SER model. We propose an artificial intelligence-assisted deep stride convolutional neural network (DSCNN) architecture using the plain nets strategy to learn salient and discriminative features from spectrogram of speech signals that are enhanced in prior steps to perform better. Local hidden patterns are learned in convolutional layers with special strides to down-sample the feature maps rather than pooling layer and global discriminative features are learned in fully connected layers. A SoftMax classifier is used for the classification of emotions in speech. The proposed technique is evaluated on Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) datasets to improve accuracy by 7.85% and 4.5%, respectively, with the model size reduced by 34.5 MB. It proves the effectiveness and significance of the proposed SER technique and reveals its applicability in real-world applications.

scholarly journals A convolutional recurrent neural network with attention framework for speech separation in monaural recordings

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Chao Sun ◽  
Min Zhang ◽  
Ruijuan Wu ◽  
Junhong Lu ◽  
Guo Xian ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Attention Mechanism ◽  
Single Type ◽  
Translation Invariance ◽  
Separation Performance ◽  
Speech Separation ◽  
Feature Maps ◽  
Front End ◽  
The Time Domain

AbstractMost speech separation studies in monaural channel use only a single type of network, and the separation effect is typically not satisfactory, posing difficulties for high quality speech separation. In this study, we propose a convolutional recurrent neural network with an attention (CRNN-A) framework for speech separation, fusing advantages of two networks together. The proposed separation framework uses a convolutional neural network (CNN) as the front-end of a recurrent neural network (RNN), alleviating the problem that a sole RNN cannot effectively learn the necessary features. This framework makes use of the translation invariance provided by CNN to extract information without modifying the original signals. Within the supplemented CNN, two different convolution kernels are designed to capture information in both the time and frequency domains of the input spectrogram. After concatenating the time-domain and the frequency-domain feature maps, the feature information of speech is exploited through consecutive convolutional layers. Finally, the feature map learned from the front-end CNN is combined with the original spectrogram and is sent to the back-end RNN. Further, the attention mechanism is further incorporated, focusing on the relationship among different feature maps. The effectiveness of the proposed method is evaluated on the standard dataset MIR-1K and the results prove that the proposed method outperforms the baseline RNN and other popular speech separation methods, in terms of GNSDR (gloabl normalised source-to-distortion ratio), GSIR (global source-to-interferences ratio), and GSAR (gloabl source-to-artifacts ratio). In summary, the proposed CRNN-A framework can effectively combine the advantages of CNN and RNN, and further optimise the separation performance via the attention mechanism. The proposed framework can shed a new light on speech separation, speech enhancement, and other related fields.

scholarly journals A Global-Local Blur Disentangling Network for Dynamic Scene Deblurring

2021 ◽  
Vol 11 (5) ◽  
pp. 2174
Author(s):  
Xiaoguang Li ◽  
Feifan Yang ◽  
Jianglu Huang ◽  
Li Zhuo
Keyword(s):  
Local Features ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Dynamic Scene ◽  
Feature Maps ◽  
Training Scheme ◽  
Real Scene ◽  
Global And Local

Images captured in a real scene usually suffer from complex non-uniform degradation, which includes both global and local blurs. It is difficult to handle the complex blur variances by a unified processing model. We propose a global-local blur disentangling network, which can effectively extract global and local blur features via two branches. A phased training scheme is designed to disentangle the global and local blur features, that is the branches are trained with task-specific datasets, respectively. A branch attention mechanism is introduced to dynamically fuse global and local features. Complex blurry images are used to train the attention module and the reconstruction module. The visualized feature maps of different branches indicated that our dual-branch network can decouple the global and local blur features efficiently. Experimental results show that the proposed dual-branch blur disentangling network can improve both the subjective and objective deblurring effects for real captured images.

scholarly journals U2-ONet: A Two-Level Nested Octave U-Structure Network with a Multi-Scale Attention Mechanism for Moving Object Segmentation

2020 ◽  
Vol 13 (1) ◽  
pp. 60
Author(s):  
Chenjie Wang ◽  
Chengyuan Li ◽  
Jun Liu ◽  
Bin Luo ◽  
Xin Su ◽  
...  
Keyword(s):  
Moving Objects ◽  
Object Segmentation ◽  
Contextual Information ◽  
Attention Mechanism ◽  
Moving Object ◽  
Feature Maps ◽  
Moving Object Segmentation ◽  
Practical Applications ◽  
Multi Scale ◽  
Spatial Redundancy

Most scenes in practical applications are dynamic scenes containing moving objects, so accurately segmenting moving objects is crucial for many computer vision applications. In order to efficiently segment all the moving objects in the scene, regardless of whether the object has a predefined semantic label, we propose a two-level nested octave U-structure network with a multi-scale attention mechanism, called U2-ONet. U2-ONet takes two RGB frames, the optical flow between these frames, and the instance segmentation of the frames as inputs. Each stage of U2-ONet is filled with the newly designed octave residual U-block (ORSU block) to enhance the ability to obtain more contextual information at different scales while reducing the spatial redundancy of the feature maps. In order to efficiently train the multi-scale deep network, we introduce a hierarchical training supervision strategy that calculates the loss at each level while adding knowledge-matching loss to keep the optimization consistent. The experimental results show that the proposed U2-ONet method can achieve a state-of-the-art performance in several general moving object segmentation datasets.

Abstract P365: Blood Pressure Evaluation From Ppg Signal Analysis and Artificial Neural Network

Hypertension ◽  
2017 ◽  
Vol 70 (suppl_1) ◽  
Author(s):  
Francesco Lamonaca ◽  
Vitaliano Spagnuolo ◽  
Serena De Prisco ◽  
Domenico L Carnì ◽  
Domenico Grimaldi
Keyword(s):  
Neural Network ◽  
Blood Pressure ◽  
Artificial Neural Network ◽  
Audio Signal ◽  
Hardware Interface ◽  
Validation Procedure ◽  
Artificial Neural ◽  
Artificial Neural Network Ann ◽  
The Time Domain ◽  
And Storage

The analysis of the PPG signal in the time domain for the evaluation of the blood pressure (BP) is proposed. Some features extracted from the PPG signal are used to train an Artificial Neural Network (ANN) to determine the function that fit the target systolic and diastolic BP. The data related to the PPG signals and BP used in the analysis are provided by the Multi-parameter Intelligent Monitoring in Intensive Care (MIMIC II) database. The pre-analysis of the signal to remove inconsistent data is also proposed. A set of 1750 valid pulse is considered. The 80% of the input samples is used for the training of the network. Instead, the 10% of the input data are used for the validation of the network and 10% for final test of this last. The results show as the error for both the systolic and diastolic BP evaluation is included in the range of ±3 mmHg. Tab.1 shows the results for 20 PPG pulses randomly selected analyzed together with the systolic and diastolic blood pressure furnished by MIMC and evaluated by the trained ANN. Tab.1 experimental results comparing MIMIC and the ANN results. Moreover, a suitable hardware to validate the ANN with the sphygmomanometer is designed and realized. This hardware allows clinicians to collect data according to the requirements of the validation procedure. With the sphygmomanometer the systolic and diastolic values are referred to two different PPG pulses. As a consequence, it is proposed a new hardware interface allowing the synchronized acquisition and storage of the PPG signal and clinician voice. For the validation, the clinician: (i) evaluates the BP on both the arms and assesses that no significant differences occur; (ii) plugs the PPG sensor on the finger of one arm; (iii) starts the recording of both the PPG signal and the audio signal; (iv) evaluates the BP on the other arm with sphygmomanometer and says the systolic and diastolic values when detected. Through suitable post processing algorithm, the Systolic and Diastolic values are associated to the corresponding PPG Pulses. Following this procedure, the dataset to further validate the ANN according the standard is obtained. Once the ANN is validated it will be implemented on smartphone to have always in the pocket a reliable measurement system for Blood Pressure, oximetry and heart rate.

scholarly journals Video Description Model Based on Temporal-Spatial and Channel Multi-Attention Mechanisms

2020 ◽  
Vol 10 (12) ◽  
pp. 4312 ◽  
Author(s):  
Jie Xu ◽  
Haoliang Wei ◽  
Linke Li ◽  
Qiuru Fu ◽  
Jinhong Guo
Keyword(s):  
Neural Network ◽  
Spatial Attention ◽  
Semantic Information ◽  
Attention Mechanism ◽  
Visual Features ◽  
Feature Maps ◽  
Global Features ◽  
Model Based ◽  
Video Description ◽  
Video Visualization

Video description plays an important role in the field of intelligent imaging technology. Attention perception mechanisms are extensively applied in video description models based on deep learning. Most existing models use a temporal-spatial attention mechanism to enhance the accuracy of models. Temporal attention mechanisms can obtain the global features of a video, whereas spatial attention mechanisms obtain local features. Nevertheless, because each channel of the convolutional neural network (CNN) feature maps has certain spatial semantic information, it is insufficient to merely divide the CNN features into regions and then apply a spatial attention mechanism. In this paper, we propose a temporal-spatial and channel attention mechanism that enables the model to take advantage of various video features and ensures the consistency of visual features between sentence descriptions to enhance the effect of the model. Meanwhile, in order to prove the effectiveness of the attention mechanism, this paper proposes a video visualization model based on the video description. Experimental results show that, our model has achieved good performance on the Microsoft Video Description (MSVD) dataset and a certain improvement on the Microsoft Research-Video to Text (MSR-VTT) dataset.

scholarly journals SG-Net: Syntax-Guided Machine Reading Comprehension

2020 ◽  
Vol 34 (05) ◽  
pp. 9636-9643
Author(s):  
Zhuosheng Zhang ◽  
Yuwei Wu ◽  
Junru Zhou ◽  
Sufeng Duan ◽  
Hai Zhao ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Performance Improvement ◽  
Language Model ◽  
Attention Mechanism ◽  
Linguistic Knowledge ◽  
Attention Network ◽  
Syntactic Constraints ◽  
Syntactic Dependency ◽  
Machine Reading

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.

scholarly journals Hybrid Attention Network for Language-Based Person Search

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5279
Author(s):  
Yang Li ◽  
Huahu Xu ◽  
Junsheng Xiao
Keyword(s):  
Image Features ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Semantic Features ◽  
Retrieval Task ◽  
Attention Network ◽  
Fine Grained ◽  
Person Search ◽  
High Level ◽  
Language Description

Language-based person search retrieves images of a target person using natural language description and is a challenging fine-grained cross-modal retrieval task. A novel hybrid attention network is proposed for the task. The network includes the following three aspects: First, a cubic attention mechanism for person image, which combines cross-layer spatial attention and channel attention. It can fully excavate both important midlevel details and key high-level semantics to obtain better discriminative fine-grained feature representation of a person image. Second, a text attention network for language description, which is based on bidirectional LSTM (BiLSTM) and self-attention mechanism. It can better learn the bidirectional semantic dependency and capture the key words of sentences, so as to extract the context information and key semantic features of the language description more effectively and accurately. Third, a cross-modal attention mechanism and a joint loss function for cross-modal learning, which can pay more attention to the relevant parts between text and image features. It can better exploit both the cross-modal and intra-modal correlation and can better solve the problem of cross-modal heterogeneity. Extensive experiments have been conducted on the CUHK-PEDES dataset. Our approach obtains higher performance than state-of-the-art approaches, demonstrating the advantage of the approach we propose.

scholarly journals Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

2019 ◽  
Vol 9 (18) ◽  
pp. 3717 ◽  
Author(s):  
Wenkuan Li ◽  
Dongyuan Li ◽  
Hongxia Yin ◽  
Lindong Zhang ◽  
Zhenfang Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Classification Model ◽  
Linguistic Knowledge ◽  
Great Success ◽  
Text Representation ◽  
Attention Network ◽  
Sentiment Lexicon

Text representation learning is an important but challenging issue for various natural language processing tasks. Recently, deep learning-based representation models have achieved great success for sentiment classification. However, these existing models focus on more semantic information rather than sentiment linguistic knowledge, which provides rich sentiment information and plays a key role in sentiment analysis. In this paper, we propose a lexicon-enhanced attention network (LAN) based on text representation to improve the performance of sentiment classification. Specifically, we first propose a lexicon-enhanced attention mechanism by combining the sentiment lexicon with an attention mechanism to incorporate sentiment linguistic knowledge into deep learning methods. Second, we introduce a multi-head attention mechanism in the deep neural network to interactively capture the contextual information from different representation subspaces at different positions. Furthermore, we stack a LAN model to build a hierarchical sentiment classification model for large-scale text. Extensive experiments are conducted to evaluate the effectiveness of the proposed models on four popular real-world sentiment classification datasets at both the sentence level and the document level. The experimental results demonstrate that our proposed models can achieve comparable or better performance than the state-of-the-art methods.

scholarly journals Loss-Based Attention for Deep Multiple Instance Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 5742-5749
Author(s):  
Xiaoshuang Shi ◽  
Fuyong Xing ◽  
Yuanpu Xie ◽  
Zizhao Zhang ◽  
Lei Cui ◽  
...  
Keyword(s):  
State Of The Art ◽  
Classification Performance ◽  
Multiple Instance Learning ◽  
Attention Mechanism ◽  
Cross Entropy ◽  
General Category ◽  
Source Codes ◽  
Model Generalization ◽  
Fully Connected ◽  
Entropy Functions

Although attention mechanisms have been widely used in deep learning for many tasks, they are rarely utilized to solve multiple instance learning (MIL) problems, where only a general category label is given for multiple instances contained in one bag. Additionally, previous deep MIL methods firstly utilize the attention mechanism to learn instance weights and then employ a fully connected layer to predict the bag label, so that the bag prediction is largely determined by the effectiveness of learned instance weights. To alleviate this issue, in this paper, we propose a novel loss based attention mechanism, which simultaneously learns instance weights and predictions, and bag predictions for deep multiple instance learning. Specifically, it calculates instance weights based on the loss function, e.g. softmax+cross-entropy, and shares the parameters with the fully connected layer, which is to predict instance and bag predictions. Additionally, a regularization term consisting of learned weights and cross-entropy functions is utilized to boost the recall of instances, and a consistency cost is used to smooth the training process of neural networks for boosting the model generalization performance. Extensive experiments on multiple types of benchmark databases demonstrate that the proposed attention mechanism is a general, effective and efficient framework, which can achieve superior bag and image classification performance over other state-of-the-art MIL methods, with obtaining higher instance precision and recall than previous attention mechanisms. Source codes are available on https://github.com/xsshi2015/Loss-Attention.

Sign in / Sign up

Export Citation Format

Share Document