Audio source separation with time-frequency velocities

2014 ◽  
Author(s):  
Guy Wolf ◽  
Stephane Mallat ◽  
Shihab Shamma
Keyword(s):  
Source Separation ◽  
Time Frequency ◽  
Audio Source Separation

scholarly journals Sensing ecosystem dynamics via audio source separation: A case study of marine soundscapes off northeastern Taiwan

2021 ◽  
Vol 17 (2) ◽  
pp. e1008698
Author(s):  
Tzu-Hao Lin ◽  
Tomonari Akamatsu ◽  
Yu Tsao
Keyword(s):  
Conservation Management ◽  
Marine Ecosystem ◽  
Source Separation ◽  
Ecosystem Dynamics ◽  
Marine Animals ◽  
Time Frequency ◽  
Audio Source Separation ◽  
Diversity Assessment ◽  
Separation Model

Remote acquisition of information on ecosystem dynamics is essential for conservation management, especially for the deep ocean. Soundscape offers unique opportunities to study the behavior of soniferous marine animals and their interactions with various noise-generating activities at a fine temporal resolution. However, the retrieval of soundscape information remains challenging owing to limitations in audio analysis techniques that are effective in the face of highly variable interfering sources. This study investigated the application of a seafloor acoustic observatory as a long-term platform for observing marine ecosystem dynamics through audio source separation. A source separation model based on the assumption of source-specific periodicity was used to factorize time-frequency representations of long-duration underwater recordings. With minimal supervision, the model learned to discriminate source-specific spectral features and prove to be effective in the separation of sounds made by cetaceans, soniferous fish, and abiotic sources from the deep-water soundscapes off northeastern Taiwan. Results revealed phenological differences among the sound sources and identified diurnal and seasonal interactions between cetaceans and soniferous fish. The application of clustering to source separation results generated a database featuring the diversity of soundscapes and revealed a compositional shift in clusters of cetacean vocalizations and fish choruses during diurnal and seasonal cycles. The source separation model enables the transformation of single-channel audio into multiple channels encoding the dynamics of biophony, geophony, and anthropophony, which are essential for characterizing the community of soniferous animals, quality of acoustic habitat, and their interactions. Our results demonstrated the application of source separation could facilitate acoustic diversity assessment, which is a crucial task in soundscape-based ecosystem monitoring. Future implementation of soundscape information retrieval in long-term marine observation networks will lead to the use of soundscapes as a new tool for conservation management in an increasingly noisy ocean.

scholarly journals Integrating Dilated Convolution into DenseLSTM for Audio Source Separation

2021 ◽  
Vol 11 (2) ◽  
pp. 789
Author(s):  
Woon-Haeng Heo ◽  
Hyemi Kim ◽  
Oh-Wook Kwon
Keyword(s):  
Deep Learning ◽  
Speech Signal ◽  
Source Separation ◽  
Series Data ◽  
Separation Performance ◽  
Time Frequency ◽  
Dilated Convolution ◽  
Audio Source Separation ◽  
Music Signal ◽  
Learning Architectures

Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.

Sign in / Sign up

Export Citation Format

Share Document