The document proposes a convolutional-recurrent network for speech enhancement. It discusses limitations of existing approaches and motivates exploiting temporal and spectral structure using convolutions and LSTMs. Experiments on a large dataset show the proposed method outperforms state-of-the-art methods in metrics like SNR, PESQ and WER, and generalizes well to unseen noise types.
The document proposes a convolutional-recurrent network for speech enhancement. It discusses limitations of existing approaches and motivates exploiting temporal and spectral structure using convolutions and LSTMs. Experiments on a large dataset show the proposed method outperforms state-of-the-art methods in metrics like SNR, PESQ and WER, and generalizes well to unseen noise types.
The document proposes a convolutional-recurrent network for speech enhancement. It discusses limitations of existing approaches and motivates exploiting temporal and spectral structure using convolutions and LSTMs. Experiments on a large dataset show the proposed method outperforms state-of-the-art methods in metrics like SNR, PESQ and WER, and generalizes well to unseen noise types.
The document proposes a convolutional-recurrent network for speech enhancement. It discusses limitations of existing approaches and motivates exploiting temporal and spectral structure using convolutions and LSTMs. Experiments on a large dataset show the proposed method outperforms state-of-the-art methods in metrics like SNR, PESQ and WER, and generalizes well to unseen noise types.
estimator (Ephraim et al. 1984), Subspace approach (Ephraim et al. 1995)
7 Background
Classic methods are based on statistical assumptions of
noise:
Pros: • Simple, and computationally efficient • Optimality under proper assumption • Interpretable
Cons: • Limited to stationary noise • Restricted to noise with specific characteristics
8 Data-driven Approach
What if we can collect large datasets of paired signals?
9 Data-driven Approach
What if we can collect large datasets of paired signals?
Given: • Paired signals
Goal: • Build function approximator such that
In short: regression based approach, usually
10 Data-driven Approach
Parametric regression using Neural Networks:
• Flexible for representation learning • Scale linearly in and • Natural paradigm for multi-task learning by sharing common representations
Figure from Lu el al., Interspeech 2013 11
Data-driven Approach
Related work for speech enhancement
• Recurrent network for noise reduction, Maas et al., ISCA 2012
• Deep denoising auto-encoder, Lu et al., Interspeech 2013 • Weighted denoising auto-encoder, Xia et al., Interspeech 2013 • DNN with symmetric context window, Xu et al., IEEE SPL 2014 • Hybrid of DNN suppression rule, Mirsamadi et al., Interspeech 2016
12 Data-driven Approach
Speech Enhancement Pipeline:
• Short-term Fourier Transform (STFT) to obtain time-frequency signal
STFT
• Build neural networks to approximate filter function such that
Focus of this talk • Apply Inverse-STFT (ISTFT) to reconstruct sound wave
ISTFT( ) 13 Convolutional-Recurrent Networks for SE
Problem setup:
Given time-frequency signal — spectrogram pair
where
For each utterance, usually frames and frequency bins.
14 Convolutional-Recurrent Networks for SE Observations:
Existing DNN-based approaches do not fully exploit the structure of
speech signals. • Frame-based DNN regression approach does not use the temporal locality of spectrogram • Fully connected DNN regression approach does not exploit the continuity of consecutive frequency bins in spectrogram
15 Convolutional-Recurrent Networks for SE Observations:
Existing DNN-based approaches do not fully exploit the structure of
speech signals. • Frame-based DNN regression approach does not use the temporal locality of spectrogram
• Use recurrent neural networks
• Fully connected DNN regression approach does not exploit the continuity of consecutive frequency bins in spectrogram
• Use convolutional neural networks
16 Convolutional-Recurrent Networks for SE Proposed: Convolution + bi-LSTM + Linear Regression
Objective:
17 Convolutional-Recurrent Networks for SE Proposed: Convolution + bi-LSTM + Linear Regression
At a high level, why will this model work?
• Continuity of signal in time and frequency domains • Convolution kernels as linear filters to match local patterns • bi-LSTM -> symmetric context window with adaptive window size • End-to-end learning without additional assumptions on noise type 18 Convolutional-Recurrent Networks for SE Convolution
(t, f’) 19 Convolutional-Recurrent Networks for SE Concatenation of feature maps
k feature maps, each with size (t, f’)
One feature map, with size (t, kf’)
20 Convolutional-Recurrent Networks for SE bi-directional LSTM State transition function of LSTM cell:
21 Convolutional-Recurrent Networks for SE Linear Regression with Projection At each time step t:
where is the output state of bi-LSTM at time step t.
Objective function and Optimization
MSE:
Optimization algorithm: AdaDelta 22
Experiments Dataset Single channel, Microsoft-internal data • Cortana utterances: male, female and children • Sampling rate: 16kHz • Storage format: 24bits precision • Each utterance: 5~9 seconds • Noise: subset of MS noise collection, 377 files with 25 types • 48 room impulse responses from MS RIR collection
Training Validation Test (seen noise) Test (unseen noise)
# utterances 7,500 1,500 1,500 1,500
23 Experiments Evaluation Metric • Signal-to-Noise Ratio (SNR) dB
• Log-spectral Distance (LSD)
• Mean-squared Error in time domain (MSE)
• Word error rate (WER)
• Perceptual evaluation of speech quality P.862 (PESQ)
Full Download Koneman’s Color Atlas and Textbook of Diagnostic Microbiology (Color Atlas & Textbook of Diagnostic Microbiology) 7th Edition, (Ebook PDF) PDF DOCX
Full Download Koneman’s Color Atlas and Textbook of Diagnostic Microbiology (Color Atlas & Textbook of Diagnostic Microbiology) 7th Edition, (Ebook PDF) PDF DOCX
