Deep Learning-Based Amplitude Fusion for Speech Dereverberation

Mapping and masking are two important speech enhancement methods based on deep learning that aim to recover the original clean speech from corrupted speech. In practice, too large recovery errors severely restrict the improvement in speech quality. In our preliminary experiment, we demonstrated that mapping and masking methods had different conversion mechanisms and thus assumed that their recovery errors are highly likely to be complementary. Also, the complementarity was validated accordingly. Based on the principle of error minimization, we propose the fusion between mapping and masking for speech dereverberation. Specifically, we take the weighted mean of the amplitudes recovered by the two methods as the estimated amplitude of the fusion method. Experiments verify that the recovery error of the fusion method is further controlled. Compared with the existing geometric mean method, the weighted mean method we proposed has achieved better results. Speech dereverberation experiments manifest that the weighted mean method improves PESQ and SNR by 5.8% and 25.0%, respectively, compared with the traditional masking method.

Download Full-text

Low-complexity artificial noise suppression methods for deep learning-based speech enhancement algorithms

EURASIP Journal on Audio Speech and Music Processing ◽

10.1186/s13636-021-00204-9 ◽

2021 ◽

Vol 2021 (1) ◽

Author(s):

Yuxuan Ke ◽

Andong Li ◽

Chengshi Zheng ◽

Renhua Peng ◽

Xiaodong Li

Keyword(s):

Deep Learning ◽

Speech Enhancement ◽

Noise Suppression ◽

Signal To Noise Ratio ◽

Low Complexity ◽

Speech Quality ◽

Artificial Noise ◽

Noise Power ◽

Noise Masking ◽

Residual Noise

AbstractDeep learning-based speech enhancement algorithms have shown their powerful ability in removing both stationary and non-stationary noise components from noisy speech observations. But they often introduce artificial residual noise, especially when the training target does not contain the phase information, e.g., ideal ratio mask, or the clean speech magnitude and its variations. It is well-known that once the power of the residual noise components exceeds the noise masking threshold of the human auditory system, the perceptual speech quality may degrade. One intuitive way is to further suppress the residual noise components by a postprocessing scheme. However, the highly non-stationary nature of this kind of residual noise makes the noise power spectral density (PSD) estimation a challenging problem. To solve this problem, the paper proposes three strategies to estimate the noise PSD frame by frame, and then the residual noise can be removed effectively by applying a gain function based on the decision-directed approach. The objective measurement results show that the proposed postfiltering strategies outperform the conventional postfilter in terms of segmental signal-to-noise ratio (SNR) as well as speech quality improvement. Moreover, the AB subjective listening test shows that the preference percentages of the proposed strategies are over 60%.

Download Full-text

Deep Noise Tracking Network: A Hybrid Signal Processing/Deep Learning Approach to Speech Enhancement

10.21437/interspeech.2018-1020 ◽

2018 ◽

Cited By ~ 1

Author(s):

Shuai Nie ◽

Shan Liang ◽

Bin Liu ◽

Yaping Zhang ◽

Wenju Liu ◽

...

Keyword(s):

Signal Processing ◽

Deep Learning ◽

Speech Enhancement ◽

Learning Approach

Download Full-text

Deep Learning Based Real-time Speech Enhancement for Dual-microphone Mobile Phones

IEEE/ACM Transactions on Audio Speech and Language Processing ◽

10.1109/taslp.2021.3082318 ◽

2021 ◽

pp. 1-1

Author(s):

Ke Tan ◽

Xueliang Zhang ◽

Deliang Wang

Keyword(s):

Deep Learning ◽

Real Time ◽

Mobile Phones ◽

Speech Enhancement

Download Full-text

A Robust Dual-Microphone Generalized Sidelobe Canceller Using a Bone-Conduction Sensor for Speech Enhancement

Sensors ◽

10.3390/s21051878 ◽

2021 ◽

Vol 21 (5) ◽

pp. 1878

Author(s):

Yi Zhou ◽

Haiping Wang ◽

Yijing Chu ◽

Hongqing Liu

Keyword(s):

Speech Enhancement ◽

Speech Intelligibility ◽

Bone Conduction ◽

Speech Quality ◽

Generalized Sidelobe Canceller ◽

Spatially Distributed ◽

Interference Signals ◽

Adaptive Noise ◽

Adaptive Noise Canceller ◽

Sidelobe Canceller

The use of multiple spatially distributed microphones allows performing spatial filtering along with conventional temporal filtering, which can better reject the interference signals, leading to an overall improvement of the speech quality. In this paper, we propose a novel dual-microphone generalized sidelobe canceller (GSC) algorithm assisted by a bone-conduction (BC) sensor for speech enhancement, which is named BC-assisted GSC (BCA-GSC) algorithm. The BC sensor is relatively insensitive to the ambient noise compared to the conventional air-conduction (AC) microphone. Hence, BC speech can be analyzed to generate very accurate voice activity detection (VAD), even in a high noise environment. The proposed algorithm incorporates the VAD information obtained by the BC speech into the adaptive blocking matrix (ABM) and adaptive noise canceller (ANC) in GSC. By using VAD to control ABM and combining VAD with signal-to-interference ratio (SIR) to control ANC, the proposed method could suppress interferences and improve the overall performance of GSC significantly. It is verified by experiments that the proposed GSC system not only improves speech quality remarkably but also boosts speech intelligibility.

Download Full-text

Regression-Based Noise Modeling for Speech Signal Processing

Fluctuation and Noise Letters ◽

10.1142/s021947752150022x ◽

2021 ◽

pp. 2150022

Author(s):

Caio Cesar Enside de Abreu ◽

Marco Aparecido Queiroz Duarte ◽

Bruno Rodrigues de Oliveira ◽

Jozue Vieira Filho ◽

Francisco Villarreal

Keyword(s):

Speech Enhancement ◽

Speech Processing ◽

Acoustic Analysis ◽

Voice Quality ◽

Wiener Filter ◽

Processing System ◽

Speech Quality ◽

Speech Signals ◽

Speech Signal Processing ◽

Acoustic Environment

Speech processing systems are very important in different applications involving speech and voice quality such as automatic speech recognition, forensic phonetics and speech enhancement, among others. In most of them, the acoustic environmental noise is added to the original signal, decreasing the signal-to-noise ratio (SNR) and the speech quality by consequence. Therefore, estimating noise is one of the most important steps in speech processing whether to reduce it before processing or to design robust algorithms. In this paper, a new approach to estimate noise from speech signals is presented and its effectiveness is tested in the speech enhancement context. For this purpose, partial least squares (PLS) regression is used to model the acoustic environment (AE) and a Wiener filter based on a priori SNR estimation is implemented to evaluate the proposed approach. Six noise types are used to create seven acoustically modeled noises. The basic idea is to consider the AE model to identify the noise type and estimate its power to be used in a speech processing system. Speech signals processed using the proposed method and classical noise estimators are evaluated through objective measures. Results show that the proposed method produces better speech quality than state-of-the-art noise estimators, enabling it to be used in real-time applications in the field of robotic, telecommunications and acoustic analysis.

Download Full-text

Speech Enhancement Using Deep Learning Methods: A Review

Jurnal Elektronika dan Telekomunikasi ◽

10.14203/jet.v21.19-26 ◽

2021 ◽

Vol 21 (1) ◽

pp. 19

Author(s):

Asri Rizki Yuliani ◽

M. Faizal Amri ◽

Endang Suryawati ◽

Ade Ramdan ◽

Hilman Ferdinandus Pardede

Keyword(s):

Neural Network ◽

Deep Learning ◽

Speech Enhancement ◽

Speech Signal ◽

Research Field ◽

Learning Technologies ◽

Learning Approaches ◽

Speech Signal Processing ◽

Generative Adversarial Network ◽

Advantages And Disadvantages

Speech enhancement, which aims to recover the clean speech of the corrupted signal, plays an important role in the digital speech signal processing. According to the type of degradation and noise in the speech signal, approaches to speech enhancement vary. Thus, the research topic remains challenging in practice, specifically when dealing with highly non-stationary noise and reverberation. Recent advance of deep learning technologies has provided great support for the progress in speech enhancement research field. Deep learning has been known to outperform the statistical model used in the conventional speech enhancement. Hence, it deserves a dedicated survey. In this review, we described the advantages and disadvantages of recent deep learning approaches. We also discussed challenges and trends of this field. From the reviewed works, we concluded that the trend of the deep learning architecture has shifted from the standard deep neural network (DNN) to convolutional neural network (CNN), which can efficiently learn temporal information of speech signal, and generative adversarial network (GAN), that utilize two networks training.

Download Full-text