F0 Modeling for Isarn Speech Synthesis using Deep Neural Networks and Syllable-level Feature Representation

2020 ◽  
Vol 17 (6) ◽  
pp. 906-915
Author(s):  
Pongsathon Janyoi ◽  
Pusadee Seresangtakul
Keyword(s):  
Speech Synthesis ◽  
Deep Neural Networks ◽  
Markov Models ◽  
Feature Representation ◽  
Context Dependency ◽  
Dynamic Features ◽  
Synthesis System ◽  
Proposed Model ◽  
Training Sets

The generation of the fundamental frequency (F0) plays an important role in speech synthesis, which directly influences the naturalness of synthetic speech. In conventional parametric speech synthesis, F0 is predicted frame-by-frame. This method is insufficient to represent F0 contours in larger units, especially tone contours of syllables in tonal languages that deviate as a result of long-term context dependency. This work proposes a syllable-level F0 model that represents F0 contours within syllables, using syllable-level F0 parameters that comprise the sampling F0 points and dynamic features. A Deep Neural Network (DNN) was used to represent the relationships between syllable-level contextual features and syllable-level F0 parameters. The proposed model was examined using an Isarn speech synthesis system with both large and small training sets. For all training sets, the results of objective and subjective tests indicate that the proposed approach outperforms the baseline systems based on hidden Markov models and DNNS that predict F0 values at the frame level

LSTM Deep Neural Networks Postfiltering for Enhancing Synthetic Voices

2017 ◽  
Vol 32 (01) ◽  
pp. 1860008 ◽  
Author(s):  
Marvin Coto-Jiménez ◽  
John Goddard-Close
Keyword(s):  
Neural Networks ◽  
Speech Synthesis ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Markov Models ◽  
Natural Speech ◽  
Objective Measures ◽  
Recent Developments ◽  
Small Footprint ◽  
Synthetic Voices

Recent developments in speech synthesis have produced systems capable of producing speech which closely resembles natural speech, and researchers now strive to create models that more accurately mimic human voices. One such development is the incorporation of multiple linguistic styles in various languages and accents. Speech synthesis based on Hidden Markov Models (HMM) is of great interest to researchers, due to its ability to produce sophisticated features with a small footprint. Despite some progress, its quality has not yet reached the level of the current predominant unit-selection approaches, which select and concatenate recordings of real speech, and work has been conducted to try to improve HMM-based systems. In this paper, we present an application of long short-term memory (LSTM) deep neural networks as a postfiltering step in HMM-based speech synthesis. Our motivation stems from a similar desire to obtain characteristics which are closer to those of natural speech. The paper analyzes four types of postfilters obtained using five voices, which range from a single postfilter to enhance all the parameters, to a multi-stream proposal which separately enhances groups of parameters. The different proposals are evaluated using three objective measures and are statistically compared to determine any significance between them. The results described in the paper indicate that HMM-based voices can be enhanced using this approach, specially for the multi-stream postfilters on the considered objective measures.

scholarly journals Development of Vietnamese Speech Synthesis System using Deep Neural Networks

2019 ◽  
Vol 34 (4) ◽  
pp. 349-363 ◽  
Author(s):  
Thinh Van Nguyen ◽  
Bao Quoc Nguyen ◽  
Kinh Huy Phan ◽  
Hai Van Do
Keyword(s):  
Neural Networks ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Speech Synthesis ◽  
Deep Neural Networks ◽  
Hidden Markov ◽  
Experimental Results ◽  
Training Data ◽  
The Internet ◽  
Synthesis System

In this paper, we present our first Vietnamese speech synthesis system based on deep neural networks. To improve the training data collected from the Internet, a cleaning method is proposed. The experimental results indicate that by using deeper architectures we can achieve better performance for the TTS than using shallow architectures such as hidden Markov model. We also present the effect of using different amounts of data to train the TTS systems. In the VLSP TTS challenge 2018, our proposed DNN-based speech synthesis system won the first place in all three subjects including naturalness, intelligibility, and MOS.

scholarly journals Tonal Contour Generation for Isarn Speech Synthesis Using Deep Learning and Sampling-Based F0 Representation

2020 ◽  
Vol 10 (18) ◽  
pp. 6381 ◽  
Author(s):  
Pongsathon Janyoi ◽  
Pusadee Seresangtakul
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Speech Synthesis ◽  
Critical Factor ◽  
Dynamic Features ◽  
Linguistic Features ◽  
Proposed Model ◽  
F0 Contour ◽  
Tonal Contour

The modeling of fundamental frequency (F0) in speech synthesis is a critical factor affecting the intelligibility and naturalness of synthesized speech. In this paper, we focus on improving the modeling of F0 for Isarn speech synthesis. We propose the F0 model for this based on a recurrent neural network (RNN). Sampled values of F0 are used at the syllable level of continuous Isarn speech combined with their dynamic features to represent supra-segmental properties of the F0 contour. Different architectures of the deep RNNs and different combinations of linguistic features are analyzed to obtain conditions for the best performance. To assess the proposed method, we compared it with several RNN-based baselines. The results of objective and subjective tests indicate that the proposed model significantly outperformed the baseline RNN model that predicts values of F0 at the frame level, and the baseline RNN model that represents the F0 contours of syllables by using discrete cosine transform.

scholarly journals Isarn Dialect Speech Synthesis using HMM with syllable-context features

2018 ◽  
Vol 12 (2) ◽  
pp. 81-89
Author(s):  
Pongsathon Janyoi ◽  
Pusadee Seresangtakul
Keyword(s):  
Speech Synthesis ◽  
Markov Models ◽  
Synthetic Speech ◽  
Synthesis System ◽  
Fundamental Frequencies ◽  
Input Text ◽  
Regional Dialect ◽  
Context Dependent ◽  
Prosody Generation ◽  
Context Features

This paper describes the Isarn speech synthesis system, which is a regional dialect spoken in the Northeast of Thailand. In this study, we focus to improve the prosody generation of the system by using the additional context features. In order to develop the system, the speech parameters (Mel-ceptrum and fundamental frequencies of phoneme within different phonetic contexts) were modelled using Hidden Markov Models (HMM). Synthetic speech was generated by converting the input text into context-dependent phonemes. Speech parameters were generated from the trained HMM, according to the context-dependent phonemes, and were then synthesized through a speech vocoder. In this study, systems were trained using three different feature sets: basic contextual features, tonal, and syllable-context features. Objective and subjective tests were conducted to determine the performance of the proposed system. The results indicated that the addition of the syllable-context features significantly improved the naturalness of synthesized speech.

Modeling of unforced demand response programs

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Hassan Jalili ◽  
Pierluigi Siano
Keyword(s):  
Demand Response ◽  
Load Shifting ◽  
Response Models ◽  
Temperature Changes ◽  
Proposed Model ◽  
Equipment Failures ◽  
Demand Response Programs ◽  
Simultaneous Implementation ◽  
Long Term Studies

Abstract Demand response programs are useful options in reducing electricity price, congestion relief, load shifting, peak clipping, valley filling and resource adequacy from the system operator’s viewpoint. For this purpose, many models of these programs have been developed. However, the availability of these resources has not been properly modeled in demand response models making them not practical for long-term studies such as in the resource adequacy problem where considering the providers’ responding uncertainties is necessary for long-term studies. In this paper, a model considering providers’ unavailability for unforced demand response programs has been developed. Temperature changes, equipment failures, simultaneous implementation of demand side management resources, popular TV programs and family visits are the main reasons that may affect the availability of the demand response providers to fulfill their commitments. The effectiveness of the proposed model has been demonstrated by numerical simulation.

scholarly journals IoT-Based Bee Swarm Activity Acoustic Classification Using Deep Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 676
Author(s):  
Andrej Zgank
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Markov Models ◽  
Audio Signal ◽  
Audio Signals ◽  
Mel Frequency Cepstral Coefficients ◽  
Animal Activity ◽  
The Impact ◽  
Acoustic Classification ◽  
Swarm Activity

Animal activity acoustic monitoring is becoming one of the necessary tools in agriculture, including beekeeping. It can assist in the control of beehives in remote locations. It is possible to classify bee swarm activity from audio signals using such approaches. A deep neural networks IoT-based acoustic swarm classification is proposed in this paper. Audio recordings were obtained from the Open Source Beehive project. Mel-frequency cepstral coefficients features were extracted from the audio signal. The lossless WAV and lossy MP3 audio formats were compared for IoT-based solutions. An analysis was made of the impact of the deep neural network parameters on the classification results. The best overall classification accuracy with uncompressed audio was 94.09%, but MP3 compression degraded the DNN accuracy by over 10%. The evaluation of the proposed deep neural networks IoT-based bee activity acoustic classification showed improved results if compared to the previous hidden Markov models system.

scholarly journals UAV Image Multi-Labeling with Data-Efficient Transformers

2021 ◽  
Vol 11 (9) ◽  
pp. 3974
Author(s):  
Laila Bashmal ◽  
Yakoub Bazi ◽  
Mohamad Mahmoud Al Rahhal ◽  
Haikel Alhichri ◽  
Naif Al Ajlan
Keyword(s):  
Data Augmentation ◽  
Feature Representation ◽  
Aerial Image ◽  
Remote Sensing Images ◽  
Training Set ◽  
Proposed Model ◽  
Class Labels ◽  
Using Data ◽  
Uav Image

In this paper, we present an approach for the multi-label classification of remote sensing images based on data-efficient transformers. During the training phase, we generated a second view for each image from the training set using data augmentation. Then, both the image and its augmented version were reshaped into a sequence of flattened patches and then fed to the transformer encoder. The latter extracts a compact feature representation from each image with the help of a self-attention mechanism, which can handle the global dependencies between different regions of the high-resolution aerial image. On the top of the encoder, we mounted two classifiers, a token and a distiller classifier. During training, we minimized a global loss consisting of two terms, each corresponding to one of the two classifiers. In the test phase, we considered the average of the two classifiers as the final class labels. Experiments on two datasets acquired over the cities of Trento and Civezzano with a ground resolution of two-centimeter demonstrated the effectiveness of the proposed model.

scholarly journals Context-Aware Neural Machine Translation for Korean Honorific Expressions

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1589
Author(s):  
Yongkeun Hwang ◽  
Yanghoon Kim ◽  
Kyomin Jung
Keyword(s):  
Machine Translation ◽  
Deep Neural Networks ◽  
Contextual Information ◽  
Context Aware ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Sentence Level ◽  
Proposed Model ◽  
The Given ◽  
The Relationship

Neural machine translation (NMT) is one of the text generation tasks which has achieved significant improvement with the rise of deep neural networks. However, language-specific problems such as handling the translation of honorifics received little attention. In this paper, we propose a context-aware NMT to promote translation improvements of Korean honorifics. By exploiting the information such as the relationship between speakers from the surrounding sentences, our proposed model effectively manages the use of honorific expressions. Specifically, we utilize a novel encoder architecture that can represent the contextual information of the given input sentences. Furthermore, a context-aware post-editing (CAPE) technique is adopted to refine a set of inconsistent sentence-level honorific translations. To demonstrate the efficacy of the proposed method, honorific-labeled test data is required. Thus, we also design a heuristic that labels Korean sentences to distinguish between honorific and non-honorific styles. Experimental results show that our proposed method outperforms sentence-level NMT baselines both in overall translation quality and honorific translations.

scholarly journals Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Jianlei Zhang ◽  
Yukun Zeng ◽  
Binil Starly
Keyword(s):  
Neural Network ◽  
Tool Wear ◽  
Machine Tool ◽  
Recurrent Neural Network ◽  
Machine Tools ◽  
Prediction Performance ◽  
Sequential Data ◽  
Diagnosis And Prognosis ◽  
Proposed Model

AbstractData-driven approaches for machine tool wear diagnosis and prognosis are gaining attention in the past few years. The goal of our study is to advance the adaptability, flexibility, prediction performance, and prediction horizon for online monitoring and prediction. This paper proposes the use of a recent deep learning method, based on Gated Recurrent Neural Network architecture, including Long Short Term Memory (LSTM), which try to captures long-term dependencies than regular Recurrent Neural Network method for modeling sequential data, and also the mechanism to realize the online diagnosis and prognosis and remaining useful life (RUL) prediction with indirect measurement collected during the manufacturing process. Existing models are usually tool-specific and can hardly be generalized to other scenarios such as for different tools or operating environments. Different from current methods, the proposed model requires no prior knowledge about the system and thus can be generalized to different scenarios and machine tools. With inherent memory units, the proposed model can also capture long-term dependencies while learning from sequential data such as those collected by condition monitoring sensors, which means it can be accommodated to machine tools with varying life and increase the prediction performance. To prove the validity of the proposed approach, we conducted multiple experiments on a milling machine cutting tool and applied the model for online diagnosis and RUL prediction. Without loss of generality, we incorporate a system transition function and system observation function into the neural net and trained it with signal data from a minimally intrusive vibration sensor. The experiment results showed that our LSTM-based model achieved the best overall accuracy among other methods, with a minimal Mean Square Error (MSE) for tool wear prediction and RUL prediction respectively.

Sign in / Sign up

Export Citation Format

Share Document