Emotion Speech Synthesis Method Based on Multi-Channel Time–Frequency Domain Generative Adversarial Networks (MC-TFD GANs) and Mixup

As one of the most challenging and promising topics in speech field, emotion speech synthesis is a hot topic in current research. At present, the emotion expression ability, synthesis speed and robustness of synthetic speech need to be improved. Cycle-consistent Adversarial Networks (CycleGAN) provides a two-way breakthrough in the transformation of emotional corpus information. But there is still a gap between the real target and the synthesis speech. In order to narrow this gap, we propose an emotion speech synthesis method combining multi-channel Time–frequency Domain Generative Adversarial Networks (MC-TFD GANs) and Mixup. It includes three stages: multichannel Time–frequency Domain GANs (MC-TFD GANs), loss estimation based on Mixup and effective emotion region stacking based on Mixup. Among them, the gating unit GTLU (gated tanh linear units) and the image expression method of speech saliency region are designed. It combines the Time–frequency Domain MaskCycleGAN based on improved GTLU and the time-domain CycleGAN based on saliency region to form the multi-channel GAN in the first stage. Based on Mixup method, the calculation method of loss and the aggravation degree of emotion region are designed. Compared with several popular speech synthesis methods, the comparative experiments were carried out on the interactive emotional dynamic motion capture (IEMOCAP) corpus. The bi-directional three-layer long short-term memory (LSTM) model was used as the verification model. The experimental results showed that the mean opinion score (MOS) and the unweighted accuracy (UA) of the speech generated by the synthesis method were improved, and the improvements were 4% and 2.7%, respectively. The current model was superior to the existing GANs model in subjective evaluation and objective experiments, ensure that the speech generated by this model had higher reliability, better fluency and emotional expression ability.

Markov Chain and Adaboost Image Saliency Detection Algorithm Based on Conditional Random Field

The traditional salient object detection algorithms are used to apply the underlying features and prior knowledge of the images. Based on conditional random field Markov chain and Adaboost image saliency detection technology, a saliency detection method is proposed to effectively reduce the error caused by the target approaching the edge, which mainly includes the use of absorption Markov chain model to generate the initial saliency map. In this model, the transition probability of each node is defined by the difference of color and texture between each super pixel, and the absorption time of the transition node is calculated as the significant value of each super pixel. A strong classifier optimization model based on Adaboost iterative algorithm is designed.The initial saliency map is processed by the classifier to obtain an optimized saliency map, which highlights the global contrast. In order to extract the saliency region of the final saliency map, a method using conditional random field is designed to segment and extract the saliency region. The results show that the saliency area detected by this method is prominent, the overall contour is clear and has high resolution. At the same time, this method has better performance in accuracy recall curve and histogram.

A Novel Low Power Method of Combining Saliency and Segmentation for Mobile Displays

Saliency, which means the area human vision is concentrated, can be used in many applications, such as enemy detection in solider goggles and person detection in an auto-driving car. In recent years, saliency is obtained instead of human eyes using a model in an automated way in HMD (Head Mounted Display), smartphones, and VR (Virtual Reality) devices based on mobile displays; however, such a mobile device needs too much power to maintain saliency on a mobile display. Therefore, low power saliency methods have been important. CURA tried to power down, according to the saliency level, while keeping human visual satisfaction. But it still has some artifacts due to the difference in brightness at the boundary of the region divided by saliency. In this paper, we propose a new segmentation-based saliency-aware low power approach to minimize the artifacts. Unlike CURA, our work considers visual perceptuality and power management at the saliency level and at the segmented region level for each saliency. Through experiments, our work achieves low power in each region divided by saliency and in the segmented regions in each saliency region, while maintaining human visual satisfaction for saliency. In addition, it maintains good image distortion quality while removing artifacts efficiently.

Saliency Region Detection Method Based on Background and Spatial Position

Saliency region detection methods have become one of the hotspots in the field of image processing as an important method to improve the real-time and accurate analysis of massive data. Integrating more effective prior knowledge is a viable direction for improving the performance of saliency region detection methods. Most of the methods based on background prior and boundary connectivity prior assume the boundary area of the image as the background, by restraining the background to highlight the salient area. When the boundary area of the image does not describe the background well (such as a large difference in border area features), if the entire frame of the image is put together to compute the background feature, the calculation of the background feature will be inaccurate. In view of the above shortcomings, this paper proposed a saliency region detection method based on background and spatial position. This method carried on the image boundary super pixel clustering, determined the background feature according to the clustering center, and used the difference between the super pixel on the image and the background super pixel, and its spatial position to calculate the salient of the super pixels. This approach used MATLAB to program and experiment. The method was compared with a series of the state-of-the-art methods. The AUC of proposed algorithm reaches 0.839, and the MAE is 0.220, showing the effectiveness of the proposed algorithm.