Fusion Based AER System Using Deep Learning Approach for Amplitude and Frequency Analysis

Automatic emotion recognition from Speech (AERS) systems based on acoustical analysis reveal that some emotional classes persist with ambiguity. This study employed an alternative method aimed at providing deep understanding into the amplitude–frequency, impacts of various emotions in order to aid in the advancement of near term, more effectively in classifying AER approaches. The study was undertaken by converting narrow 20 ms frames of speech into RGB or grey-scale spectrogram images. The features have been used to fine-tune a feature selection system that had previously been trained to recognise emotions. Two different Linear and Mel spectral scales are used to demonstrate a spectrogram. An inductive approach for in sighting the amplitude and frequency features of various emotional classes. We propose a two-channel profound combination of deep fusion network model for the efficient categorization of images. Linear and Mel- spectrogram is acquired from Speech-signal, which is prepared in the recurrence area to input Deep Neural Network. The proposed model Alex-Net with five convolutional layers and two fully connected layers acquire most vital features form spectrogram images plotted on the amplitude-frequency scale. The state-of-the-art is compared with benchmark dataset (EMO-DB). RGB and saliency images are fed to pre-trained Alex-Net tested both EMO-DB and Telugu dataset with an accuracy of 72.18% and fused image features less computations reaching to an accuracy 75.12%. The proposed model show that Transfer learning predict efficiently than Fine-tune network. When tested on Emo-DB dataset, the propȯsed system adequately learns discriminant features from speech spectrȯgrams and outperforms many stȧte-of-the-art techniques.

Flaw Detection in Highly Scattering Materials Using a Simple Ultrasonic Sensor Employing Adaptive Template Matching

Ultrasonic sensors have been extensively used in the nondestructive testing of materials for flaw detection. For polycrystalline materials, however, due to the scattering nature of the material, which results in strong grain noise and attenuation of the ultrasonic signal, accurate detection of flaws is particularly difficult. In this paper, a novel flaw-detection method using a simple ultrasonic sensor is proposed by exploiting time-frequency features of an ultrasonic signal. Since grain scattering mostly happens in the Rayleigh scattering region, it is possible to separate grain-scattered noise from flaw echoes in the frequency domain employing their spectral difference. We start with the spectral modeling of grain noise and flaw echo, and how the two spectra evolve with time is established. Then, a time-adaptive spectrum model for flaw echo is proposed, which serves as a template for the flaw-detection procedure. Next, a specially designed similarity measure is proposed, based on which the similarity between the template spectrum and the spectrum of the signal at each time point is evaluated sequentially, producing a series of matching coefficients termed moving window spectrum similarity (MWSS). The time-delay information of flaws is directly indicated by the peaks of MWSSs. Finally, the performance of the proposed method is validated by both simulated and experimental signals, showing satisfactory accuracy and efficiency.

Fault Diagnosis Method for Rotating Machinery Based on Hierarchical Amplitude-Aware Permutation Entropy and Pairwise Feature Proximity

With a view to solving the defect that multiscale amplitude-aware permutation entropy (MAAPE) can only quantify the low-frequency features of time series and ignore the high-frequency features which are equally important, a novel nonlinear time series feature extraction method, hierarchical amplitude-aware permutation entropy (HAAPE), is proposed. By constructing high and low-frequency operators, this method can extract the features of different frequency bands of time series simultaneously, so as to avoid the issue of information loss. In view of its advantages, HAAPE is introduced into the field of fault diagnosis to extract fault features from vibration signals of rotating machinery. Combined with the pairwise feature proximity (PWFP) feature selection method and gray wolf algorithm optimization support vector machine (GWO-SVM), a new intelligent fault diagnosis method for rotating machinery is proposed. In our method, firstly, HAPPE is adopted to extract the original high and low-frequency fault features of rotating machinery. After that, PWFP is used to sort the original features, and the important features are filtered to obtain low-dimensional sensitive feature vectors. Finally, the sensitive feature vectors are input into GWO-SVM for training and testing, so as to realize the fault identification of rotating machinery. The performance of the proposed method is verified using two data sets of bearing and gearbox. The results show that the proposed method enjoys obvious advantages over the existing methods, and the identification accuracy reaches 100%.

Environmental sound classification using temporal-frequency attention based convolutional neural network

Environmental sound classification is one of the important issues in the audio recognition field. Compared with structured sounds such as speech and music, the time–frequency structure of environmental sounds is more complicated. In order to learn time and frequency features from Log-Mel spectrogram more effectively, a temporal-frequency attention based convolutional neural network model (TFCNN) is proposed in this paper. Firstly, an experiment that is used as motivation in proposed method is designed to verify the effect of a specific frequency band in the spectrogram on model classification. Secondly, two new attention mechanisms, temporal attention mechanism and frequency attention mechanism, are proposed. These mechanisms can focus on key frequency bands and semantic related time frames on the spectrogram to reduce the influence of background noise and irrelevant frequency bands. Then, a feature information complementarity is formed by combining these mechanisms to more accurately capture the critical time–frequency features. In such a way, the representation ability of the network model can be greatly improved. Finally, experiments on two public data sets, UrbanSound 8 K and ESC-50, demonstrate the effectiveness of the proposed method.

Rotor UAV’s Micro-Doppler Signal Detection and Parameter Estimation Based on FRFT-FSST

The micro-Doppler signal generated by the rotors of an Unmanned Aerial Vehicle (UAV) contains the structural features and motion information of the target, which can be used for detection and classification of the target, however, the standard STFT has the problems such as the lower time-frequency resolution and larger error in rotor parameter estimation, an FRFT (Fractional Fourier Transform)-FSST(STFT based synchrosqueezing)-based method for micro-Doppler signal detection and parameter estimation is proposed in this paper. Firstly, the FRFT is used in the proposed method to eliminate the influence of the velocity and acceleration of the target on the time-frequency features of the echo signal from the rotors. Secondly, the higher time-frequency resolution of FSST is used to extract the time-frequency features of micro-Doppler signals. Moreover, the specific solution methodologies for the selection of window length in STFT and the estimation of rotor parameters are given in the proposed method. Finally, the effectiveness and accuracy of the proposed method for target detection and rotor parameter estimation are verified through simulation and measured data.

A Novel Method of Temporomandibular Joint Hypermobility Diagnosis Based on Signal Analysis

Despite the temporomandibular joint (TMJ) being a well-known anatomical structure its diagnosis may become difficult because physiological sounds accompanying joint movement can falsely indicate pathological symptoms. One example of such a situation is temporomandibular joint hypermobility (TMJH), which still requires comprehensive study. The commonly used official research diagnostic criteria for temporomandibular disorders (RDC/TMD) does not support the recognition of TMJH. Therefore, in this paper the authors propose a novel diagnostic method of TMJH based on the digital time–frequency analysis of sounds generated by TMJ. Forty-seven volunteers were diagnosed using the RDC/TMD questionnaire and auscultated with the Littmann 3200 electronic stethoscope on both sides of the head simultaneously. Recorded TMJ sounds were transferred to the computer via Bluetooth® for numerical analysis. The representation of the signals in the time–frequency domain was computed with the use of the Python Numpy and Matplotlib libraries and short-time Fourier transform. The research reveals characteristic time–frequency features in acoustic signals which can be used to detect TMJH. It is also proved that TMJH is a rare disorder; however, its prevalence at the level of around 4% is still significant.