scholarly journals Multi-channel spectrograms for speech processing applications using deep learning methods

2020 ◽  
Author(s):  
T. Arias-Vergara ◽  
P. Klumpp ◽  
J. C. Vasquez-Correa ◽  
E. Nöth ◽  
J. R. Orozco-Arroyave ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Speech Processing ◽  
Single Channel ◽  
Audio Signal ◽  
Frequency Component ◽  
Feature Maps ◽  
Time Frequency ◽  
Convolutional Networks ◽  
Channel Input

Abstract Time–frequency representations of the speech signals provide dynamic information about how the frequency component changes with time. In order to process this information, deep learning models with convolution layers can be used to obtain feature maps. In many speech processing applications, the time–frequency representations are obtained by applying the short-time Fourier transform and using single-channel input tensors to feed the models. However, this may limit the potential of convolutional networks to learn different representations of the audio signal. In this paper, we propose a methodology to combine three different time–frequency representations of the signals by computing continuous wavelet transform, Mel-spectrograms, and Gammatone spectrograms and combining then into 3D-channel spectrograms to analyze speech in two different applications: (1) automatic detection of speech deficits in cochlear implant users and (2) phoneme class recognition to extract phone-attribute features. For this, two different deep learning-based models are considered: convolutional neural networks and recurrent neural networks with convolution layers.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

Analysis of Vibration Signals in Monitoring Titanium End Milling Process Using Triaxial Accelerometer

2021 ◽  
Author(s):  
John Henry Navarro-Devia ◽  
Dzung Viet Dao ◽  
Yun Chen ◽  
Huaizhong Li
Keyword(s):  
Titanium Alloys ◽  
Frequency Domain ◽  
Single Channel ◽  
End Milling ◽  
Cutting Parameters ◽  
Frequency Component ◽  
Spindle Speed ◽  
Triaxial Accelerometer ◽  
Vibration Signals ◽  
Time Frequency

Abstract Vibrations during milling of hard-to-cut materials can cause low productivity, inferior quality and short tool life. It is one of the common issues in the machining of hard-to-cut materials employed in aerospace applications, such as titanium alloys. This paper presents an analysis of the vibration signals in the 3 axes of movement during titanium end milling, under diverse cutting parameters, manipulating spindle speed and feed rate. Signals were obtained using a triaxial accelerometer and processed in MATLAB. The analysis was conducted in the frequency-domain and the time-frequency domain. The results show that high-frequency vibration could occur in any direction with different amplitudes. Response on each axis depends on spindle speed, feed, and type of milling. A frequency component continually appeared in each axis regardless of cutting conditions and is located near the natural frequencies. Finally, the triaxial accelerations were compared for the milling cases with a new and a worn tool. Results highlight the importance and need for continuous monitoring of vibration in the 3 axes, instead of only using a single-channel signal, providing experimental data which could expand knowledge relating to the milling of titanium alloys.

scholarly journals Integrating Audio Signal Processing and Deep Learning Algorithms for Gait Pattern Classification in Brazilian Gaited Horses

2021 ◽  
Vol 2 ◽  
Author(s):  
Anderson Antonio Carvalho Alves ◽  
Lucas Tassoni Andrietta ◽  
Rafael Zinni Lopes ◽  
Fernando Oliveira Bussiman ◽  
Fabyano Fonseca e Silva ◽  
...  
Keyword(s):  
Neural Networks ◽  
Signal Processing ◽  
Deep Learning ◽  
Audio Signal ◽  
Gait Pattern ◽  
Classification Performance ◽  
Audio Signal Processing ◽  
Gait Patterns ◽  
Audio Features ◽  
Gaited Horses

This study focused on assessing the usefulness of using audio signal processing in the gaited horse industry. A total of 196 short-time audio files (4 s) were collected from video recordings of Brazilian gaited horses. These files were converted into waveform signals (196 samples by 80,000 columns) and divided into training (N = 164) and validation (N = 32) datasets. Twelve single-valued audio features were initially extracted to summarize the training data according to the gait patterns (Marcha Batida—MB and Marcha Picada—MP). After preliminary analyses, high-dimensional arrays of the Mel Frequency Cepstral Coefficients (MFCC), Onset Strength (OS), and Tempogram (TEMP) were extracted and used as input information in the classification algorithms. A principal component analysis (PCA) was performed using the 12 single-valued features set and each audio-feature dataset—AFD (MFCC, OS, and TEMP) for prior data visualization. Machine learning (random forest, RF; support vector machine, SVM) and deep learning (multilayer perceptron neural networks, MLP; convolution neural networks, CNN) algorithms were used to classify the gait types. A five-fold cross-validation scheme with 10 repetitions was employed for assessing the models' predictive performance. The classification performance across models and AFD was also validated with independent observations. The models and AFD were compared based on the classification accuracy (ACC), specificity (SPEC), sensitivity (SEN), and area under the curve (AUC). In the logistic regression analysis, five out of the 12 audio features extracted were significant (p < 0.05) between the gait types. ACC averages ranged from 0.806 to 0.932 for MFCC, from 0.758 to 0.948 for OS and, from 0.936 to 0.968 for TEMP. Overall, the TEMP dataset provided the best classification accuracies for all models. The most suitable method for audio-based horse gait pattern classification was CNN. Both cross and independent validation schemes confirmed that high values of ACC, SPEC, SEN, and AUC are expected for yet-to-be-observed labels, except for MFCC-based models, in which clear overfitting was observed. Using audio-generated data for describing gait phenotypes in Brazilian horses is a promising approach, as the two gait patterns were correctly distinguished. The highest classification performance was achieved by combining CNN and the rhythmic-descriptive AFD.

scholarly journals An Investigation of Deep Learning Models for EEG-Based Emotion Recognition

2020 ◽  
Vol 14 ◽  
Author(s):  
Yaqing Zhang ◽  
Jinling Chen ◽  
Jen Hong Tan ◽  
Yuxuan Chen ◽  
Yunyi Chen ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Emotion Recognition ◽  
Real Life ◽  
Learning Rate ◽  
Learning Models ◽  
Learning Methods ◽  
Time Frequency ◽  
Comparison Results

Emotion is the human brain reacting to objective things. In real life, human emotions are complex and changeable, so research into emotion recognition is of great significance in real life applications. Recently, many deep learning and machine learning methods have been widely applied in emotion recognition based on EEG signals. However, the traditional machine learning method has a major disadvantage in that the feature extraction process is usually cumbersome, which relies heavily on human experts. Then, end-to-end deep learning methods emerged as an effective method to address this disadvantage with the help of raw signal features and time-frequency spectrums. Here, we investigated the application of several deep learning models to the research field of EEG-based emotion recognition, including deep neural networks (DNN), convolutional neural networks (CNN), long short-term memory (LSTM), and a hybrid model of CNN and LSTM (CNN-LSTM). The experiments were carried on the well-known DEAP dataset. Experimental results show that the CNN and CNN-LSTM models had high classification performance in EEG-based emotion recognition, and their accurate extraction rate of RAW data reached 90.12 and 94.17%, respectively. The performance of the DNN model was not as accurate as other models, but the training speed was fast. The LSTM model was not as stable as the CNN and CNN-LSTM models. Moreover, with the same number of parameters, the training speed of the LSTM was much slower and it was difficult to achieve convergence. Additional parameter comparison experiments with other models, including epoch, learning rate, and dropout probability, were also conducted in the paper. Comparison results prove that the DNN model converged to optimal with fewer epochs and a higher learning rate. In contrast, the CNN model needed more epochs to learn. As for dropout probability, reducing the parameters by ~50% each time was appropriate.

scholarly journals Monaural Singing Voice and Accompaniment Separation Based on Gated Nested U-Net Architecture

Symmetry ◽  
2020 ◽  
Vol 12 (6) ◽  
pp. 1051
Author(s):  
Haibo Geng ◽  
Ying Hu ◽  
Hao Huang
Keyword(s):  
Time Domain ◽  
Singing Voice ◽  
Feature Maps ◽  
Time Frequency ◽  
Convolutional Networks ◽  
Full Resolution ◽  
Mask Layer ◽  
Proposed Model ◽  
Phase Spectra ◽  
Separation Model

This paper proposes a separation model adopting gated nested U-Net (GNU-Net) architecture, which is essentially a deeply supervised symmetric encoder–decoder network that can generate full-resolution feature maps. Through a series of nested skip pathways, it can reduce the semantic gap between the feature maps of encoder and decoder subnetworks. In the GNU-Net architecture, only the backbone not including nested part is applied with gated linear units (GLUs) instead of conventional convolutional networks. The outputs of GNU-Net are further fed into a time-frequency (T-F) mask layer to generate two masks of singing voice and accompaniment. Then, those two estimated masks along with the magnitude and phase spectra of mixture can be transformed into time-domain signals. We explored two types of T-F mask layer, discriminative training network and difference mask layer. The experiment results show the latter to be better. We evaluated our proposed model by comparing with three models, and also with ideal T-F masks. The results demonstrate that our proposed model outperforms compared models, and it’s performance comes near to ideal ratio mask (IRM). More importantly, our proposed model can output separated singing voice and accompaniment simultaneously, while the three compared models can only separate one source with trained model.

scholarly journals Configuration-Invariant Sound Localization Technique Using Azimuth-Frequency Representation and Convolutional Neural Networks

Sensors ◽  
2020 ◽  
Vol 20 (13) ◽  
pp. 3768
Author(s):  
Chanjun Chun ◽  
Kwang Myung Jeon ◽  
Wooyeol Choi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Sound Localization ◽  
Speech Processing ◽  
Image Method ◽  
Frame Size ◽  
Localization Technique ◽  
Time Frequency ◽  
Input Size ◽  
Frequency Representation

Deep neural networks (DNNs) have achieved significant advancements in speech processing, and numerous types of DNN architectures have been proposed in the field of sound localization. When a DNN model is deployed for sound localization, a fixed input size is required. This is generally determined by the number of microphones, the fast Fourier transform size, and the frame size. if the numbers or configurations of the microphones change, the DNN model should be retrained because the size of the input features changes. in this paper, we propose a configuration-invariant sound localization technique using the azimuth-frequency representation and convolutional neural networks (CNNs). the proposed CNN model receives the azimuth-frequency representation instead of time-frequency features as the input features. the proposed model was evaluated in different environments from the microphone configuration in which it was originally trained. for evaluation, single sound source is simulated using the image method. Through the evaluations, it was confirmed that the localization performance was superior to the conventional steered response power phase transform (SRP-PHAT) and multiple signal classification (MUSIC) methods.

scholarly journals TGV Upsampling: A Making-Up Operation for Semantic Segmentation

2019 ◽  
Vol 2019 ◽  
pp. 1-12
Author(s):  
Xu Yin ◽  
Yan Li ◽  
Byeong-Seok Shin
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Detailed Analysis ◽  
Semantic Segmentation ◽  
Information Loss ◽  
Network Architectures ◽  
Feature Maps ◽  
Extraction Processes ◽  
Multiple Uses ◽  
Improved Accuracy

With the widespread use of deep learning methods, semantic segmentation has achieved great improvements in recent years. However, many researchers have pointed out that with multiple uses of convolution and pooling operations, great information loss would occur in the extraction processes. To solve this problem, various operations or network architectures have been suggested to make up for the loss of information. We observed a trend in many studies to design a network as a symmetric type, with both parts representing the “encoding” and “decoding” stages. By “upsampling” operations in the “decoding” stage, feature maps are constructed in a certain way that would more or less make up for the losses in previous layers. In this paper, we focus on upsampling operations, make a detailed analysis, and compare current methods used in several famous neural networks. We also combine the knowledge on image restoration and design a new upsampled layer (or operation) named the TGV upsampling algorithm. We successfully replaced upsampling layers in the previous research with our new method. We found that our model can better preserve detailed textures and edges of feature maps and can, on average, achieve 1.4–2.3% improved accuracy compared to the original models.

scholarly journals Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

2019 ◽  
Author(s):  
Chunlei Liu ◽  
Wenrui Ding ◽  
Xin Xia ◽  
Yuan Hu ◽  
Baochang Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Superior Performance ◽  
Feature Maps ◽  
Unified Framework ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Computation Efficiency ◽  
Significant Performance ◽  
Binary Network

Binarized  convolutional neural networks (BCNNs) are widely used to improve memory and computation efficiency of deep convolutional neural networks (DCNNs) for mobile and AI chips based applications. However, current BCNNs are not able to fully explore their corresponding full-precision models, causing a significant performance gap between them. In this paper, we propose rectified binary convolutional networks (RBCNs), towards optimized BCNNs, by combining full-precision kernels and feature maps to rectify the binarization process in a unified framework. In particular, we use a GAN to train the 1-bit binary network with the guidance of its corresponding full-precision model, which significantly improves the performance of BCNNs. The rectified convolutional layers are generic and flexible, and can be easily incorporated into existing DCNNs such as WideResNets and ResNets. Extensive experiments demonstrate the superior performance of the proposed RBCNs over state-of-the-art BCNNs. In particular, our method shows strong generalization on the object tracking task.

scholarly journals METHODS FOR INCREASING THE CLASSIFICATION ACCURACY BASED ON MODIFICATIONS OF THE BASIC ARCHITECTURE OF CONVOLUTIONAL NEURAL NETWORKS

ScienceRise ◽  
2020 ◽  
pp. 10-16
Author(s):  
Svitlana Shapovalova ◽  
Yurii Moskalenko
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Classification Accuracy ◽  
Malignant Tumors ◽  
Classification Problem ◽  
Video Memory ◽  
Backbone Networks ◽  
Convolutional Networks ◽  
High Recognition Accuracy ◽  
Technological Product

Object of research: basic architectures of deep learning neural networks. Investigated problem: insufficient accuracy of solving the classification problem based on the basic architectures of deep learning neural networks. An increase in accuracy requires a significant complication of the architecture, which, in turn, leads to an increase in the required computing resources, as well as the consumption of video memory and the cost of learning/output time. Therefore, the problem arises of determining such methods for modifying basic architectures that improve the classification accuracy and require insignificant additional computing resources. Main scientific results: based on the analysis of existing methods for improving the classification accuracy on the convolutional networks of basic architectures, it is determined what is most effective: scaling the ScanNet architecture, learning the ensemble of TreeNet models, integrating several CBNet backbone networks. For computational experiments, these modifications of the basic architectures are implemented, as well as their combinations: ScanNet + TreeNet, ScanNet + CBNet. The effectiveness of these methods in comparison with basic architectures has been proven when solving the problem of recognizing malignant tumors with diagnostic images – SIIM-ISIC Melanoma Classification, the train/test set of which is presented on the Kaggle platform. The accuracy value for the area under the ROC curve metric has increased from 0.94489 (basic architecture network) to 0.96317 (network with ScanNet + CBNet modifications). At the same time, the output compared to the basic architecture (EfficientNet-b5) increased from 440 to 490 seconds, and the consumption of video memory increased from 8 to 9.2 gigabytes, which is acceptable. Innovative technological product: methods for achieving high recognition accuracy from a diagnostic signal based on deep learning neural networks of basic architectures. Scope of application of the innovative technological product: automatic diagnostics systems in the following areas: medicine, seismology, astronomy (classification by images) onboard control systems and systems for monitoring transport and vehicle flows or visitors (recognition of scenes with camera frames).

Sign in / Sign up

Export Citation Format

Share Document