scholarly journals Towards better performance with heterogeneous training data in acoustic modeling using deep neural networks

2014 ◽  
Author(s):  
Yan Huang ◽  
Malcolm Slaney ◽  
Michael L. Seltzer ◽  
Yifan Gong
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Training Data ◽  
Acoustic Modeling

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups

2012 ◽  
Vol 29 (6) ◽  
pp. 82-97 ◽  
Author(s):  
Geoffrey Hinton ◽  
Li Deng ◽  
Dong Yu ◽  
George Dahl ◽  
Abdel-rahman Mohamed ◽  
...  
Keyword(s):  
Neural Networks ◽  
Speech Recognition ◽  
Deep Neural Networks ◽  
Acoustic Modeling ◽  
Research Groups

scholarly journals Deep Neural Networks Techniques using for Learning Automata Based Incremental Learning Method

2021 ◽  
Vol 12 (6) ◽  
pp. 69-73
Author(s):  
C. Swetha Reddy Et.al
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Visual Recognition ◽  
Deep Neural Networks ◽  
Learning Automata ◽  
Training Data ◽  
Neuron Models ◽  
Effectiveness Of Training ◽  
Learning Machine ◽  
Incremental Learning Method

Surprisingly comprehensive learning methods are implemented in many large learning machine data, such as visual recognition and visual language processing. Much of the success of advanced training in recent years is due to leadership training, which requires a set of information for specific tasks, before such training. However, in reality, selected tasks related to personal study are gradually accumulated over time as it is difficult to collect and submit training data manually. It provides a way to continue learning some information columns and examples of steps that are specific to the new class and called additional learning. In this post, we recommend the best machine training method for further training for deep neural networks. The basic idea is to learn a deep system with strong connections that can be "activated" or "turned off" at different stages. The approach you suggest allows you to reduce the distribution of old services as you learn new for example new training, which increases the effectiveness of training in the additional training phase. Experiments with MNIST and CIFAR-100 show that our approach can be implemented in other long-term phases in deep neuron models and achieve better results from zero-base training.

scholarly journals Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis

2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Mohammad Amin Nabian ◽  
Hadi Meidani
Keyword(s):  
Neural Networks ◽  
Engineering Design ◽  
Physical System ◽  
Deep Neural Networks ◽  
Complete Information ◽  
Training Data ◽  
Data Driven ◽  
Training Approach ◽  
Domain Expertise ◽  
Generalization Errors

Abstract In this paper, we introduce a physics-driven regularization method for training of deep neural networks (DNNs) for use in engineering design and analysis problems. In particular, we focus on the prediction of a physical system, for which in addition to training data, partial or complete information on a set of governing laws is also available. These laws often appear in the form of differential equations, derived from first principles, empirically validated laws, or domain expertise, and are usually neglected in a data-driven prediction of engineering systems. We propose a training approach that utilizes the known governing laws and regularizes data-driven DNN models by penalizing divergence from those laws. The first two numerical examples are synthetic examples, where we show that in constructing a DNN model that best fits the measurements from a physical system, the use of our proposed regularization results in DNNs that are more interpretable with smaller generalization errors, compared with other common regularization methods. The last two examples concern metamodeling for a random Burgers’ system and for aerodynamic analysis of passenger vehicles, where we demonstrate that the proposed regularization provides superior generalization accuracy compared with other common alternatives.

scholarly journals Development of Vietnamese Speech Synthesis System using Deep Neural Networks

2019 ◽  
Vol 34 (4) ◽  
pp. 349-363 ◽  
Author(s):  
Thinh Van Nguyen ◽  
Bao Quoc Nguyen ◽  
Kinh Huy Phan ◽  
Hai Van Do
Keyword(s):  
Neural Networks ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Speech Synthesis ◽  
Deep Neural Networks ◽  
Hidden Markov ◽  
Experimental Results ◽  
Training Data ◽  
The Internet ◽  
Synthesis System

In this paper, we present our first Vietnamese speech synthesis system based on deep neural networks. To improve the training data collected from the Internet, a cleaning method is proposed. The experimental results indicate that by using deeper architectures we can achieve better performance for the TTS than using shallow architectures such as hidden Markov model. We also present the effect of using different amounts of data to train the TTS systems. In the VLSP TTS challenge 2018, our proposed DNN-based speech synthesis system won the first place in all three subjects including naturalness, intelligibility, and MOS.

scholarly journals Distinguishing between recent balancing selection and incomplete sweep using deep neural networks

2020 ◽  
Author(s):  
Ulas Isildak ◽  
Alessandro Stella ◽  
Matteo Fumagalli
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Balancing Selection ◽  
False Positive Rate ◽  
Genomic Data ◽  
Training Data ◽  
Functional Changes ◽  
Wide Range ◽  
Recent Selection

1AbstractBalancing selection is an important adaptive mechanism underpinning a wide range of phenotypes. Despite its relevance, the detection of recent balancing selection from genomic data is challenging as its signatures are qualitatively similar to those left by ongoing positive selection. In this study we developed and implemented two deep neural networks and tested their performance to predict loci under recent selection, either due to balancing selection or incomplete sweep, from population genomic data. Specifically, we generated forward-intime simulations to train and test an artificial neural network (ANN) and a convolutional neural network (CNN). ANN received as input multiple summary statistics calculated on the locus of interest, while CNN was applied directly on the matrix of haplotypes. We found that both architectures have high accuracy to identify loci under recent selection. CNN generally outperformed ANN to distinguish between signals of balancing selection and incomplete sweep and was less affected by incorrect training data. We deployed both trained networks on neutral genomic regions in European populations and demonstrated a lower false positive rate for CNN than ANN. We finally deployed CNN within the MEFV gene region and identified several common variants predicted to be under incomplete sweep in a European population. Notably, two of these variants are functional changes and could modulate susceptibility to Familial Mediterranean Fever, possibly as a consequence of past adaptation to pathogens. In conclusion, deep neural networks were able to characterise signals of selection on intermediate-frequency variants, an analysis currently inaccessible by commonly used strategies.

scholarly journals Towards Understanding the Risks of Gradient Inversion in Federated Learning

2021 ◽  
Author(s):  
Ali Hatamizadeh ◽  
Hongxu Yin ◽  
Pavlo Molchanov ◽  
Andriy Myronenko ◽  
Wenqi Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Data Privacy ◽  
Deep Neural Networks ◽  
Differential Privacy ◽  
Use Cases ◽  
Training Data ◽  
Model Accuracy ◽  
Healthcare Applications ◽  
Raw Data ◽  
Collaborative Training

Abstract Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. This capability makes it especially interesting for healthcare applications where patient and data privacy is of utmost concern. However, recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack that works for more realistic scenarios where the clients’ training involves updating the Batch Normalization (BN) statistics. Furthermore, we present new ways to measure and visualize potential data leakage in FL. Our work is a step towards establishing reproducible methods of measuring data leakage in FL and could help determine the optimal tradeoffs between privacy-preserving techniques, such as differential privacy, and model accuracy based on quantifiable metrics.

scholarly journals SatImNet: Structured and Harmonised Training Data for Enhanced Satellite Imagery Classification

2020 ◽  
Vol 12 (20) ◽  
pp. 3358
Author(s):  
Vasileios Syrris ◽  
Ondrej Pesek ◽  
Pierre Soille
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Supervised Classification ◽  
Deep Neural Networks ◽  
Satellite Image ◽  
Data Retrieval ◽  
Remote Sensing Image ◽  
Training Data ◽  
Data Sets ◽  
Remote Sensing Image Classification

Automatic supervised classification with complex modelling such as deep neural networks requires the availability of representative training data sets. While there exists a plethora of data sets that can be used for this purpose, they are usually very heterogeneous and not interoperable. In this context, the present work has a twofold objective: (i) to describe procedures of open-source training data management, integration, and data retrieval, and (ii) to demonstrate the practical use of varying source training data for remote sensing image classification. For the former, we propose SatImNet, a collection of open training data, structured and harmonized according to specific rules. For the latter, two modelling approaches based on convolutional neural networks have been designed and configured to deal with satellite image classification and segmentation.

Sign in / Sign up

Export Citation Format

Share Document