TRAINING DATA SAMPLING FOR CONVENTIONAL NEURAL NETWORKS CONFIGURING

A steady-statemathematical model of an activated sludgeprocess with a secondary settler was developed. With a limited number of training data samples obtained from the simulation at steady state, a feedforward neural network was established which exhibits an excellent capability for the operational prediction and determination.

Download Full-text

Improving Semi-Supervised Learning for Audio Classification with FixMatch

Electronics ◽

10.3390/electronics10151807 ◽

2021 ◽

Vol 10 (15) ◽

pp. 1807

Author(s):

Sascha Grollmisch ◽

Estefanía Cano

Keyword(s):

Neural Networks ◽

Supervised Learning ◽

Transfer Learning ◽

Data Transfer ◽

State Of The Art ◽

Training Data ◽

Audio Classification ◽

Image Domain ◽

Full Dataset ◽

Audio Data

Including unlabeled data in the training process of neural networks using Semi-Supervised Learning (SSL) has shown impressive results in the image domain, where state-of-the-art results were obtained with only a fraction of the labeled data. The commonality between recent SSL methods is that they strongly rely on the augmentation of unannotated data. This is vastly unexplored for audio data. In this work, SSL using the state-of-the-art FixMatch approach is evaluated on three audio classification tasks, including music, industrial sounds, and acoustic scenes. The performance of FixMatch is compared to Convolutional Neural Networks (CNN) trained from scratch, Transfer Learning, and SSL using the Mean Teacher approach. Additionally, a simple yet effective approach for selecting suitable augmentation methods for FixMatch is introduced. FixMatch with the proposed modifications always outperformed Mean Teacher and the CNNs trained from scratch. For the industrial sounds and music datasets, the CNN baseline performance using the full dataset was reached with less than 5% of the initial training data, demonstrating the potential of recent SSL methods for audio data. Transfer Learning outperformed FixMatch only for the most challenging dataset from acoustic scene classification, showing that there is still room for improvement.

Download Full-text

Experiments with Neural Networks in the Identification and Control of a Magnetic Levitation System Using a Low-Cost Platform

Applied Sciences ◽

10.3390/app11062535 ◽

2021 ◽

Vol 11 (6) ◽

pp. 2535

Author(s):

Bruno E. Silva ◽

Ramiro S. Barbosa

Keyword(s):

Neural Networks ◽

Neural Control ◽

Magnetic Levitation ◽

Low Cost ◽

Training Data ◽

Neural Controllers ◽

Levitation System ◽

Internal Model Controller ◽

Magnetic Levitation System ◽

And Performance

In this article, we designed and implemented neural controllers to control a nonlinear and unstable magnetic levitation system composed of an electromagnet and a magnetic disk. The objective was to evaluate the implementation and performance of neural control algorithms in a low-cost hardware. In a first phase, we designed two classical controllers with the objective to provide the training data for the neural controllers. After, we identified several neural models of the levitation system using Nonlinear AutoRegressive eXogenous (NARX)-type neural networks that were used to emulate the forward dynamics of the system. Finally, we designed and implemented three neural control structures: the inverse controller, the internal model controller, and the model reference controller for the control of the levitation system. The neural controllers were tested on a low-cost Arduino control platform through MATLAB/Simulink. The experimental results proved the good performance of the neural controllers.

Download Full-text

Autonomous flight cycles and extreme landings of airliners beyond the current limits and capabilities using artificial neural networks

Applied Intelligence ◽

10.1007/s10489-021-02202-y ◽

2021 ◽

Author(s):

Haitham Baomar ◽

Peter J. Bentley

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Weather Conditions ◽

Extreme Weather ◽

Training Data ◽

Aviation Industry ◽

Control Models ◽

Novel Approach ◽

Network Modules ◽

Artificial Neural

AbstractWe describe the Intelligent Autopilot System (IAS), a fully autonomous autopilot capable of piloting large jets such as airliners by learning from experienced human pilots using Artificial Neural Networks. The IAS is capable of autonomously executing the required piloting tasks and handling the different flight phases to fly an aircraft from one airport to another including takeoff, climb, cruise, navigate, descent, approach, and land in simulation. In addition, the IAS is capable of autonomously landing large jets in the presence of extreme weather conditions including severe crosswind, gust, wind shear, and turbulence. The IAS is a potential solution to the limitations and robustness problems of modern autopilots such as the inability to execute complete flights, the inability to handle extreme weather conditions especially during approach and landing where the aircraft’s speed is relatively low, and the uncertainty factor is high, and the pilots shortage problem compared to the increasing aircraft demand. In this paper, we present the work done by collaborating with the aviation industry to provide training data for the IAS to learn from. The training data is used by Artificial Neural Networks to generate control models automatically. The control models imitate the skills of the human pilot when executing all the piloting tasks required to pilot an aircraft between two airports. In addition, we introduce new ANNs trained to control the aircraft’s elevators, elevators’ trim, throttle, flaps, and new ailerons and rudder ANNs to counter the effects of extreme weather conditions and land safely. Experiments show that small datasets containing single demonstrations are sufficient to train the IAS and achieve excellent performance by using clearly separable and traceable neural network modules which eliminate the black-box problem of large Artificial Intelligence methods such as Deep Learning. In addition, experiments show that the IAS can handle landing in extreme weather conditions beyond the capabilities of modern autopilots and even experienced human pilots. The proposed IAS is a novel approach towards achieving full control autonomy of large jets using ANN models that match the skills and abilities of experienced human pilots and beyond.

Download Full-text

Localization of Scattering Objects Using Neural Networks

Sensors ◽

10.3390/s21010011 ◽

2020 ◽

Vol 21 (1) ◽

pp. 11

Author(s):

Domonkos Haffner ◽

Ferenc Izsák

Keyword(s):

Neural Networks ◽

Plane Waves ◽

Measurement Data ◽

Scattered Wave ◽

Training Data ◽

Data Set ◽

Main Compound ◽

Incident Plane ◽

The Neural Networks ◽

Simulation Package

The localization of multiple scattering objects is performed while using scattered waves. An up-to-date approach: neural networks are used to estimate the corresponding locations. In the scattering phenomenon under investigation, we assume known incident plane waves, fully reflecting balls with known diameters and measurement data of the scattered wave on one fixed segment. The training data are constructed while using the simulation package μ-diff in Matlab. The structure of the neural networks, which are widely used for similar purposes, is further developed. A complex locally connected layer is the main compound of the proposed setup. With this and an appropriate preprocessing of the training data set, the number of parameters can be kept at a relatively low level. As a result, using a relatively large training data set, the unknown locations of the objects can be estimated effectively.

Download Full-text

Estimation of electrical resistivity using artificial neural networks: a case study from Lublin Basin, SE Poland

Acta Geophysica ◽

10.1007/s11600-021-00554-0 ◽

2021 ◽

Author(s):

Jakub Ważny ◽

Michał Stefaniuk ◽

Adam Cygal

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Training Data ◽

Multilayer Perceptrons ◽

Borehole Data ◽

Geological Interpretation ◽

Se Poland ◽

Geological Conditions ◽

Artificial Neural ◽

Reliable Solution

AbstractArtificial neural networks method (ANNs) is a common estimation tool used for geophysical applications. Considering borehole data, when the need arises to supplement a missing well log interval or whole logging—ANNs provide a reliable solution. Supervised training of the network on a reliable set of borehole data values with further application of this network on unknown wells allows creation of synthetic values of missing geophysical parameters, e.g., resistivity. The main assumptions for boreholes are: representation of similar geological conditions and the use of similar techniques of well data collection. In the analyzed case, a set of Multilayer Perceptrons were trained on five separate chronostratigraphic intervals of borehole, considered as training data. The task was to predict missing deep laterolog (LLD) logging in a borehole representing the same sequence of layers within the Lublin Basin area. Correlation between well logs data exceeded 0.8. Subsequently, magnetotelluric parametric soundings were modeled and inverted on both boreholes. Analysis showed that congenial Occam 1D models had better fitting of TM mode of MT data in each case. Ipso facto, synthetic LLD log could be considered as a basis for geophysical and geological interpretation. ANNs provided solution for supplementing datasets based on this analytical approach.

Download Full-text

Laplacian networks: bounding indicator function smoothness for neural networks robustness

APSIPA Transactions on Signal and Information Processing ◽

10.1017/atsip.2021.2 ◽

2021 ◽

Vol 10 ◽

Author(s):

Carlos Lassance ◽

Vincent Gripon ◽

Antonio Ortega

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Supervised Learning ◽

Indicator Function ◽

Training Data ◽

Theoretical Justification ◽

The Past ◽

Noisy Examples

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

Download Full-text

Exploiting heterogeneity in operational neural networks by synaptic plasticity

Neural Computing and Applications ◽

10.1007/s00521-020-05543-w ◽

2021 ◽

Author(s):

Serkan Kiranyaz ◽

Junaid Malik ◽

Habib Ben Abdallah ◽

Turker Ince ◽

Alexandros Iosifidis ◽

...

Keyword(s):

Neural Networks ◽

Synaptic Plasticity ◽

Network Model ◽

Neuron Model ◽

Linear Operators ◽

Training Data ◽

Learning Performance ◽

Minimal Network ◽

Hidden Layer ◽

Hidden Neurons

AbstractThe recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the “Synaptic Plasticity” paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an “elite” ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens.

Download Full-text

Prediction Models for Truck Accidents at Freeway Ramps in Washington State Using Regression and Artificial Intelligence Techniques

Transportation Research Record Journal of the Transportation Research Board ◽

10.3141/1635-04 ◽

1998 ◽

Vol 1635 (1) ◽

pp. 30-36 ◽

Cited By ~ 9

Author(s):

Wael H. Awad ◽

Bruce N. Janson

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Linear Regression ◽

Prediction Models ◽

Washington State ◽

Training Data ◽

Coefficient Of Determination ◽

Training Process ◽

Truck Accidents ◽

High Level

Three different modeling approaches were applied to explain truck accidents at interchanges in Washington State during a 27-month period. Three models were developed for each ramp type including linear regression, neural networks, and a hybrid system using fuzzy logic and neural networks. The study showed that linear regression was able to predict accident frequencies that fell within one standard deviation from the overall mean of the dependent variable. However, the coefficient of determination was very low in all cases. The other two artificial intelligence (AI) approaches showed a high level of performance in identifying different patterns of accidents in the training data and presented a better fit when compared to the regression model. However, the ability of these AI models to predict test data that were not included in the training process showed unsatisfactory results.

Download Full-text

Generalization-Based Acquisition of Training Data for Motor Primitive Learning by Neural Networks

Applied Sciences ◽

10.3390/app11031013 ◽

2021 ◽

Vol 11 (3) ◽

pp. 1013

Author(s):

Zvezdan Lončarević ◽

Rok Pahič ◽

Aleš Ude ◽

Andrej Gams

Keyword(s):

Neural Networks ◽

Dimensionality Reduction ◽

Gaussian Process Regression ◽

Search Space ◽

Robot Learning ◽

Training Data ◽

Practical Applications ◽

Latent Space ◽

Real Robot ◽

Low Dimensional

Autonomous robot learning in unstructured environments often faces the problem that the dimensionality of the search space is too large for practical applications. Dimensionality reduction techniques have been developed to address this problem and describe motor skills in low-dimensional latent spaces. Most of these techniques require the availability of a sufficiently large database of example task executions to compute the latent space. However, the generation of many example task executions on a real robot is tedious, and prone to errors and equipment failures. The main result of this paper is a new approach for efficient database gathering by performing a small number of task executions with a real robot and applying statistical generalization, e.g., Gaussian process regression, to generate more data. We have shown in our experiments that the data generated this way can be used for dimensionality reduction with autoencoder neural networks. The resulting latent spaces can be exploited to implement robot learning more efficiently. The proposed approach has been evaluated on the problem of robotic throwing at a target. Simulation and real-world results with a humanoid robot TALOS are provided. They confirm the effectiveness of generalization-based database acquisition and the efficiency of learning in a low-dimensional latent space.

Download Full-text