Chapter 8. A Constraint-Based Approach to Learning and Reasoning

2021 ◽  
Author(s):  
Michelangelo Diligenti ◽  
Francesco Giannini ◽  
Marco Gori ◽  
Marco Maggini ◽  
Giuseppe Marra
Keyword(s):  
Neural Networks ◽  
Prior Knowledge ◽  
Domain Knowledge ◽  
Graphical Model ◽  
Expressive Power ◽  
Training Data ◽  
Markov Logic ◽  
Exact Inference ◽  
Logic Models ◽  
Effectiveness Of Training

Neural-symbolic models bridge the gap between sub-symbolic and symbolic approaches, both of which have significant limitations. Sub-symbolic approaches, like neural networks, require a large amount of labeled data to be successful, whereas symbolic approaches, like logic reasoners, require a small amount of prior domain knowledge but do not easily scale to large collections of data. This chapter presents a general approach to integrate learning and reasoning that is based on the translation of the available prior knowledge into an undirected graphical model. Potentials on the graphical model are designed to accommodate dependencies among random variables by means of a set of trainable functions, like those computed by neural networks. The resulting neural-symbolic framework can effectively leverage the training data, when available, while exploiting high-level logic reasoning in a certain domain of discourse. Although exact inference is intractable within this model, different tractable models can be derived by making different assumptions. In particular, three models are presented in this chapter: Semantic-Based Regularization, Deep Logic Models and Relational Neural Machines. Semantic-Based Regularization is a scalable neural-symbolic model, that does not adapt the parameters of the reasoner, under the assumption that the provided prior knowledge is correct and must be exactly satisfied. Deep Logic Models preserve the scalability of Semantic-Based Regularization, while providing a flexible exploitation of logic knowledge by co-training the parameters of the reasoner during the learning procedure. Finally, Relational Neural Machines provide the fundamental advantages of perfectly replicating the effectiveness of training from supervised data of standard deep architectures, and of preserving the same generality and expressive power of Markov Logic Networks, when considering pure reasoning on symbolic data. The bonding between learning and reasoning is very general as any (deep) learner can be adopted, and any output structure expressed via First-Order Logic can be integrated. However, exact inference within a Relational Neural Machine is still intractable, and different factorizations are discussed to increase the scalability of the approach.

scholarly journals Deep Neural Networks Techniques using for Learning Automata Based Incremental Learning Method

2021 ◽  
Vol 12 (6) ◽  
pp. 69-73
Author(s):  
C. Swetha Reddy Et.al
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Visual Recognition ◽  
Deep Neural Networks ◽  
Learning Automata ◽  
Training Data ◽  
Neuron Models ◽  
Effectiveness Of Training ◽  
Learning Machine ◽  
Incremental Learning Method

Surprisingly comprehensive learning methods are implemented in many large learning machine data, such as visual recognition and visual language processing. Much of the success of advanced training in recent years is due to leadership training, which requires a set of information for specific tasks, before such training. However, in reality, selected tasks related to personal study are gradually accumulated over time as it is difficult to collect and submit training data manually. It provides a way to continue learning some information columns and examples of steps that are specific to the new class and called additional learning. In this post, we recommend the best machine training method for further training for deep neural networks. The basic idea is to learn a deep system with strong connections that can be "activated" or "turned off" at different stages. The approach you suggest allows you to reduce the distribution of old services as you learn new for example new training, which increases the effectiveness of training in the additional training phase. Experiments with MNIST and CIFAR-100 show that our approach can be implemented in other long-term phases in deep neuron models and achieve better results from zero-base training.

scholarly journals The extrapolation of artificial neural networks for the modelling of rainfall—runoff relationships

2005 ◽  
Vol 7 (4) ◽  
pp. 291-296 ◽  
Author(s):  
P. Hettiarachchi ◽  
M. J. Hall ◽  
A. W. Minns
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Domain Knowledge ◽  
Low Frequency ◽  
Training Data ◽  
Rainfall Runoff ◽  
Ann Model ◽  
Data Set ◽  
Feed Forward ◽  
Artificial Neural

The last decade has seen increasing interest in the application of Artificial Neural Networks (ANNs) for the modelling of the relationship between rainfall and streamflow. Since multi-layer, feed-forward ANNs have the property of being universal approximators, they are able to capture the essence of most input–output relationships, provided that an underlying deterministic relationship exists. Unfortunately, owing to the standardisation of inputs and outputs that is required to run ANNs, a problem arises in extrapolation: if the training data set does not contain the maximum possible output value, an unmodified network will be unable to synthesise this peak value. The occurrence of high magnitude, low frequency events within short periods of record is largely fortuitous. Therefore, the confidence in the neural network model can be greatly enhanced if some methodology can be found for incorporating domain knowledge about such events into the calibration and verification procedure in addition to the available measured data sets. One possible form of additional domain knowledge is the Estimated Maximum Flood (EMF), a notional event with a small but non-negligible probability of exceedence. This study investigates the suitability of including an EMF estimate in the training set of a rainfall–runoff ANN in order to improve the extrapolation characteristics of the network. A study has been carried out in which EMFs have been included, along with recorded flood events, in the training of ANN models for six catchments in the south west of England. The results demonstrate that, with prior transformation of the runoff data to logarithms of flows, the inclusion of domain knowledge in the form of such extreme synthetic events improves the generalisation capabilities of the ANN model and does not disrupt the training process. Where guidelines are available for EMF estimation, the application of this approach is recommended as an alternative means of overcoming the inherent extrapolation problems of multi-layer, feed-forward ANNs.

Seismic stratigraphy interpretation by deep convolutional neural networks: A semisupervised workflow

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA77-WA86 ◽  
Author(s):  
Haibin Di ◽  
Zhun Li ◽  
Hiren Maniar ◽  
Aria Abubakar
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Seismic Data ◽  
Network Architecture ◽  
Domain Knowledge ◽  
Model Building ◽  
Seismic Stratigraphy ◽  
Training Data ◽  
Data Sets ◽  
Deep Convolutional Neural Networks

Depicting geologic sequences from 3D seismic surveying is of significant value to subsurface reservoir exploration, but it is usually time- and labor-intensive for manual interpretation by experienced seismic interpreters. We have developed a semisupervised workflow for efficient seismic stratigraphy interpretation by using the state-of-the-art deep convolutional neural networks (CNNs). Specifically, the workflow consists of two components: (1) seismic feature self-learning (SFSL) and (2) stratigraphy model building (SMB), each of which is formulated as a deep CNN. Whereas the SMB is supervised by knowledge from domain experts and the associated CNN uses a similar network architecture typically used in image segmentation, the SFSL is designed as an unsupervised process and thus can be performed backstage while an expert prepares the training labels for the SMB CNN. Compared with conventional approaches, the our workflow is superior in two aspects. First, the SMB CNN, initialized by the SFSL CNN, successfully inherits the prior knowledge of the seismic features in the target seismic data. Therefore, it becomes feasible for completing the supervised training of the SMB CNN more efficiently using only a small amount of training data, for example, less than 0.1% of the available seismic data as demonstrated in this paper. Second, for the convenience of seismic experts in translating their domain knowledge into training labels, our workflow is designed to be applicable to three scenarios, trace-wise, paintbrushing, and full-sectional annotation. The performance of the new workflow is well-verified through application to three real seismic data sets. We conclude that the new workflow is not only capable of providing robust stratigraphy interpretation for a given seismic volume, but it also holds great potential for other problems in seismic data analysis.

scholarly journals Handling Black Swan Events in Deep Learning with Diversely Extrapolated Neural Networks

2020 ◽  
Author(s):  
Maxime Wabartha ◽  
Audrey Durand ◽  
Vincent François-Lavet ◽  
Joelle Pineau
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Expressive Power ◽  
Training Data ◽  
Imitation Learning ◽  
Learning Problems ◽  
Black Swan ◽  
Regression Problem ◽  
Data Points ◽  
Low Dimensional

By virtue of their expressive power, neural networks (NNs) are well suited to fitting large, complex datasets, yet they are also known to produce similar predictions for points outside the training distribution. As such, they are, like humans, under the influence of the Black Swan theory: models tend to be extremely "surprised" by rare events, leading to potentially disastrous consequences, while justifying these same events in hindsight. To avoid this pitfall, we introduce DENN, an ensemble approach building a set of Diversely Extrapolated Neural Networks that fits the training data and is able to generalize more diversely when extrapolating to novel data points. This leads DENN to output highly uncertain predictions for unexpected inputs. We achieve this by adding a diversity term in the loss function used to train the model, computed at specific inputs. We first illustrate the usefulness of the method on a low-dimensional regression problem. Then, we show how the loss can be adapted to tackle anomaly detection during classification, as well as safe imitation learning problems.

Operational Determination of the Activated Sludge Process Using Neural Networks

1992 ◽  
Vol 26 (9-11) ◽  
pp. 2461-2464 ◽  
Author(s):  
R. D. Tyagi ◽  
Y. G. Du
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Steady State ◽  
Activated Sludge ◽  
Feedforward Neural Network ◽  
Training Data ◽  
Activated Sludge Process

A steady-statemathematical model of an activated sludgeprocess with a secondary settler was developed. With a limited number of training data samples obtained from the simulation at steady state, a feedforward neural network was established which exhibits an excellent capability for the operational prediction and determination.

scholarly journals Improving Semi-Supervised Learning for Audio Classification with FixMatch

Electronics ◽  
2021 ◽  
Vol 10 (15) ◽  
pp. 1807
Author(s):  
Sascha Grollmisch ◽  
Estefanía Cano
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Transfer Learning ◽  
Data Transfer ◽  
State Of The Art ◽  
Training Data ◽  
Audio Classification ◽  
Image Domain ◽  
Full Dataset ◽  
Audio Data

Including unlabeled data in the training process of neural networks using Semi-Supervised Learning (SSL) has shown impressive results in the image domain, where state-of-the-art results were obtained with only a fraction of the labeled data. The commonality between recent SSL methods is that they strongly rely on the augmentation of unannotated data. This is vastly unexplored for audio data. In this work, SSL using the state-of-the-art FixMatch approach is evaluated on three audio classification tasks, including music, industrial sounds, and acoustic scenes. The performance of FixMatch is compared to Convolutional Neural Networks (CNN) trained from scratch, Transfer Learning, and SSL using the Mean Teacher approach. Additionally, a simple yet effective approach for selecting suitable augmentation methods for FixMatch is introduced. FixMatch with the proposed modifications always outperformed Mean Teacher and the CNNs trained from scratch. For the industrial sounds and music datasets, the CNN baseline performance using the full dataset was reached with less than 5% of the initial training data, demonstrating the potential of recent SSL methods for audio data. Transfer Learning outperformed FixMatch only for the most challenging dataset from acoustic scene classification, showing that there is still room for improvement.

scholarly journals Experiments with Neural Networks in the Identification and Control of a Magnetic Levitation System Using a Low-Cost Platform

2021 ◽  
Vol 11 (6) ◽  
pp. 2535
Author(s):  
Bruno E. Silva ◽  
Ramiro S. Barbosa
Keyword(s):  
Neural Networks ◽  
Neural Control ◽  
Magnetic Levitation ◽  
Low Cost ◽  
Training Data ◽  
Neural Controllers ◽  
Levitation System ◽  
Internal Model Controller ◽  
Magnetic Levitation System ◽  
And Performance

In this article, we designed and implemented neural controllers to control a nonlinear and unstable magnetic levitation system composed of an electromagnet and a magnetic disk. The objective was to evaluate the implementation and performance of neural control algorithms in a low-cost hardware. In a first phase, we designed two classical controllers with the objective to provide the training data for the neural controllers. After, we identified several neural models of the levitation system using Nonlinear AutoRegressive eXogenous (NARX)-type neural networks that were used to emulate the forward dynamics of the system. Finally, we designed and implemented three neural control structures: the inverse controller, the internal model controller, and the model reference controller for the control of the levitation system. The neural controllers were tested on a low-cost Arduino control platform through MATLAB/Simulink. The experimental results proved the good performance of the neural controllers.

scholarly journals Autonomous flight cycles and extreme landings of airliners beyond the current limits and capabilities using artificial neural networks

2021 ◽  
Author(s):  
Haitham Baomar ◽  
Peter J. Bentley
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Weather Conditions ◽  
Extreme Weather ◽  
Training Data ◽  
Aviation Industry ◽  
Control Models ◽  
Novel Approach ◽  
Network Modules ◽  
Artificial Neural

AbstractWe describe the Intelligent Autopilot System (IAS), a fully autonomous autopilot capable of piloting large jets such as airliners by learning from experienced human pilots using Artificial Neural Networks. The IAS is capable of autonomously executing the required piloting tasks and handling the different flight phases to fly an aircraft from one airport to another including takeoff, climb, cruise, navigate, descent, approach, and land in simulation. In addition, the IAS is capable of autonomously landing large jets in the presence of extreme weather conditions including severe crosswind, gust, wind shear, and turbulence. The IAS is a potential solution to the limitations and robustness problems of modern autopilots such as the inability to execute complete flights, the inability to handle extreme weather conditions especially during approach and landing where the aircraft’s speed is relatively low, and the uncertainty factor is high, and the pilots shortage problem compared to the increasing aircraft demand. In this paper, we present the work done by collaborating with the aviation industry to provide training data for the IAS to learn from. The training data is used by Artificial Neural Networks to generate control models automatically. The control models imitate the skills of the human pilot when executing all the piloting tasks required to pilot an aircraft between two airports. In addition, we introduce new ANNs trained to control the aircraft’s elevators, elevators’ trim, throttle, flaps, and new ailerons and rudder ANNs to counter the effects of extreme weather conditions and land safely. Experiments show that small datasets containing single demonstrations are sufficient to train the IAS and achieve excellent performance by using clearly separable and traceable neural network modules which eliminate the black-box problem of large Artificial Intelligence methods such as Deep Learning. In addition, experiments show that the IAS can handle landing in extreme weather conditions beyond the capabilities of modern autopilots and even experienced human pilots. The proposed IAS is a novel approach towards achieving full control autonomy of large jets using ANN models that match the skills and abilities of experienced human pilots and beyond.

scholarly journals Localization of Scattering Objects Using Neural Networks

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 11
Author(s):  
Domonkos Haffner ◽  
Ferenc Izsák
Keyword(s):  
Neural Networks ◽  
Plane Waves ◽  
Measurement Data ◽  
Scattered Wave ◽  
Training Data ◽  
Data Set ◽  
Main Compound ◽  
Incident Plane ◽  
The Neural Networks ◽  
Simulation Package

The localization of multiple scattering objects is performed while using scattered waves. An up-to-date approach: neural networks are used to estimate the corresponding locations. In the scattering phenomenon under investigation, we assume known incident plane waves, fully reflecting balls with known diameters and measurement data of the scattered wave on one fixed segment. The training data are constructed while using the simulation package μ-diff in Matlab. The structure of the neural networks, which are widely used for similar purposes, is further developed. A complex locally connected layer is the main compound of the proposed setup. With this and an appropriate preprocessing of the training data set, the number of parameters can be kept at a relatively low level. As a result, using a relatively large training data set, the unknown locations of the objects can be estimated effectively.

scholarly journals Estimation of electrical resistivity using artificial neural networks: a case study from Lublin Basin, SE Poland

2021 ◽  
Author(s):  
Jakub Ważny ◽  
Michał Stefaniuk ◽  
Adam Cygal
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Training Data ◽  
Multilayer Perceptrons ◽  
Borehole Data ◽  
Geological Interpretation ◽  
Se Poland ◽  
Geological Conditions ◽  
Artificial Neural ◽  
Reliable Solution

AbstractArtificial neural networks method (ANNs) is a common estimation tool used for geophysical applications. Considering borehole data, when the need arises to supplement a missing well log interval or whole logging—ANNs provide a reliable solution. Supervised training of the network on a reliable set of borehole data values with further application of this network on unknown wells allows creation of synthetic values of missing geophysical parameters, e.g., resistivity. The main assumptions for boreholes are: representation of similar geological conditions and the use of similar techniques of well data collection. In the analyzed case, a set of Multilayer Perceptrons were trained on five separate chronostratigraphic intervals of borehole, considered as training data. The task was to predict missing deep laterolog (LLD) logging in a borehole representing the same sequence of layers within the Lublin Basin area. Correlation between well logs data exceeded 0.8. Subsequently, magnetotelluric parametric soundings were modeled and inverted on both boreholes. Analysis showed that congenial Occam 1D models had better fitting of TM mode of MT data in each case. Ipso facto, synthetic LLD log could be considered as a basis for geophysical and geological interpretation. ANNs provided solution for supplementing datasets based on this analytical approach.

Sign in / Sign up

Export Citation Format

Share Document