scholarly journals Continuous Online Sequence Learning with an Unsupervised Neural Network Model

2016 ◽  
Vol 28 (11) ◽  
pp. 2474-2504 ◽  
Author(s):  
Yuwei Cui ◽  
Subutai Ahmad ◽  
Jeff Hawkins
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real World ◽  
Sequence Learning ◽  
Streaming Data ◽  
Variable Order ◽  
Unsupervised Neural Network ◽  
Sequence Memory ◽  
Prediction Problems ◽  
Temporal Sequences

The ability to recognize and predict temporal sequences of sensory inputs is vital for survival in natural environments. Based on many known properties of cortical neurons, hierarchical temporal memory (HTM) sequence memory recently has been proposed as a theoretical framework for sequence learning in the cortex. In this letter, we analyze properties of HTM sequence memory and apply it to sequence learning and prediction problems with streaming data. We show the model is able to continuously learn a large number of variable order temporal sequences using an unsupervised Hebbian-like learning rule. The sparse temporal codes formed by the model can robustly handle branching temporal sequences by maintaining multiple predictions until there is sufficient disambiguating evidence. We compare the HTM sequence memory with other sequence learning algorithms, including statistical methods—autoregressive integrated moving average; feedforward neural networks—time delay neural network and online sequential extreme learning machine; and recurrent neural networks—long short-term memory and echo-state networks on sequence prediction problems with both artificial and real-world data. The HTM model achieves comparable accuracy to other state-of-the-art algorithms. The model also exhibits properties that are critical for sequence learning, including continuous online learning, the ability to handle multiple predictions and branching sequences with high-order statistics, robustness to sensor noise and fault tolerance, and good performance without task-specific hyperparameter tuning. Therefore, the HTM sequence memory not only advances our understanding of how the brain may solve the sequence learning problem but is also applicable to real-world sequence learning problems from continuous data streams.

Using Rough Sets for Practical Feature Selection in a Rough Sets/Neural Network Framework for Knowledge Discovery

2000 ◽  
Vol 4 (1) ◽  
pp. 31-38 ◽  
Author(s):  
Ilona Jagielska ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Feature Selection ◽  
Decision Support ◽  
Knowledge Discovery ◽  
Real World ◽  
Rough Sets ◽  
Network Models ◽  
Feature Subset Selection ◽  
Neural Network Models

An important task in knowledge discovery is feature selection. This paper describes a practical approach to feature subset selection proposed as part of a hybrid rough sets/neural network framework for knowledge discovery for decision support. In this framework neural networks and rough sets are combined and used cooperatively during the system life cycle. The reason for combining rough sets with neural networks in the proposed framework is twofold. Firstly, rough sets based systems provide domain knowledge expressed in the form of If-then rules as well as tools for data analysis. Secondly, rough sets are used in this framework in the task of feature selection for neural network models. This paper examines the feature selection aspect of the framework. An empirical study that tested the approach on artificial datasets and real-world datasets was carried out. Experimental results indicate that the proposed approach can improve the performance of neural network models. The framework was also applied in the development of a real-world decision support system. The experience with this application has shown that the approach can support the users in the task of feature selection.

scholarly journals An Autoencoder-Based Deep Learning Classifier for Efficient Diagnosis of Autism

Children ◽  
2020 ◽  
Vol 7 (10) ◽  
pp. 182
Author(s):  
Harshini Sewani ◽  
Rasha Kashef
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Data Exchange ◽  
Neurodevelopmental Disorder ◽  
Autism Spectrum ◽  
Imaging Data ◽  
Neural Network Learning ◽  
Network Learning ◽  
Unsupervised Neural Network

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by a lack of social communication and social interaction. Autism is a mental disorder investigated by social and computational intelligence scientists utilizing advanced technologies such as machine learning models to enhance clinicians’ ability to provide robust diagnosis and prognosis of autism. However, with dynamic changes in autism behaviour patterns, these models’ quality and accuracy have become a great challenge for clinical practitioners. We applied a deep neural network learning on a large brain image dataset obtained from ABIDE (autism brain imaging data exchange) to provide an efficient diagnosis of ASD, especially for children. Our deep learning model combines unsupervised neural network learning, an autoencoder, and supervised deep learning using convolutional neural networks. Our proposed algorithm outperforms individual-based classifiers measured by various validations and assessment measures. Experimental results indicate that the autoencoder combined with the convolution neural networks provides the best performance by achieving 84.05% accuracy and Area under the Curve (AUC) value of 0.78.

scholarly journals ANALYSIS OF SOM & SOFM TECHNIQUES USED IN SATELLITE IMAGERY

2018 ◽  
Vol 4 (2) ◽  
pp. 563-565
Author(s):  
Rachita Sharma ◽  
Sanjay Kumar Dubey
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Time Series ◽  
Satellite Imagery ◽  
Weather Forecasting ◽  
Learning Algorithm ◽  
Time Series Prediction ◽  
Research Area ◽  
Unsupervised Neural Network

This paper describes the introduction of Supervised and Unsupervised Techniques with the comparison of SOFM (Self Organized Feature Map) used for Satellite Imagery. In this we have explained the way of spatial and temporal changes detection used in forecasting in satellite imagery. Forecasting is based on time series of images using Artificial Neural Network. Recently neural networks have gained a lot of interest in time series prediction due to their ability to learn effectively nonlinear dependencies from large volume of possibly noisy data with a learning algorithm. Unsupervised neural networks reveal useful information from the temporal sequence and they reported power in cluster analysis and dimensionality reduction. In unsupervised learning, no pre classification and pre labeling of the input data is needed. SOFM is one of the unsupervised neural network used for time series prediction .In time series prediction the goal is to construct a model that can predict the future of the measured process under interest. There are various approaches to time series prediction that have been used over the years. It is a research area having application in diverse fields like weather forecasting, speech recognition, remote sensing. Advances in remote sensing technology and availability of high resolution images in recent years have motivated many researchers to study patterns in the images for the purpose of trend analysis

scholarly journals A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

2020 ◽  
Vol 16 (11) ◽  
pp. e1008342
Author(s):  
Zhewei Zhang ◽  
Huzi Cheng ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Sequence Learning ◽  
Empirical Studies ◽  
Activity Patterns ◽  
Network Models ◽  
Neural Network Approach ◽  
Encode Task ◽  
The Brain

The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain.

Models of Artificial Multi-Polynomial Higher Order Neural Networks

2021 ◽  
pp. 97-136
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real World ◽  
Network Models ◽  
Higher Order ◽  
Neural Network Models ◽  
Real World Data ◽  
World Data ◽  
Higher Order Neural Networks ◽  
Improved Accuracy

This chapter introduces multi-polynomial higher order neural network models (MPHONN) with higher accuracy. Using Sun workstation, C++, and Motif, a MPHONN simulator has been built. Real-world data cannot always be modeled simply and simulated with high accuracy by a single polynomial function. Thus, ordinary higher order neural networks could fail to simulate complicated real-world data. But MPHONN model can simulate multi-polynomial functions and can produce results with improved accuracy through experiments. By using MPHONN for financial modeling and simulation, experimental results show that MPHONN can always have 0.5051% to 0.8661% more accuracy than ordinary higher order neural network models.

scholarly journals Concepts, Methods, and Performances of Particle Swarm Optimization, Backpropagation, and Neural Networks

2018 ◽  
Vol 2018 ◽  
pp. 1-7 ◽  
Author(s):  
Leke Zajmi ◽  
Falah Y. H. Ahmed ◽  
Adam Amril Jaharadak
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Particle Swarm Optimization ◽  
Real World ◽  
Particle Swarm ◽  
Swarm Optimization ◽  
Advantages And Disadvantages ◽  
Artificial Neural ◽  
Real World Problems

With the advancement of Machine Learning, since its beginning and over the last years, a special attention has been given to the Artificial Neural Network. As an inspiration from natural selection of animal groups and human’s neural system, the Artificial Neural Network also known as Neural Networks has become the new computational power which is used for solving real world problems. Neural Networks alone as a concept involve various methods for achieving their success; thus, this review paper describes an overview of such methods called Particle Swarm Optimization, Backpropagation, and Neural Network itself, respectively. A brief explanation of the concepts, history, performances, advantages, and disadvantages is given, followed by the latest researches done on these methods. A description of solutions and applications on various industrial sectors such as Medicine or Information Technology has been provided. The last part briefly discusses the directions, current, and future challenges of Neural Networks towards achieving the highest success rate in solving real world problems.

scholarly journals Neural Networks for Driver Behavior Analysis

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 342
Author(s):  
Fabio Martinelli ◽  
Fiammetta Marulli ◽  
Francesco Mercaldo ◽  
Antonella Santone
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Real World ◽  
Experimental Analysis ◽  
Network Architecture ◽  
Driver Behavior ◽  
Neural Network Architecture ◽  
Real World Data ◽  
Fully Connected

The proliferation of info-entertainment systems in nowadays vehicles has provided a really cheap and easy-to-deploy platform with the ability to gather information about the vehicle under analysis. With the purpose to provide an architecture to increase safety and security in automotive context, in this paper we propose a fully connected neural network architecture considering position-based features aimed to detect in real-time: (i) the driver, (ii) the driving style and (iii) the path. The experimental analysis performed on real-world data shows that the proposed method obtains encouraging results.

Unsupervised Learning and Recall of Temporal Sequences: An Application to Robotics

1999 ◽  
Vol 09 (03) ◽  
pp. 235-242 ◽  
Author(s):  
GUILHERME DE A. BARRETO ◽  
ALUIZIO F.R. ARAÚJO
Keyword(s):  
Neural Network ◽  
Fault Tolerance ◽  
Neural Network Model ◽  
Noise Sensitivity ◽  
Correct Order ◽  
Temporal Features ◽  
Proposed Model ◽  
Unsupervised Neural Network ◽  
Complex Sequences ◽  
Temporal Sequences

This paper describes an unsupervised neural network model for learning and recall of temporal patterns. The model comprises two groups of synaptic weights, named competitive feedforward and Hebbian feedback, which are responsible for encoding the static and temporal features of the sequence respectively. Three additional mechanisms allow the network to deal with complex sequences: context units, a neuron commitment equation, and redundancy in the representation of sequence states. The proposed network encodes a set of robot trajectories which may contain states in common, and retrieves them accurately in the correct order. Further tests evaluate the fault-tolerance and noise sensitivity of the proposed model.

scholarly journals Interpretable PID parameter tuning for control engineering using general dynamic neural networks: An extensive comparison

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0243320
Author(s):  
Johannes Günther ◽  
Elias Reichensdörfer ◽  
Patrick M. Pilarski ◽  
Klaus Diepold
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Control Systems ◽  
Real World ◽  
Pid Controllers ◽  
Control Engineering ◽  
Dynamic Neural Networks ◽  
Automation Systems ◽  
Better Than ◽  
General Dynamic

Modern automation systems largely rely on closed loop control, wherein a controller interacts with a controlled process via actions, based on observations. These systems are increasingly complex, yet most deployed controllers are linear Proportional-Integral-Derivative (PID) controllers. PID controllers perform well on linear and near-linear systems but their simplicity is at odds with the robustness required to reliably control complex processes. Modern machine learning techniques offer a way to extend PID controllers beyond their linear control capabilities by using neural networks. However, such an extension comes at the cost of losing stability guarantees and controller interpretability. In this paper, we examine the utility of extending PID controllers with recurrent neural networks—–namely, General Dynamic Neural Networks (GDNN); we show that GDNN (neural) PID controllers perform well on a range of complex control systems and highlight how they can be a scalable and interpretable option for modern control systems. To do so, we provide an extensive study using four benchmark systems that represent the most common control engineering benchmarks. All control environments are evaluated with and without noise as well as with and without disturbances. The neural PID controller performs better than standard PID control in 15 of 16 tasks and better than model-based control in 13 of 16 tasks. As a second contribution, we address the lack of interpretability that prevents neural networks from being used in real-world control processes. We use bounded-input bounded-output stability analysis to evaluate the parameters suggested by the neural network, making them understandable for engineers. This combination of rigorous evaluation paired with better interpretability is an important step towards the acceptance of neural-network-based control approaches for real-world systems. It is furthermore an important step towards interpretable and safely applied artificial intelligence.

Sign in / Sign up

Export Citation Format

Share Document