scholarly journals InteractionNN: A Neural Network for Learning Hidden Features in Sparse Prediction

2019 ◽  
Author(s):  
Xiaowang Zhang ◽  
Qiang Gao ◽  
Zhiyong Feng
Keyword(s):  
Neural Network ◽  
Nonlinear Interaction ◽  
Feature Interaction ◽  
Feed Forward Neural Network ◽  
Low Level ◽  
Feature Interactions ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
High Level ◽  
Dense Features

In this paper, we present a neural network (InteractionNN) for sparse predictive analysis where hidden features of sparse data can be learned by multilevel feature interaction. To characterize multilevel interaction of features, InteractionNN consists of three modules, namely, nonlinear interaction pooling, layer-lossing, and embedding. Nonlinear interaction pooling (NI pooling) is a hierarchical structure and, by shortcut connection, constructs low-level feature interactions from basic dense features to elementary features. Layer-lossing is a feed-forward neural network where high-level feature interactions can be learned from low-level feature interactions via correlation of all layers with target. Moreover, embedding is to extract basic dense features from sparse features of data which can help in reducing our proposed model computational complex. Finally, our experiment evaluates on the two benchmark datasets and the experimental results show that InteractionNN performs better than most of state-of-the-art models in sparse regression.

scholarly journals APPROACH TO THE SYNTHESIS OF NEURAL NETWORK STRUCTURE DURING CLASSIFICATION

2020 ◽  
pp. 20-26
Author(s):  
Avazjon R. Marakhimov ◽  
Kabul K. Khudaybergenov
Keyword(s):  
Neural Network ◽  
Pattern Recognition ◽  
Artificial Neural Network ◽  
Feed Forward Neural Network ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Neural Network Structure ◽  
Artificial Neural ◽  
Hidden Layer ◽  
Hidden Neurons

Evaluating the number of hidden neurons necessary for solving of pattern recognition and classification tasks is one of the key problems in artificial neural networks. Multilayer perceptron is the most useful artificial neural network to estimate the functional structure in classification. In this paper, we show that artificial neural network with a two hidden layer feed forward neural network with d inputs, d neurons in the first hidden layer, 2d+2 neurons in the second hidden layer, k outputs and with a sigmoidal infinitely differentiable function can solve classification and pattern problems with arbitrary accuracy. This result can be applied to design pattern recognition and classification models with optimal structure in the number of hidden neurons and hidden layers. The experimental results over well-known benchmark datasets show that the convergence and the accuracy of the proposed model of artificial neural network are acceptable. Findings in this paper are experimentally analyzed on four different datasets from machine learning repository.

scholarly journals A Proposed Heuristic Optimization Algorithm for Detecting Network Attacks

2019 ◽  
Vol 2 (4) ◽  
pp. 530
Author(s):  
Amr Hassan Yassin ◽  
Hany Hamdy Hussien
Keyword(s):  
Neural Network ◽  
Heuristic Algorithms ◽  
Feed Forward Neural Network ◽  
Data Set ◽  
Network Attacks ◽  
Security Issues ◽  
Proposed Model ◽  
Network Intrusions ◽  
Attack Patterns ◽  
Selection Of

Due to the exponential growth of E-Business and computing capabilities over the web for a pay-for-use groundwork, the risk factors regarding security issues also increase rapidly. As the usage increases, it becomes very difficult to identify malicious attacks since the attack patterns change. Therefore, host machines in the network must continually be monitored for intrusions since they are the final endpoint of any network. The purpose of this work is to introduce a generalized neural network model that has the ability to detect network intrusions. Two recent heuristic algorithms inspired by the behavior of natural phenomena, namely, the particle swarm optimization (PSO) and gravitational search (GSA) algorithms are introduced. These algorithms are combined together to train a feed forward neural network (FNN) for the purpose of utilizing the effectiveness of these algorithms to reduce the problems of getting stuck in local minima and the time-consuming convergence rate. Dimension reduction focuses on using information obtained from NSL-KDD Cup 99 data set for the selection of some features to discover the type of attacks. Detecting the network attacks and the performance of the proposed model are evaluated under different patterns of network data.

scholarly journals Reconstructing feedback representations in ventral visual pathway with a generative adversarial autoencoder

2020 ◽  
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh
Keyword(s):  
Neural Network ◽  
Visual Information ◽  
Visual Pathway ◽  
Functional Magnetic Resonance ◽  
Low Level ◽  
Ventral Visual Pathway ◽  
High Level ◽  
Feedback Connections ◽  
The Brain ◽  
Insight Into

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.

scholarly journals A High-Density Crowd Counting Method Based on Convolutional Feature Fusion

2018 ◽  
Vol 8 (12) ◽  
pp. 2367 ◽  
Author(s):  
Hongling Luo ◽  
Jun Sang ◽  
Weiqun Wu ◽  
Hong Xiang ◽  
Zhili Xiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Large Scale ◽  
Feature Fusion ◽  
High Density ◽  
Counting Problem ◽  
Crowd Counting ◽  
Low Level ◽  
Density Map ◽  
High Level

In recent years, the trampling events due to overcrowding have occurred frequently, which leads to the demand for crowd counting under a high-density environment. At present, there are few studies on monitoring crowds in a large-scale crowded environment, while there exists technology drawbacks and a lack of mature systems. Aiming to solve the crowd counting problem with high-density under complex environments, a feature fusion-based deep convolutional neural network method FF-CNN (Feature Fusion of Convolutional Neural Network) was proposed in this paper. The proposed FF-CNN mapped the crowd image to its crowd density map, and then obtained the head count by integration. The geometry adaptive kernels were adopted to generate high-quality density maps which were used as ground truths for network training. The deconvolution technique was used to achieve the fusion of high-level and low-level features to get richer features, and two loss functions, i.e., density map loss and absolute count loss, were used for joint optimization. In order to increase the sample diversity, the original images were cropped with a random cropping method for each iteration. The experimental results of FF-CNN on the ShanghaiTech public dataset showed that the fusion of low-level and high-level features can extract richer features to improve the precision of density map estimation, and further improve the accuracy of crowd counting.

scholarly journals Graph Dilated Network with Rejection Mechanism

2020 ◽  
Vol 10 (7) ◽  
pp. 2421
Author(s):  
Bencheng Yan ◽  
Chaokun Wang ◽  
Gaoyang Guo
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Great Success ◽  
Convolution Kernel ◽  
Graph Structure ◽  
Proposed Model ◽  
Rejection Mechanism ◽  
Laplacian Smoothing ◽  
Graph Neural Networks ◽  
High Level

Recently, graph neural networks (GNNs) have achieved great success in dealing with graph-based data. The basic idea of GNNs is iteratively aggregating the information from neighbors, which is a special form of Laplacian smoothing. However, most of GNNs fall into the over-smoothing problem, i.e., when the model goes deeper, the learned representations become indistinguishable. This reflects the inability of the current GNNs to explore the global graph structure. In this paper, we propose a novel graph neural network to address this problem. A rejection mechanism is designed to address the over-smoothing problem, and a dilated graph convolution kernel is presented to capture the high-level graph structure. A number of experimental results demonstrate that the proposed model outperforms the state-of-the-art GNNs, and can effectively overcome the over-smoothing problem.

scholarly journals Active Shock Absorber Control Based on Time-Delay Neural Network

Energies ◽  
2020 ◽  
Vol 13 (5) ◽  
pp. 1091
Author(s):  
Alexander Alyukov ◽  
Yuri Rozhdestvenskiy ◽  
Sergei Aliukov
Keyword(s):  
Neural Network ◽  
Time Delay ◽  
Shock Absorber ◽  
Learning Ability ◽  
Control Input ◽  
Low Level ◽  
External Data ◽  
Good Learning ◽  
Suspension Model ◽  
High Level

A controlled suspension usually consists of a high-level and a low-level controller. The purpose the high-level controller is to analyze external data on vehicle conditions and make decisions on the required value of the force on the shock absorber rod, while the purpose of the low-level controller is to ensure the implementation of the desired force value by controlling the actuators. Many works have focused on the design of high-level controllers of active suspensions, in which it is considered that the shock absorber can instantly and absolutely accurately implement a given control input. However, active shock absorbers are complex systems that have hysteresis. In addition, electro-pneumatic and hydraulic elements are often used in the design, which have a long response time and often low accuracy. The application of methods of control theory in such systems is often difficult due to the complexity of constructing their mathematical models. In this article, the authors propose an effective low-level controller for an active shock absorber based on a time-delay neural network. Neural networks in this case show good learning ability. The low-level controller is implemented in a simplified suspension model and the simulation results are presented for a number of typical cases.

scholarly journals A Novel CapsNet based Image Reconstruction and Regression Analysis

2020 ◽  
Vol 2 (3) ◽  
pp. 156-164 ◽  
Author(s):  
Dr. Akey Sungheetha ◽  
Dr. Rajesh Sharma R
Keyword(s):  
Neural Network ◽  
Regression Analysis ◽  
Network Models ◽  
Superior Performance ◽  
Neural Network Models ◽  
Innovative Method ◽  
Proposed Model ◽  
Retrieval Efficiency ◽  
High Level ◽  
Better Than

In the field of image processing, all types of computation models are almost evolved to solve the issues through encoded neurons. However, compared with decoding orientation and regression analysis, still the doors are open due to its complexity. At present technologies uses two steps such as, decoding the intermediate terms and reconstruction using decoded information. The performance in terms of regression analysis is lagging due to the decoded intermediate terms. Conventional neural network models perform better in feature classification and representation, though the performance is reduced while handling high level features. Considering these issues in image classification and regression, the proposed model is designed with capsule network as an innovative method which is suitable to handle high level features. The experimental results of the proposed model are compared with conventional neural network models such as BPNN and CNN to validate the superior performance. The proposed model achieves better retrieval efficiency of 95.4% which is much better than other neural network models.

scholarly journals Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition

2017 ◽  
Author(s):  
Xinge Zhu ◽  
Liang Li ◽  
Weigang Zhang ◽  
Tianrong Rao ◽  
Min Xu ◽  
...  
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Feature Fusion ◽  
Feature Representation ◽  
Low Level ◽  
Learning Framework ◽  
Independent Entity ◽  
Internet Images ◽  
High Level ◽  
Different Levels

Visual emotion recognition aims to associate images with appropriate emotions. There are different visual stimuli that can affect human emotion from low-level to high-level, such as color, texture, part, object, etc. However, most existing methods treat different levels of features as independent entity without having effective method for feature fusion. In this paper, we propose a unified CNN-RNN model to predict the emotion based on the fused features from different levels by exploiting the dependency among them. Our proposed architecture leverages convolutional neural network (CNN) with multiple layers to extract different levels of features with in a multi-task learning framework, in which two related loss functions are introduced to learn the feature representation. Considering the dependencies within the low-level and high-level features, a new bidirectional recurrent neural network (RNN) is proposed to integrate the learned features from different layers in the CNN model. Extensive experiments on both Internet images and art photo datasets demonstrate that our method outperforms the state-of-the-art methods with at least 7% performance improvement.

scholarly journals Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

2019 ◽  
Vol 33 ◽  
pp. 10067-10068 ◽  
Author(s):  
Jingyun Xu ◽  
Yi Cai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Classification Methods ◽  
Short Text ◽  
Proposed Model ◽  
High Level ◽  
Context Features

Some text classification methods don’t work well on short texts due to the data sparsity. What’s more, they don’t fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the contextrelevant concept features. The experimental results on three datasets show that our proposed model outperforms the stateof-the-art models.

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

Sign in / Sign up

Export Citation Format

Share Document