scholarly journals STOCHASTIC PSEUDOSPIN NEURAL NETWORK WITH TRIDIAGONAL SYNAPTIC CONNECTIONS

2021 ◽  
pp. 114-122
Author(s):  
R. М. Peleshchak ◽  
V. V. Lytvyn ◽  
О. І. Cherniak ◽  
І. R. Peleshchak ◽  
М. V. Doroshenko
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Connection Matrix ◽  
Synaptic Connections ◽  
Hessenberg Matrix ◽  
Tuning Time ◽  
Hebb Rule ◽  
The Matrix ◽  
Dipole Glass ◽  
Fully Connected

Context. To reduce the computational resource time in the problems of diagnosing and recognizing distorted images based on a fully connected stochastic pseudospin neural network, it becomes necessary to thin out synaptic connections between neurons, which is solved using the method of diagonalizing the matrix of synaptic connections without losing interaction between all neurons in the network. Objective. To create an architecture of a stochastic pseudo-spin neural network with diagonal synaptic connections without loosing the interaction between all the neurons in the layer to reduce its learning time. Method. The paper uses the Hausholder method, the method of compressing input images based on the diagonalization of the matrix of synaptic connections and the computer mathematics system MATLAB for converting a fully connected neural network into a tridiagonal form with hidden synaptic connections between all neurons. Results. We developed a model of a stochastic neural network architecture with sparse renormalized synaptic connections that take into account deleted synaptic connections. Based on the transformation of the synaptic connection matrix of a fully connected neural network into a Hessenberg matrix with tridiagonal synaptic connections, we proposed a renormalized local Hebb rule. Using the computer mathematics system “WolframMathematica 11.3”, we calculated, as a function of the number of neurons N, the relative tuning time of synaptic connections (per iteration) in a stochastic pseudospin neural network with a tridiagonal connection Matrix, relative to the tuning time of synaptic connections (per iteration) in a fully connected synaptic neural network. Conclusions. We found that with an increase in the number of neurons, the tuning time of synaptic connections (per iteration) in a stochastic pseudospin neural network with a tridiagonal connection Matrix, relative to the tuning time of synaptic connections (per iteration) in a fully connected synaptic neural network, decreases according to a hyperbolic law. Depending on the direction of pseudospin neurons, we proposed a classification of a renormalized neural network with a ferromagnetic structure, an antiferromagnetic structure, and a dipole glass.

scholarly journals Solving Bisymmetric Solution of a Class of Matrix Equations Based on Linear Saturated System Model Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-6
Author(s):  
Feng Zhang
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
System Model ◽  
Matrix Equations ◽  
Characteristic Parameters ◽  
Neural Network Architecture ◽  
Complicated Process ◽  
The Matrix ◽  
Neural Network Structure ◽  
Low Efficiency

In order to solve the complicated process and low efficiency and low accuracy of solving a class of matrix equations, this paper introduces the linear saturated system model neural network architecture to solve the bisymmetric solution of a class of matrix equations. Firstly, a class of matrix equations is constructed to determine the key problems of solving the equations. Secondly, the linear saturated system model neural network structure is constructed to determine the characteristic parameters in the process of bisymmetric solution. Then, the matrix equations is solved by using backpropagation neural network topology. Finally, the class normalization is realized by using the objective function of bisymmetric solution, and the bisymmetric solution of a class of matrix equations is realized. In order to verify the solving effect of the method in this paper, three indexes (accuracy, correction accuracy, and solving time) are designed in the experiment. The experimental results show that the proposed method can effectively reduce the solving time, can improve the accuracy and correction effect of the bisymmetric solution, and has high practicability.

scholarly journals Progressive Convolutional Neural Network for Incremental Learning

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1879
Author(s):  
Zahid Ali Siddiqui ◽  
Unsang Park
Keyword(s):  
Neural Network ◽  
Incremental Learning ◽  
Network Architecture ◽  
Structural Changes ◽  
Training Time ◽  
Learning Technique ◽  
Incremental Training ◽  
Fully Connected ◽  
Fine Tune ◽  
Entire Network

In this paper, we present a novel incremental learning technique to solve the catastrophic forgetting problem observed in the CNN architectures. We used a progressive deep neural network to incrementally learn new classes while keeping the performance of the network unchanged on old classes. The incremental training requires us to train the network only for new classes and fine-tune the final fully connected layer, without needing to train the entire network again, which significantly reduces the training time. We evaluate the proposed architecture extensively on image classification task using Fashion MNIST, CIFAR-100 and ImageNet-1000 datasets. Experimental results show that the proposed network architecture not only alleviates catastrophic forgetting but can also leverages prior knowledge via lateral connections to previously learned classes and their features. In addition, the proposed scheme is easily scalable and does not require structural changes on the network trained on the old task, which are highly required properties in embedded systems.

scholarly journals Neural Network Identifiability for a Family of Sigmoidal Nonlinearities

2021 ◽  
Author(s):  
Verner Vlačić ◽  
Helmut Bölcskei
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Uniform Norm ◽  
Feed Forward Neural Network ◽  
The Subject ◽  
Hidden Layer ◽  
Arbitrary Precision ◽  
Necessary And Sufficient ◽  
Fully Connected

AbstractThis paper addresses the following question of neural network identifiability: Does the input–output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? The existing literature on the subject (Sussman in Neural Netw 5(4):589–593, 1992; Albertini et al. in Artificial neural networks for speech and vision, 1993; Fefferman in Rev Mat Iberoam 10(3):507–555, 1994) suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided that the networks under consideration satisfy certain “genericity conditions.” The results in Sussman (1992) and Albertini et al. (1993) apply to networks with a single hidden layer and in Fefferman (1994) the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

scholarly journals Neural Networks for Driver Behavior Analysis

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 342
Author(s):  
Fabio Martinelli ◽  
Fiammetta Marulli ◽  
Francesco Mercaldo ◽  
Antonella Santone
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Real World ◽  
Experimental Analysis ◽  
Network Architecture ◽  
Driver Behavior ◽  
Neural Network Architecture ◽  
Real World Data ◽  
Fully Connected

The proliferation of info-entertainment systems in nowadays vehicles has provided a really cheap and easy-to-deploy platform with the ability to gather information about the vehicle under analysis. With the purpose to provide an architecture to increase safety and security in automotive context, in this paper we propose a fully connected neural network architecture considering position-based features aimed to detect in real-time: (i) the driver, (ii) the driving style and (iii) the path. The experimental analysis performed on real-world data shows that the proposed method obtains encouraging results.

scholarly journals Remote Sensing Image Scene Classification Using CNN-CapsNet

2019 ◽  
Vol 11 (5) ◽  
pp. 494 ◽  
Author(s):  
Wei Zhang ◽  
Ping Tang ◽  
Lijun Zhao
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Network Architecture ◽  
Spatial Information ◽  
Feature Learning ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Scene Classification ◽  
Feature Maps ◽  
Fully Connected

Remote sensing image scene classification is one of the most challenging problems in understanding high-resolution remote sensing images. Deep learning techniques, especially the convolutional neural network (CNN), have improved the performance of remote sensing image scene classification due to the powerful perspective of feature learning and reasoning. However, several fully connected layers are always added to the end of CNN models, which is not efficient in capturing the hierarchical structure of the entities in the images and does not fully consider the spatial information that is important to classification. Fortunately, capsule network (CapsNet), which is a novel network architecture that uses a group of neurons as a capsule or vector to replace the neuron in the traditional neural network and can encode the properties and spatial information of features in an image to achieve equivariance, has become an active area in the classification field in the past two years. Motivated by this idea, this paper proposes an effective remote sensing image scene classification architecture named CNN-CapsNet to make full use of the merits of these two models: CNN and CapsNet. First, a CNN without fully connected layers is used as an initial feature maps extractor. In detail, a pretrained deep CNN model that was fully trained on the ImageNet dataset is selected as a feature extractor in this paper. Then, the initial feature maps are fed into a newly designed CapsNet to obtain the final classification result. The proposed architecture is extensively evaluated on three public challenging benchmark remote sensing image datasets: the UC Merced Land-Use dataset with 21 scene categories, AID dataset with 30 scene categories, and the NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that the proposed method can lead to a competitive classification performance compared with the state-of-the-art methods.

scholarly journals AutoGenome V2: New Multimodal Approach Developed for Multi-Omics Research

2020 ◽  
Author(s):  
Chi Xu ◽  
Denghui Liu ◽  
Lei Zhang ◽  
Zhimeng Xu ◽  
Wenjun He ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Network Architecture ◽  
Drug Repositioning ◽  
Molecular System ◽  
Gene Prediction ◽  
Cancer Type ◽  
Omics Data ◽  
Multimodal Approach ◽  
Fully Connected

AbstractDeep learning is very promising in solving problems in omics research, such as genomics, epigenomics, proteomics, and metabolics. The design of neural network architecture is very important in modeling omics data against different scientific problems. Residual fully-connected neural network (RFCN) was proposed to provide better neural network architectures for modeling omics data. The next challenge for omics research is how to integrate informations from different omics data using deep learning, so that information from different molecular system levels could be combined to predict the target. In this paper, we present a novel multimodal approach that could efficiently integrate information from different omics data and achieve better accuracy than previous approaches. We evaluate our method in four different tasks: drug repositioning, target gene prediction, breast cancer subtyping and cancer type prediction, and all the four tasks achieved state of art performances. The multimodal approach is implemented in AutoGenome V2 and is also powered with all the previous AutoML convenience to facilitate biomedical researchers.

scholarly journals Building a mathematical model and an algorithm for training a neural network with sparse dipole synaptic connections for image recognition

2021 ◽  
Vol 6 (4 (114)) ◽  
pp. 21-27
Author(s):  
Vasyl Lytvyn ◽  
Roman Peleshchak ◽  
Ivan Peleshchak ◽  
Oksana Cherniak ◽  
Lyubomyr Demkiv
Keyword(s):  
Neural Network ◽  
Computer Experiment ◽  
Dipole Interaction ◽  
Dipole Approximation ◽  
Synaptic Connections ◽  
Training Algorithm ◽  
Learning Time ◽  
The Neural Network ◽  
Fully Connected ◽  
Weight Coefficients

Large enough structured neural networks are used for solving the tasks to recognize distorted images involving computer systems. One such neural network that can completely restore a distorted image is a fully connected pseudospin (dipole) neural network that possesses associative memory. When submitting some image to its input, it automatically selects and outputs the image that is closest to the input one. This image is stored in the neural network memory within the Hopfield paradigm. Within this paradigm, it is possible to memorize and reproduce arrays of information that have their own internal structure. In order to reduce learning time, the size of the neural network is minimized by simplifying its structure based on one of the approaches: underlying the first is «regularization» while the second is based on the removal of synaptic connections from the neural network. In this work, the simplification of the structure of a fully connected dipole neural network is based on the dipole-dipole interaction between the nearest adjacent neurons of the network. It is proposed to minimize the size of a neural network through dipole-dipole synaptic connections between the nearest neurons, which reduces the time of the computational resource in the recognition of distorted images. The ratio for weight coefficients of synaptic connections between neurons in dipole approximation has been derived. A training algorithm has been built for a dipole neural network with sparse synaptic connections, which is based on the dipole-dipole interaction between the nearest neurons. A computer experiment was conducted that showed that the neural network with sparse dipole connections recognizes distorted images 3 times faster (numbers from 0 to 9, which are shown at 25 pixels), compared to a fully connected neural network

ENCODER-DECODER ATTENTION NETWORK (EDANET) FOR POLYP SEGMENTATION IN COLONOSCOPY IMAGES

2021 ◽  
pp. 8-10
Author(s):  
Madhura Prakash M ◽  
Krishnamurthy G. N
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Surgical Planning ◽  
Vital Role ◽  
Colorectal Polyps ◽  
Colon Polyps ◽  
Feature Maps ◽  
Neural Network Architecture ◽  
Multi Level ◽  
Fully Connected

Colorectal cancer (CRC) is one of the most common malignancies that can develop from high-risk colon polyps. Colonoscopy is a standard for examination and detection of colorectal polyps.[1] Segmentation and distinction of polyps can play a vital role in treatment (e.g., surgical planning) and predictive decision making. This paper proposes a neural network architecture called EDANet, using attention gates to effectively combine multi-level features to yield accurate polyp segmentation. The Encoder is a fully connected Convolution Neural Network (CNN) and the decoder part is a Cascaded Partial Decoder. Encoder and Decoder sub-networks are connected through a series of nested, dense skip pathways. The skip pathways aim at reducing the semantic gap between the feature maps of the Encoder and Decoder sub-networks. The proposed system trains the model on several epochs and it unies the previous epoch mask with the feature map of the current training epoch. The previous epoch mask is then used to provide a hard attention to the learnt feature maps at different convolutional layers. Experimental results demonstrate that the model trained and tested on the Kvasir-SEG dataset achieves a dice coefcient of 0.7874, mean Intersection over Union (mIoU) of 0.7010, recall of 0.7987, and a precision of 0.8577.

Sign in / Sign up

Export Citation Format

Share Document