scholarly journals Architecture Analysis of an FPGA-Based Hopfield Neural Network

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Miguel Angelo de Abreu de Sousa ◽  
Edson Lemos Horta ◽  
Sergio Takeo Kofuji ◽  
Emilio Del-Moral-Hernandez
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
Neural Model ◽  
Hopfield Neural Network ◽  
Network Capacity ◽  
Chip Area ◽  
Characteristics Analysis ◽  
Field Programmable ◽  
Implementation Methodology

Interconnections between electronic circuits and neural computation have been a strongly researched topic in the machine learning field in order to approach several practical requirements, including decreasing training and operation times in high performance applications and reducing cost, size, and energy consumption for autonomous or embedded developments. Field programmable gate array (FPGA) hardware shows some inherent features typically associated with neural networks, such as, parallel processing, modular executions, and dynamic adaptation, and works on different types of FPGA-based neural networks were presented in recent years. This paper aims to address different aspects of architectural characteristics analysis on a Hopfield Neural Network implemented in FPGA, such as maximum operating frequency and chip-area occupancy according to the network capacity. Also, the FPGA implementation methodology, which does not employ multipliers in the architecture developed for the Hopfield neural model, is presented, in detail.

scholarly journals Rethinking arithmetic for deep neural networks

2020 ◽  
Vol 378 (2166) ◽  
pp. 20190051
Author(s):  
G. A. Constantinides
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Directed Graph ◽  
High Performance ◽  
Deep Neural Networks ◽  
Numerical Algorithms ◽  
Future Research ◽  
Data Coding ◽  
Open Questions ◽  
Field Programmable

We consider efficiency in the implementation of deep neural networks. Hardware accelerators are gaining interest as machine learning becomes one of the drivers of high-performance computing. In these accelerators, the directed graph describing a neural network can be implemented as a directed graph describing a Boolean circuit. We make this observation precise, leading naturally to an understanding of practical neural networks as discrete functions, and show that the so-called binarized neural networks are functionally complete. In general, our results suggest that it is valuable to consider Boolean circuits as neural networks , leading to the question of which circuit topologies are promising. We argue that continuity is central to generalization in learning, explore the interaction between data coding, network topology, and node functionality for continuity and pose some open questions for future research. As a first step to bridging the gap between continuous and Boolean views of neural network accelerators, we present some recent results from our work on LUTNet, a novel Field-Programmable Gate Array inference approach. Finally, we conclude with additional possible fruitful avenues for research bridging the continuous and discrete views of neural networks. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

scholarly journals Communication Failure Resilient Distributed Neural Network for Edge Devices

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1614
Author(s):  
Jonghun Jeong ◽  
Jong Sung Park ◽  
Hoeseok Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
State Of The Art ◽  
Wearable Devices ◽  
Communication Failure ◽  
Canadian Institute ◽  
Multiple Devices ◽  
Knowledge Distillation ◽  
Partitioning Technique

Recently, the necessity to run high-performance neural networks (NN) is increasing even in resource-constrained embedded systems such as wearable devices. However, due to the high computational and memory requirements of the NN applications, it is typically infeasible to execute them on a single device. Instead, it has been proposed to run a single NN application cooperatively on top of multiple devices, a so-called distributed neural network. In the distributed neural network, workloads of a single big NN application are distributed over multiple tiny devices. While the computation overhead could effectively be alleviated by this approach, the existing distributed NN techniques, such as MoDNN, still suffer from large traffics between the devices and vulnerability to communication failures. In order to get rid of such big communication overheads, a knowledge distillation based distributed NN, called Network of Neural Networks (NoNN), was proposed, which partitions the filters in the final convolutional layer of the original NN into multiple independent subsets and derives smaller NNs out of each subset. However, NoNN also has limitations in that the partitioning result may be unbalanced and it considerably compromises the correlation between filters in the original NN, which may result in an unacceptable accuracy degradation in case of communication failure. In this paper, in order to overcome these issues, we propose to enhance the partitioning strategy of NoNN in two aspects. First, we enhance the redundancy of the filters that are used to derive multiple smaller NNs by means of averaging to increase the immunity of the distributed NN to communication failure. Second, we propose a novel partitioning technique, modified from Eigenvector-based partitioning, to preserve the correlation between filters as much as possible while keeping the consistent number of filters distributed to each device. Throughout extensive experiments with the CIFAR-100 (Canadian Institute For Advanced Research-100) dataset, it has been observed that the proposed approach maintains high inference accuracy (over 70%, 1.53× improvement over the state-of-the-art approach), on average, even when a half of eight devices in a distributed NN fail to deliver their partial inference results.

FPGA implementation and image encryption application of a new PRNG based on a memristive Hopfield neural network with a special activation gradient

2021 ◽  
Author(s):  
Fei Yu ◽  
Zinan Zhang ◽  
Hui Shen ◽  
Yuanyuan Huang ◽  
Shuo Cai ◽  
...  
Keyword(s):  
Neural Network ◽  
Image Encryption ◽  
High Performance ◽  
Random Sequence ◽  
Random Number Generator ◽  
Security Analysis ◽  
Hopfield Neural Network ◽  
Data Encryption ◽  
Design Tool ◽  
Processing Unit

Abstract In this paper, a memristive Hopfield neural network with a special activation gradient (MHNN) is proposed by adding a suitable memristor to the Hopfield neural network (HNN) with a special activation gradient. The MHNN is simulated and dynamic analyzed, and implemented on FPGA. Then, a new pseudo-random number generator (PRNG) based on MHNN is proposed. The post-processing unit of the PRNG is composed of nonlinear post-processor and XOR calculator, which effectively ensures the randomness of PRNG. The experiments in this paper comply with the IEEE 754-1985 high precision 32-bit floating point standard and are done on the Vivado design tool using a Xilinx XC7Z020CLG400-2 FPGA chip and the Verilog-HDL hardware programming language. The random sequence generated by the PRNG proposed in this paper has passed the NIST SP800-22 test suite and security analysis, proving its randomness and high performance. Finally, an image encryption system based on PRNG is proposed and implemented on FPGA, which proves the value of the image encryption system in the field of data encryption connected to the Internet of Things (IoT).

scholarly journals Nonlinear Dynamics and Chaos in Fractional-Order Hopfield Neural Networks with Delay

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Xia Huang ◽  
Zhen Wang ◽  
Yuxia Li
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Fractional Order ◽  
Hopfield Neural Network ◽  
Hopfield Neural Networks ◽  
Phase Portraits ◽  
Chaotic Attractors ◽  
Chaotic Motions ◽  
Dynamical Behaviors ◽  
Nonlinear Dynamics And Chaos

A fractional-order two-neuron Hopfield neural network with delay is proposed based on the classic well-known Hopfield neural networks, and further, the complex dynamical behaviors of such a network are investigated. A great variety of interesting dynamical phenomena, including single-periodic, multiple-periodic, and chaotic motions, are found to exist. The existence of chaotic attractors is verified by the bifurcation diagram and phase portraits as well.

scholarly journals Semiotic Aggregation in Deep Learning

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1365
Author(s):  
Bogdan Muşat ◽  
Răzvan Andonie
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Decision Model ◽  
Deep Neural Networks ◽  
Neural Model ◽  
Network Layers ◽  
Saliency Maps ◽  
Spatial Entropy ◽  
Insight Into

Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In our experiments, we visualize the superization process and show how the obtained knowledge can be used to explain the neural decision model. In addition, we attempt to optimize the architecture of the neural model employing a semiotic greedy technique. To the extent of our knowledge, this is the first application of computational semiotics in the analysis and interpretation of deep neural networks.

scholarly journals Neural network activation similarity: a new measure to assist decision making in chemical toxicology

2020 ◽  
Vol 11 (28) ◽  
pp. 7335-7348 ◽  
Author(s):  
Timothy E. H. Allen ◽  
Andrew J. Wedlake ◽  
Elena Gelžinytė ◽  
Charles Gong ◽  
Jonathan M. Goodman ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Making ◽  
Deep Learning ◽  
Test Data ◽  
High Performance ◽  
Chemical Binding ◽  
Biological Targets ◽  
Network Activation ◽  
Chemical Toxicology

Deep learning neural networks, constructed for the prediction of chemical binding at 79 pharmacologically important human biological targets, show extremely high performance on test data (accuracy 92.2 ± 4.2%, MCC 0.814 ± 0.093, ROC-AUC 0.96 ± 0.04).

scholarly journals Optimizing Convolution Neural Network on the TI C6678 multicore DSP

2018 ◽  
Vol 246 ◽  
pp. 03044 ◽  
Author(s):  
Guozhao Zeng ◽  
Xiao Hu ◽  
Yueyue Chen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Digital Signal Processor ◽  
High Performance ◽  
Digital Signal ◽  
Automatic Translation ◽  
Convolution Operation ◽  
Multicore Dsp

Convolutional Neural Networks (CNNs) have become the most advanced algorithms for deep learning. They are widely used in image processing, object detection and automatic translation. As the demand for CNNs continues to increase, the platforms on which they are deployed continue to expand. As an excellent low-power, high-performance, embedded solution, Digital Signal Processor (DSP) is used frequently in many key areas. This paper attempts to deploy the CNN to Texas Instruments (TI)’s TMS320C6678 multi-core DSP and optimize the main operations (convolution) to accommodate the DSP structure. The efficiency of the improved convolution operation has increased by tens of times.

Efficiency testing of artificial neural networks in predicting the properties of carbon nanomaterials as potential systems for nervous tissue stimulation and regeneration

2017 ◽  
Vol 13 (1) ◽  
Author(s):  
Martyna Sasiada ◽  
Aneta Fraczek-Szczypta ◽  
Ryszard Tadeusiewicz
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Nervous Tissue ◽  
Carbon Nanomaterials ◽  
Real Life ◽  
Neural Model ◽  
Carbon Nanomaterial ◽  
Neural Models ◽  
Predictive Learning ◽  
The Many

AbstractA new method of predicting the properties of carbon nanomaterials from carbon nanotubes and graphene oxide, using electrophoretic deposition (EPD) on a metal surface, was investigated. The main goal is to obtain the basis for nervous tissue stimulation and regeneration. Because of the many variations of the EPD method, costly and time-consuming experiments are necessary for optimization of the produced systems. To limit such costs and workload, we propose a neural network-based model that can predict the properties of selected carbon nanomaterial systems before they are produced. The choice of neural networks as predictive learning models is based on many studies in the literature that report neural models as good interpretations of real-life processes. The use of a neural network model can reduce experimentation with unpromising methods of systems processing and preparation. Instead, it allows a focus on experiments with these systems, which are promising according to the prediction given by the neural model. The performed tests showed that the proposed method of predictive learning of carbon nanomaterial properties is easy and effective. The experiments showed that the prediction results were consistent with those obtained in the real system.

SYNCHRONIZATION CONTROL OF NEURAL NETWORKS SUBJECT TO TIME-VARYING DELAYS AND INPUT NONLINEARITY

2006 ◽  
Vol 16 (12) ◽  
pp. 3643-3654 ◽  
Author(s):  
JUN-JUH YAN ◽  
TEH-LU LIAO ◽  
JUI-SHENG LIN ◽  
CHAO-JUNG CHENG
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Variable Structure ◽  
Hopfield Neural Network ◽  
Cellular Neural Network ◽  
Control Technique ◽  
Time Varying ◽  
Input Nonlinearity ◽  
Structure Control ◽  
Synchronization Problem

This paper investigates the synchronization problem for a particular class of neural networks subject to time-varying delays and input nonlinearity. Using the variable structure control technique, a memoryless decentralized control law is established which guarantees exponential synchronization even when input nonlinearity is present. The proposed controller is suitable for application in delayed cellular neural networks and Hopfield neural networks with no restriction on the derivative of the time-varying delays. A two-dimensional cellular neural network and a four-dimensional Hopfield neural network, both with time-varying delays, are presented as illustrative examples to demonstrate the effectiveness of the proposed synchronization scheme.

scholarly journals An Optimal Implementation on FPGA of a Hopfield Neural Network

2011 ◽  
Vol 2011 ◽  
pp. 1-9 ◽  
Author(s):  
W. Mansour ◽  
R. Ayoubi ◽  
H. Ziade ◽  
R. Velazco ◽  
W. EL Falou
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
High Performance ◽  
Optimization Problems ◽  
Parallel Architecture ◽  
Hopfield Neural Network ◽  
Noise Removal ◽  
Combinatorial Optimization Problems ◽  
Artificial Neural Network Ann ◽  
Optimal Implementation

The associative Hopfield memory is a form of recurrent Artificial Neural Network (ANN) that can be used in applications such as pattern recognition, noise removal, information retrieval, and combinatorial optimization problems. This paper presents the implementation of the Hopfield Neural Network (HNN) parallel architecture on a SRAM-based FPGA. The main advantage of the proposed implementation is its high performance and cost effectiveness: it requires O(1) multiplications and O(log⁡ N) additions, whereas most others require O(N) multiplications and O(N) additions.

Sign in / Sign up

Export Citation Format

Share Document