scholarly journals An Analysis on the Architecture and the Size of Quantized Hardware Neural Networks Based on Memristors

Electronics ◽  
2021 ◽  
Vol 10 (24) ◽  
pp. 3141
Author(s):  
Rocio Romero-Zaliz ◽  
Antonio Cantudo ◽  
Eduardo Perez ◽  
Francisco Jimenez-Molinos ◽  
Christian Wenger ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Resistive Switching ◽  
Hardware Implementation ◽  
Synaptic Weight ◽  
Simulation Experiments ◽  
Trade Off ◽  
Specific Solution ◽  
Hidden Layer

We have performed different simulation experiments in relation to hardware neural networks (NN) to analyze the role of the number of synapses for different NN architectures in the network accuracy, considering different datasets. A technology that stands upon 4-kbit 1T1R ReRAM arrays, where resistive switching devices based on HfO2 dielectrics are employed, is taken as a reference. In our study, fully dense (FdNN) and convolutional neural networks (CNN) were considered, where the NN size in terms of the number of synapses and of hidden layer neurons were varied. CNNs work better when the number of synapses to be used is limited. If quantized synaptic weights are included, we observed that NN accuracy decreases significantly as the number of synapses is reduced; in this respect, a trade-off between the number of synapses and the NN accuracy has to be achieved. Consequently, the CNN architecture must be carefully designed; in particular, it was noticed that different datasets need specific architectures according to their complexity to achieve good results. It was shown that due to the number of variables that can be changed in the optimization of a NN hardware implementation, a specific solution has to be worked in each case in terms of synaptic weight levels, NN architecture, etc.

scholarly journals Hardware implementation of radial-basis neural networks with Gaussian activation functions on FPGA

2021 ◽  
Author(s):  
Volodymyr Shymkovych ◽  
Sergii Telenyk ◽  
Petro Kravets
Keyword(s):  
Neural Networks ◽  
Hardware Implementation ◽  
Gaussian Function ◽  
Activation Function ◽  
Rbf Neural Networks ◽  
Activation Functions ◽  
Rbf Network ◽  
Combination Scheme ◽  
Radial Basis ◽  
Hidden Layer

AbstractThis article introduces a method for realizing the Gaussian activation function of radial-basis (RBF) neural networks with their hardware implementation on field-programmable gaits area (FPGAs). The results of modeling of the Gaussian function on FPGA chips of different families have been presented. RBF neural networks of various topologies have been synthesized and investigated. The hardware component implemented by this algorithm is an RBF neural network with four neurons of the latent layer and one neuron with a sigmoid activation function on an FPGA using 16-bit numbers with a fixed point, which took 1193 logic matrix gate (LUTs—LookUpTable). Each hidden layer neuron of the RBF network is designed on an FPGA as a separate computing unit. The speed as a total delay of the combination scheme of the block RBF network was 101.579 ns. The implementation of the Gaussian activation functions of the hidden layer of the RBF network occupies 106 LUTs, and the speed of the Gaussian activation functions is 29.33 ns. The absolute error is ± 0.005. The Spartan 3 family of chips for modeling has been used to get these results. Modeling on chips of other series has been also introduced in the article. RBF neural networks of various topologies have been synthesized and investigated. Hardware implementation of RBF neural networks with such speed allows them to be used in real-time control systems for high-speed objects.

scholarly journals Automated cephalometric landmark detection with confidence regions using Bayesian convolutional neural networks

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Jeong-Hoon Lee ◽  
Hee-Jin Yu ◽  
Min-ji Kim ◽  
Jin-Woo Kim ◽  
Jongeun Choi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Region Of Interest ◽  
Confidence Regions ◽  
Grand Challenge ◽  
Uncertainty Prediction ◽  
Integral Role ◽  
Roi Extraction ◽  
Aided Diagnosis

Abstract Background Despite the integral role of cephalometric analysis in orthodontics, there have been limitations regarding the reliability, accuracy, etc. of cephalometric landmarks tracing. Attempts on developing automatic plotting systems have continuously been made but they are insufficient for clinical applications due to low reliability of specific landmarks. In this study, we aimed to develop a novel framework for locating cephalometric landmarks with confidence regions using Bayesian Convolutional Neural Networks (BCNN). Methods We have trained our model with the dataset from the ISBI 2015 grand challenge in dental X-ray image analysis. The overall algorithm consisted of a region of interest (ROI) extraction of landmarks and landmarks estimation considering uncertainty. Prediction data produced from the Bayesian model has been dealt with post-processing methods with respect to pixel probabilities and uncertainties. Results Our framework showed a mean landmark error (LE) of 1.53 ± 1.74 mm and achieved a successful detection rate (SDR) of 82.11, 92.28 and 95.95%, respectively, in the 2, 3, and 4 mm range. Especially, the most erroneous point in preceding studies, Gonion, reduced nearly halves of its error compared to the others. Additionally, our results demonstrated significantly higher performance in identifying anatomical abnormalities. By providing confidence regions (95%) that consider uncertainty, our framework can provide clinical convenience and contribute to making better decisions. Conclusion Our framework provides cephalometric landmarks and their confidence regions, which could be used as a computer-aided diagnosis tool and education.

scholarly journals Regularizing Deep Neural Networks with an Ensemble-based Decorrelation Method

2018 ◽  
Author(s):  
Shuqin Gu ◽  
Yuexian Hou ◽  
Lipeng Zhang ◽  
Yazhou Zhang
Keyword(s):  
Neural Networks ◽  
Ensemble Learning ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Experimental Results ◽  
Excellent Performance ◽  
Hidden Layer ◽  
Base Learner ◽  
Fully Connected

Although Deep Neural Networks (DNNs) have achieved excellent performance in many tasks, improving the generalization capacity of DNNs still remains a challenge. In this work, we propose a novel regularizer named Ensemble-based Decorrelation Method (EDM), which is motivated by the idea of the ensemble learning to improve generalization capacity of DNNs. EDM can be applied to hidden layers in fully connected neural networks or convolutional neural networks. We treat each hidden layer as an ensemble of several base learners through dividing all the hidden units into several non-overlap groups, and each group will be viewed as a base learner. EDM encourages DNNs to learn more diverse representations by minimizing the covariance between all base learners during the training step. Experimental results on MNIST and CIFAR datasets demonstrate that EDM can effectively reduce the overfitting and improve the generalization capacity of DNNs  

scholarly journals Object Detectors’ Convolutional Neural Networks backbones : a review and a comparative study

2021 ◽  
Vol 9 (11) ◽  
pp. 1379-1386
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Object Detection ◽  
Convolutional Neural Networks ◽  
Crucial Role ◽  
Extended Version ◽  
Backbone Networks ◽  
Detection Algorithms ◽  
Wide Range

Computer vision is a scientific field that deals with how computers can acquire significant level comprehension from computerized images or videos. One of the keystones of computer vision is object detection that aims to identify relevant features from video or image to detect objects. Backbone is the first stage in object detection algorithms that play a crucial role in object detection. Object detectors are usually provided with backbone networks designed for image classification. Object detection performance is highly based on features extracted by backbones, for instance, by simply replacing a backbone with its extended version, a large accuracy metric grows up. Additionally, the backbone's importance is demonstrated by its efficiency in real-time object detection. In this paper, we aim to accumulate the crucial role of the deep learning era and convolutional neural networks in particular in object detection tasks. We have analyzed and have been concentrating on a wide range of reviews on convolutional neural networks used as the backbone of object detection models. Building, therefore, a review of backbones that help researchers and scientists to use it as a guideline for their works.

scholarly journals Evaluation of the Classification Accuracy of the Kidney Biopsy Direct Immunofluorescence through Convolutional Neural Networks

2020 ◽  
Vol 15 (10) ◽  
pp. 1445-1454 ◽  
Author(s):  
Giulia Ligabue ◽  
Federico Pollastri ◽  
Francesco Fontana ◽  
Marco Leonelli ◽  
Luciana Furci ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Kidney Biopsy ◽  
Ground Truth ◽  
Light Chains ◽  
Direct Immunofluorescence ◽  
Support Tool ◽  
Glomerular Deposits ◽  
Web Platform

Background and objectivesImmunohistopathology is an essential technique in the diagnostic workflow of a kidney biopsy. Deep learning is an effective tool in the elaboration of medical imaging. We wanted to evaluate the role of a convolutional neural network as a support tool for kidney immunofluorescence reporting.Design, setting, participants, & measurementsHigh-magnification (×400) immunofluorescence images of kidney biopsies performed from the year 2001 to 2018 were collected. The report, adopted at the Division of Nephrology of the AOU Policlinico di Modena, describes the specimen in terms of “appearance,” “distribution,” “location,” and “intensity” of the glomerular deposits identified with fluorescent antibodies against IgG, IgA, IgM, C1q and C3 complement fractions, fibrinogen, and κ- and λ-light chains. The report was used as ground truth for the training of the convolutional neural networks.ResultsIn total, 12,259 immunofluorescence images of 2542 subjects undergoing kidney biopsy were collected. The test set analysis showed accuracy values between 0.79 (“irregular capillary wall” feature) and 0.94 (“fine granular” feature). The agreement test of the results obtained by the convolutional neural networks with respect to the ground truth showed similar values to three pathologists of our center. Convolutional neural networks were 117 times faster than human evaluators in analyzing 180 test images. A web platform, where it is possible to upload digitized images of immunofluorescence specimens, is available to evaluate the potential of our approach.ConclusionsThe data showed that the accuracy of convolutional neural networks is comparable with that of pathologists experienced in the field.

Neuromorphic Memristive Chips: Design and Technology

2021 ◽  
Vol 23 (6) ◽  
pp. 285-294
Author(s):  
N.V. Andreeva ◽  
◽  
V.V. Luchinin ◽  
E.A. Ryndin ◽  
M.G. Anchkov ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Resistive Switching ◽  
Hardware Implementation ◽  
Functional Materials ◽  
Design And Technology ◽  
Spiking Neural Network ◽  
Chip Design ◽  
Neuromorphic Hardware ◽  
Metal Insulator Metal

Memristive neuromorphic chips exploit a prospective class of novel functional materials (memristors) to deploy a new architecture of spiking neural networks for developing basic blocks of brain-like systems. Memristor-based neuromorphic hardware solutions for multi-agent systems are considered as challenges in frontier areas of chip design for fast and energy-efficient computing. As functional materials, metal oxide thin films with resistive switching and memory effects (memristive structures) are recognized as a potential elemental base for new components of neuromorphic engineering, enabling a combination of both data storage and processing in a single unit. A key design issue in this case is a hardware defined functionality of neural networks. The gradient change of resistive properties of memristive elements and its non-volatile memory behavior ensure the possibility of spiking neural network organization with unsupervised learning through hardware implementation of basic synaptic mechanisms, such as Hebb's learning rules including spike — timing dependent plasticity, long-term potentiation and depression. This paper provides an overview of scientific researches carrying out at Saint Petersburg Electrotechnical University "LETI" since 2014 in the field of novel electronic components for neuromorphic hardware solutions of brain-like chip design. Among the most promising concepts developed by ETU "LETI" are: the design of metal-insulator-metal structures exhibiting multilevel resistive switching (gradient tuning of resistive properties and bipolar resistive switching are combined together in a sin¬gle memristive element) for further use as artificial synaptic devices in neuromorphic chips; computing schemes for spatio-temporal pattern recognition based on spiking neural network architecture implementation; breadboard models of analogue circuits of hardware implementation of neuromorphic blocks for brain-like system developing.

Rich feature hierarchies for accurate object detection and semantic segmentation

2021 ◽  
pp. 115-126
Author(s):  
A.Y. Virasova ◽  
D.I. Klimov ◽  
O.E. Khromov ◽  
I.R. Gubaidullin ◽  
V.V. Oreshko
Keyword(s):  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Networks ◽  
Hardware Implementation ◽  
Training Data ◽  
Fine Tuning ◽  
Preliminary Training ◽  
Detection Techniques ◽  
Foreign Production ◽  
Element Base

Formulation of the problem. Over the past few years, there has been little progress in object detection techniques. The most efficient are complex computational ensemble methods, which usually combine several low-level image properties with high-level properties. However, every day interest in artificial intelligence is growing, and the idea of using neural networks on board a spacecraft, with the possibility of making decisions and issuing one-time commands, is very promising, since it makes it possible to analyze a large data stream in real time without resorting to ground station, thereby not losing information when transmitting a packet. The purpose of the work is to conduct research on the possibility of effective use of modern models of neural networks, to develop a methodology for their use in the problem of object detection and analysis of the element base for hardware implementation with the possibility of using convolutional neural networks for thermovideotelemetry on board a spacecraft. Results of work. An approach has been formulated that combines two key ideas: 1) application of high-throughput convolutional neural networks for downward processing of image regions in order to localize and segment objects; 2) preliminary training for the auxiliary task, followed by fine tuning of the domain, which gives a significant increase in performance in the case when the training data is insufficient. The analysis of the element base for the possibility of hardware implementation of neural networks on board a spacecraft using electrical radio products of domestic and foreign production is carried out. Practical significance. The efficiency of preliminary network training for an auxiliary task is shown, followed by fine tuning of the subject area. A technique is described that makes it possible to increase the average accuracy of detecting objects in an image by more than 30%. The analysis of the existing element base, the possibility of hardware implementation of neural networks on board the spacecraft using electrical radio products of domestic and foreign production, as well as the criteria for selecting key elements.

Sign in / Sign up

Export Citation Format

Share Document