Towards Neuromorphic Learning Machines using Emerging Memory Devices with Brain-like Energy Efficiency

The ongoing revolution in Deep Learning is redefining the nature of computing that is driven by the increasing amount of pattern classification and cognitive tasks. Specialized digital hardware for deep learning still holds its predominance due to the flexibility offered by the software implementation and maturity of algorithms. However, it is being increasingly desired that cognitive computing occurs at the edge, i.e. on hand-held devices that are energy constrained, which is a energy prohibitive when employing digital von Neumann architectures. Recent explorations in digital neuromorphic hardware have shown promise, but offer low neurosynaptic density needed for scaling to applications such as intelligent cognitive assistants (ICA). Large-scale integration of CMOS mixed-signal integrated circuits and nanoscale emerging memory devices can enable a new generation of Neuromorphic computers that can alleviate the von Neumann bottleneck for cognitive computing tasks. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures promise machine learning capability at chip-scale form factors, and several orders of magnitude reduction in energy consumption. Practical demonstration of such architectures has been impeded as the performance of these emerging devices falls short of the expected behavior from the idealized analog synapses, or weights, and new learning algorithms are needed to take advantage of the device behavior. In this work, we discuss the challenges involved and present a pathway to realize ultra-lo-power mixed-signal NeuSoC, from device arrays and circuits to spike-based deep learning algorithms, with ‘brain-like’ energy-efficiency.

Download Full-text

Towards Neuromorphic Learning Machines using Emerging Memory Devices with Brain-like Energy Efficiency

10.20944/preprints201807.0362.v2 ◽

2018 ◽

Author(s):

Vishal Saxena ◽

Xinyu Wu ◽

Ira Srivastava ◽

Kehan Zhu

Keyword(s):

Machine Learning ◽

Energy Efficiency ◽

Deep Learning ◽

Learning Algorithms ◽

Memory Devices ◽

Cognitive Computing ◽

Mixed Signal ◽

Von Neumann ◽

Neuromorphic Hardware ◽

Emerging Memory

The ongoing revolution in Deep Learning is redefining the nature of computing that is driven by the increasing amount of pattern classification and cognitive tasks. Specialized digital hardware for deep learning still holds its predominance due to the flexibility offered by the software implementation and maturity of algorithms. However, it is being increasingly desired that cognitive computing occurs at the edge, i.e. on hand-held devices that are energy constrained, which is energy prohibitive when employing digital von Neumann architectures. Recent explorations in digital neuromorphic hardware have shown promise, but offer low neurosynaptic density needed for scaling to applications such as intelligent cognitive assistants (ICA). Large-scale integration of nanoscale emerging memory devices with Complementary Metal Oxide Semiconductor (CMOS) mixed-signal integrated circuits can herald a new generation of Neuromorphic computers that will transcend the von Neumann bottleneck for cognitive computing tasks. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures promise machine learning capability at chip-scale form factor, and several orders of magnitude improvement in energy efficiency. Practical demonstration of such architectures has been limited as performance of emerging memory devices falls short of the expected behavior from the idealized memristor-based analog synapses, or weights, and novel machine learning algorithms are needed to take advantage of the device behavior. In this work, we review the challenges involved and present a pathway to realize ultra-low-power mixed-signal NeuSoC, from device arrays and circuits to spike-based deep learning algorithms, with ‘brain-like’ energy-efficiency.

Download Full-text

Towards Neuromorphic Learning Machines Using Emerging Memory Devices with Brain-Like Energy Efficiency

Journal of Low Power Electronics and Applications ◽

10.3390/jlpea8040034 ◽

2018 ◽

Vol 8 (4) ◽

pp. 34 ◽

Cited By ~ 10

Author(s):

Vishal Saxena ◽

Xinyu Wu ◽

Ira Srivastava ◽

Kehan Zhu

Keyword(s):

Machine Learning ◽

Energy Efficiency ◽

Deep Learning ◽

Large Scale ◽

Learning Algorithms ◽

Memory Devices ◽

Cognitive Computing ◽

Mixed Signal ◽

Von Neumann ◽

Emerging Memory

The ongoing revolution in Deep Learning is redefining the nature of computing that is driven by the increasing amount of pattern classification and cognitive tasks. Specialized digital hardware for deep learning still holds its predominance due to the flexibility offered by the software implementation and maturity of algorithms. However, it is being increasingly desired that cognitive computing occurs at the edge, i.e., on hand-held devices that are energy constrained, which is energy prohibitive when employing digital von Neumann architectures. Recent explorations in digital neuromorphic hardware have shown promise, but offer low neurosynaptic density needed for scaling to applications such as intelligent cognitive assistants (ICA). Large-scale integration of nanoscale emerging memory devices with Complementary Metal Oxide Semiconductor (CMOS) mixed-signal integrated circuits can herald a new generation of Neuromorphic computers that will transcend the von Neumann bottleneck for cognitive computing tasks. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures promise machine learning capability at chip-scale form factor, and several orders of magnitude improvement in energy efficiency. Practical demonstration of such architectures has been limited as performance of emerging memory devices falls short of the expected behavior from the idealized memristor-based analog synapses, or weights, and novel machine learning algorithms are needed to take advantage of the device behavior. In this article, we review the challenges involved and present a pathway to realize large-scale mixed-signal NeuSoCs, from device arrays and circuits to spike-based deep learning algorithms with ‘brain-like’ energy-efficiency.

Download Full-text

Leveraging Deep Learning Techniques for Malaria Parasite Detection Using Mobile Application

Wireless Communications and Mobile Computing ◽

10.1155/2020/8895429 ◽

2020 ◽

Vol 2020 ◽

pp. 1-15

Author(s):

Mehedi Masud ◽

Hesham Alhumyani ◽

Sultan S. Alshamrani ◽

Omar Cheikhrouhou ◽

Saleh Ibrahim ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Mobile Application ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

Machine Learning Techniques ◽

Stochastic Gradient Descent ◽

Cognitive Computing ◽

Human Errors ◽

Learning Techniques

Malaria is a contagious disease that affects millions of lives every year. Traditional diagnosis of malaria in laboratory requires an experienced person and careful inspection to discriminate healthy and infected red blood cells (RBCs). It is also very time-consuming and may produce inaccurate reports due to human errors. Cognitive computing and deep learning algorithms simulate human intelligence to make better human decisions in applications like sentiment analysis, speech recognition, face detection, disease detection, and prediction. Due to the advancement of cognitive computing and machine learning techniques, they are now widely used to detect and predict early disease symptoms in healthcare field. With the early prediction results, healthcare professionals can provide better decisions for patient diagnosis and treatment. Machine learning algorithms also aid the humans to process huge and complex medical datasets and then analyze them into clinical insights. This paper looks for leveraging deep learning algorithms for detecting a deadly disease, malaria, for mobile healthcare solution of patients building an effective mobile system. The objective of this paper is to show how deep learning architecture such as convolutional neural network (CNN) which can be useful in real-time malaria detection effectively and accurately from input images and to reduce manual labor with a mobile application. To this end, we evaluate the performance of a custom CNN model using a cyclical stochastic gradient descent (SGD) optimizer with an automatic learning rate finder and obtain an accuracy of 97.30% in classifying healthy and infected cell images with a high degree of precision and sensitivity. This outcome of the paper will facilitate microscopy diagnosis of malaria to a mobile application so that reliability of the treatment and lack of medical expertise can be solved.

Download Full-text

The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

10.21203/rs.3.rs-701752/v1 ◽

2021 ◽

Author(s):

Alpha Renner ◽

Forrest Sheldon ◽

Anatoly Zlotnik ◽

Louis Tao ◽

Andrew Sornborger

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Low Power ◽

Large Scale ◽

Learning Algorithms ◽

Vlsi Circuits ◽

Machine Learning Algorithms ◽

Backpropagation Algorithm ◽

Neuromorphic Hardware ◽

On Chip

Abstract The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.

Download Full-text

Developing an Artificial Intelligence Project in your Radiology Department

Indian Journal of Musculoskeletal Radiology ◽

10.25259/ijmsr_50_2019 ◽

2020 ◽

Vol 2 ◽

pp. 58-61 ◽

Cited By ~ 1

Author(s):

Syed Junaid ◽

Asad Saeed ◽

Zeili Yang ◽

Thomas Micic ◽

Rajesh Botchu

Keyword(s):

Neural Network ◽

Artificial Intelligence ◽

Health Care ◽

Deep Learning ◽

Learning Algorithms ◽

Radiology Department ◽

Short Article ◽

Computing Power ◽

Road Map ◽

Key Concepts

The advances in deep learning algorithms, exponential computing power, and availability of digital patient data like never before have led to the wave of interest and investment in artificial intelligence in health care. No radiology conference is complete without a substantial dedication to AI. Many radiology departments are keen to get involved but are unsure of where and how to begin. This short article provides a simple road map to aid departments to get involved with the technology, demystify key concepts, and pique an interest in the field. We have broken down the journey into seven steps; problem, team, data, kit, neural network, validation, and governance.

Download Full-text

Faculty Opinions recommendation of Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.732283043.793567347 ◽

2019 ◽

Cited By ~ 1

Author(s):

Victoria Sanz-Moreno

Keyword(s):

Breast Cancer ◽

Deep Learning ◽

Lymph Node ◽

Lymph Node Metastases ◽

Learning Algorithms ◽

Diagnostic Assessment ◽

Node Metastases

Download Full-text

Estimation of Permeability of a Reservoir using Deep Learning Algorithms on Well Logs

SSRN Electronic Journal ◽

10.2139/ssrn.3349570 ◽

2019 ◽

Author(s):

Hera Khan ◽

Ayush Srivastav ◽

Amit Kumar Mishra

Keyword(s):

Deep Learning ◽

Learning Algorithms ◽

Well Logs

Download Full-text

A Study on the Auxiliary Diagnosis of Thyroid Disease Images Based on Multiple Dimensional Deep Learning Algorithms

Current Medical Imaging Formerly Current Medical Imaging Reviews ◽

10.2174/1573405615666190115155223 ◽

2020 ◽

Vol 16 (3) ◽

pp. 199-205

Author(s):

Yuejun Liu ◽

Yifei Xu ◽

Xiangzheng Meng ◽

Xuguang Wang ◽

Tianxu Bai

Keyword(s):

Deep Learning ◽

Learning Algorithms ◽

Region Of Interest ◽

Classification Performance ◽

Thyroid Diseases ◽

Great Success ◽

Learning Models ◽

Good Classification Performance ◽

Spect Images

Background: Medical imaging plays an important role in the diagnosis of thyroid diseases. In the field of machine learning, multiple dimensional deep learning algorithms are widely used in image classification and recognition, and have achieved great success. Objective: The method based on multiple dimensional deep learning is employed for the auxiliary diagnosis of thyroid diseases based on SPECT images. The performances of different deep learning models are evaluated and compared. Methods: Thyroid SPECT images are collected with three types, they are hyperthyroidism, normal and hypothyroidism. In the pre-processing, the region of interest of thyroid is segmented and the amount of data sample is expanded. Four CNN models, including CNN, Inception, VGG16 and RNN, are used to evaluate deep learning methods. Results: Deep learning based methods have good classification performance, the accuracy is 92.9%-96.2%, AUC is 97.8%-99.6%. VGG16 model has the best performance, the accuracy is 96.2% and AUC is 99.6%. Especially, the VGG16 model with a changing learning rate works best. Conclusion: The standard CNN, Inception, VGG16, and RNN four deep learning models are efficient for the classification of thyroid diseases with SPECT images. The accuracy of the assisted diagnostic method based on deep learning is higher than that of other methods reported in the literature.

Download Full-text