Runtime Long-Term Reliability Management Using Stochastic Computing in Deep Neural Networks

2021 ◽  
Author(s):  
Yibo Liu ◽  
Shuyuan Yu ◽  
Shaoyi Peng ◽  
Sheldon X.-D. Tan
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Reliability Management ◽  
Stochastic Computing

scholarly journals COSMO: Computing with Stochastic Numbers in Memory

2022 ◽  
Vol 18 (2) ◽  
pp. 1-25
Author(s):  
Saransh Gupta ◽  
Mohsen Imani ◽  
Joonseop Sim ◽  
Andrew Huang ◽  
Fan Wu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Energy Efficient ◽  
Deep Neural Networks ◽  
Parallel Architecture ◽  
Low Energy ◽  
Stochastic Computing ◽  
Wide Range ◽  
Low Energy Consumption ◽  
Sc Addition

Stochastic computing (SC) reduces the complexity of computation by representing numbers with long streams of independent bits. However, increasing performance in SC comes with either an increase in area or a loss in accuracy. Processing in memory (PIM) computes data in-place while having high memory density and supporting bit-parallel operations with low energy consumption. In this article, we propose COSMO, an architecture for co mputing with s tochastic numbers in me mo ry, which enables SC in memory. The proposed architecture is general and can be used for a wide range of applications. It is a highly dense and parallel architecture that supports most SC encodings and operations in memory. It maximizes the performance and energy efficiency of SC by introducing several innovations: (i) in-memory parallel stochastic number generation, (ii) efficient implication-based logic in memory, (iii) novel memory bit line segmenting, (iv) a new memory-compatible SC addition operation, and (v) enabling flexible block allocation. To show the generality and efficiency of our stochastic architecture, we implement image processing, deep neural networks (DNNs), and hyperdimensional (HD) computing on the proposed hardware. Our evaluations show that running DNN inference on COSMO is 141× faster and 80× more energy efficient as compared to GPU.

scholarly journals Forms of explanation and understanding for neuroscience and artificial intelligence

2021 ◽  
Author(s):  
Jessica A. F. Thompson
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Scientific Explanation ◽  
Scientific Progress ◽  
Neural Systems ◽  
Relational Reasoning ◽  
Integrated Science ◽  
Philosophical Questions

Much of the controversy evoked by the use of deep neural networks as models of biological neural systems amount to debates over what constitutes scientific progress in neuroscience. In order to discuss what constitutes scientific progress, one must have a goal in mind (progress towards what?). One such long term goal is to produce scientific explanations of intelligent capacities (e.g., object recognition, relational reasoning). I argue that the most pressing philosophical questions at the intersection of neuroscience and artificial intelligence are ultimately concerned with defining the phenomena to be explained and with what constitute valid explanations of such phenomena. I propose that a foundation in the philosophy of scientific explanation and understanding can scaffold future discussions about how an integrated science of intelligence might progress. Towards this vision, I review relevant theories of scientific explanation and discuss strategies for unifying the scientific goals of neuroscience and AI.

scholarly journals Forms of explanation and understanding for neuroscience and artificial intelligence

2021 ◽  
Author(s):  
Jessica Anne Farrell Thompson
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Scientific Explanation ◽  
Scientific Progress ◽  
Neural Systems ◽  
Relational Reasoning ◽  
Integrated Science ◽  
Philosophical Questions

Much of the controversy evoked by the use of deep neural networks (DNNs) as models of biological neural systems amount to debates over what constitutes scientific progress in neuroscience. In order to discuss what constitutes scientific progress, one must have a goal in mind (progress towards what?). One such long term goal is to produce scientific explanations of intelligent capacities (e.g. object recognition, relational reasoning). I argue that the most pressing philosophical questions at the intersection of neuroscience and artificial intelligence are ultimately concerned with defining the phenomena to be explained and with what constitute valid explanations of such phenomena. As such, I propose that a foundation in the philosophy of scientific explanation and understanding can scaffold future discussions about how an integrated science of intelligence might progress. Towards this vision, I review several of the most relevant theories of scientific explanation and begin to outline candidate forms of explanation for neural and cognitive phenomena.

scholarly journals Artificial intelligence in cardiology

2020 ◽  
pp. 8-11
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Automated Image Analysis ◽  
Specific Domain ◽  
Feature Discovery ◽  
Medical Disciplines ◽  
Second Wave

In the first wave of artificial intelligence (AI), rule-based expert systems were developed, with modest success, to help generalists who lacked expertise in a specific domain. The second wave of AI, originally called artificial neural networks but now described as machine learning, began to have an impact with multilayer networks in the 1980s. Deep learning, which enables automated feature discovery, has enjoyed spectacular success in several medical disciplines, including cardiology, from automated image analysis to the identification of the electrocardiographic signature of atrial fibrillation during sinus rhythm. Machine learning is now embedded within the NHS Long-Term Plan in England, but its widespread adoption may be limited by the “black-box” nature of deep neural networks.

scholarly journals Universal Approximation Property and Equivalence of Stochastic Computing-Based Neural Networks and Binary Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 5369-5376 ◽  
Author(s):  
Yanzhi Wang ◽  
Zheng Zhan ◽  
Liang Zhao ◽  
Jian Tang ◽  
Siyue Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Approximation Property ◽  
Network Size ◽  
Universal Approximation ◽  
Approximate Computing ◽  
Stochastic Computing ◽  
Hardware Implementations ◽  
Pros And Cons

Large-scale deep neural networks are both memory and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated. Specific forms of binary neural networks (BNNs) and stochastic computing-based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper we prove that the “ideal” SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior), which is a new angle from the original approximation property. The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a “bridge” to prove for BNNs. Besides the universal approximation property, we also derive an appropriate bound for bit length M in order to provide insights for the actual neural network implementations. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growth of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable.

Sign in / Sign up

Export Citation Format

Share Document