VLSI implementation of deep neural networks using integral stochastic computing

2016 ◽  
Author(s):  
Arash Ardakani ◽  
Francois Leduc-Primeau ◽  
Naoya Onizawa ◽  
Takahiro Hanyu ◽  
Warren J. Gross
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Vlsi Implementation ◽  
Stochastic Computing

scholarly journals COSMO: Computing with Stochastic Numbers in Memory

2022 ◽  
Vol 18 (2) ◽  
pp. 1-25
Author(s):  
Saransh Gupta ◽  
Mohsen Imani ◽  
Joonseop Sim ◽  
Andrew Huang ◽  
Fan Wu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Energy Efficient ◽  
Deep Neural Networks ◽  
Parallel Architecture ◽  
Low Energy ◽  
Stochastic Computing ◽  
Wide Range ◽  
Low Energy Consumption ◽  
Sc Addition

Stochastic computing (SC) reduces the complexity of computation by representing numbers with long streams of independent bits. However, increasing performance in SC comes with either an increase in area or a loss in accuracy. Processing in memory (PIM) computes data in-place while having high memory density and supporting bit-parallel operations with low energy consumption. In this article, we propose COSMO, an architecture for co mputing with s tochastic numbers in me mo ry, which enables SC in memory. The proposed architecture is general and can be used for a wide range of applications. It is a highly dense and parallel architecture that supports most SC encodings and operations in memory. It maximizes the performance and energy efficiency of SC by introducing several innovations: (i) in-memory parallel stochastic number generation, (ii) efficient implication-based logic in memory, (iii) novel memory bit line segmenting, (iv) a new memory-compatible SC addition operation, and (v) enabling flexible block allocation. To show the generality and efficiency of our stochastic architecture, we implement image processing, deep neural networks (DNNs), and hyperdimensional (HD) computing on the proposed hardware. Our evaluations show that running DNN inference on COSMO is 141× faster and 80× more energy efficient as compared to GPU.

scholarly journals Universal Approximation Property and Equivalence of Stochastic Computing-Based Neural Networks and Binary Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 5369-5376 ◽  
Author(s):  
Yanzhi Wang ◽  
Zheng Zhan ◽  
Liang Zhao ◽  
Jian Tang ◽  
Siyue Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Approximation Property ◽  
Network Size ◽  
Universal Approximation ◽  
Approximate Computing ◽  
Stochastic Computing ◽  
Hardware Implementations ◽  
Pros And Cons

Large-scale deep neural networks are both memory and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated. Specific forms of binary neural networks (BNNs) and stochastic computing-based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper we prove that the “ideal” SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior), which is a new angle from the original approximation property. The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a “bridge” to prove for BNNs. Besides the universal approximation property, we also derive an appropriate bound for bit length M in order to provide insights for the actual neural network implementations. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growth of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable.

Exploiting Correlation in Stochastic Computing based Deep Neural Networks

2021 ◽  
Author(s):  
Christiam F. Frasser ◽  
Pablo Linares-Serrano ◽  
Alejandro Moran ◽  
Joan Font-Rossello ◽  
V. Canals ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Stochastic Computing

Stochastic Computing for Deep Neural Networks

2021 ◽  
pp. 161-178
Author(s):  
Sunny Bodiwala ◽  
Nirali Nanavati
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Stochastic Computing
Sign in / Sign up

Export Citation Format

Share Document