Adaptive weight-bit inversion for state error reduction for robust and efficient deep neural network inference using MLC NAND Flash

2022 ◽  
pp. 102400
Author(s):  
Jaehun Jang ◽  
Jong Hwan Ko
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Network Inference ◽  
Error Reduction ◽  
Nand Flash ◽  
Adaptive Weight ◽  
State Error

scholarly journals Collaborative Intelligence: Accelerating Deep Neural Network Inference via Device-Edge Synergy

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Nanliang Shan ◽  
Zecong Ye ◽  
Xiaolong Cui
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
Network Inference ◽  
Partition Scheme ◽  
Model Inference ◽  
Cloud Server ◽  
Hardware Configuration ◽  
Collaborative Intelligence ◽  
Service Latency

With the development of mobile edge computing (MEC), more and more intelligent services and applications based on deep neural networks are deployed on mobile devices to meet the diverse and personalized needs of users. Unfortunately, deploying and inferencing deep learning models on resource-constrained devices are challenging. The traditional cloud-based method usually runs the deep learning model on the cloud server. Since a large amount of input data needs to be transmitted to the server through WAN, it will cause a large service latency. This is unacceptable for most current latency-sensitive and computation-intensive applications. In this paper, we propose Cogent, an execution framework that accelerates deep neural network inference through device-edge synergy. In the Cogent framework, it is divided into two operation stages, including the automatic pruning and partition stage and the containerized deployment stage. Cogent uses reinforcement learning (RL) to automatically predict pruning and partition strategies based on feedback from the hardware configuration and system conditions so that the pruned and partitioned model can better adapt to the system environment and user hardware configuration. Then through containerized deployment to the device and the edge server to accelerate model inference, experiments show that the learning-based hardware-aware automatic pruning and partition scheme can significantly reduce the service latency, and it accelerates the overall model inference process while maintaining accuracy. Using this method can accelerate up to 8.89× without loss of accuracy of more than 7%.

scholarly journals Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

2020 ◽  
Vol 19 (1) ◽  
pp. 447-457 ◽  
Author(s):  
En Li ◽  
Liekang Zeng ◽  
Zhi Zhou ◽  
Xu Chen
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Network Inference ◽  
Edge Computing ◽  
On Demand

SNAP: A 1.67 — 21.55TOPS/W Sparse Neural Acceleration Processor for Unstructured Sparse Deep Neural Network Inference in 16nm CMOS

2019 ◽  
Author(s):  
Jie-Fang Zhang ◽  
Ching-En Lee ◽  
Chester Liu ◽  
Yakun Sophia Shao ◽  
Stephen W. Keckler ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Network Inference

scholarly journals Accurate deep neural network inference using computational phase-change memory

2020 ◽  
Vol 11 (1) ◽  
Author(s):  
Vinay Joshi ◽  
Manuel Le Gallo ◽  
Simon Haefeli ◽  
Irem Boybat ◽  
S. R. Nandakumar ◽  
...  
Keyword(s):  
Neural Network ◽  
Phase Change ◽  
Deep Neural Network ◽  
Network Inference ◽  
Phase Change Memory ◽  
Change Memory

Fast Sparse Deep Neural Network Inference with Flexible SpMM Optimization Space Exploration

2021 ◽  
Author(s):  
Jie Xin ◽  
Xianqi Ye ◽  
Long Zheng ◽  
Qinggang Wang ◽  
Yu Huang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Network Inference ◽  
Space Exploration

S-FLASH: A NAND Flash-based Deep Neural Network Accelerator Exploiting Bit-level Sparsity

2021 ◽  
pp. 1-1
Author(s):  
Myeonggu Kang ◽  
Hyeonuk Kim ◽  
Hyein Shin ◽  
Jaehyeong Sim ◽  
Kyeonghan Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Nand Flash
Sign in / Sign up

Export Citation Format

Share Document