How Can Deep Neural Networks Be Generated Efficiently for Devices with Limited Resources?

2018 ◽  
pp. 24-33
Author(s):  
Unai Elordi ◽  
Luis Unzueta ◽  
Ignacio Arganda-Carreras ◽  
Oihana Otaegui
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Limited Resources

scholarly journals AxP: A HW-SW Co-Design Pipeline for Energy-Efficient Approximated ConvNets via Associative Matching

2021 ◽  
Vol 11 (23) ◽  
pp. 11164
Author(s):  
Luca Mocerino ◽  
Andrea Calimera
Keyword(s):  
Neural Networks ◽  
Speech Processing ◽  
Energy Efficient ◽  
High Performance ◽  
Deep Neural Networks ◽  
Energy Savings ◽  
Limited Resources ◽  
Model Accuracy ◽  
Rule Based ◽  
Floating Point Unit

The reduction in energy consumption is key for deep neural networks (DNNs) to ensure usability and reliability, whether they are deployed on low-power end-nodes with limited resources or high-performance platforms that serve large pools of users. Leveraging the over-parametrization shown by many DNN models, convolutional neural networks (ConvNets) in particular, energy efficiency can be improved substantially preserving the model accuracy. The solution proposed in this work exploits the intrinsic redundancy of ConvNets to maximize the reuse of partial arithmetic results during the inference stages. Specifically, the weight-set of a given ConvNet is discretized through a clustering procedure such that the largest possible number of inner multiplications fall into predefined bins; this allows an off-line computation of the most frequent results, which in turn can be stored locally and retrieved when needed during the forward pass. Such a reuse mechanism leads to remarkable energy savings with the aid of a custom processing element (PE) that integrates an associative memory with a standard floating-point unit (FPU). Moreover, the adoption of an approximate associative rule based on a partial bit-match increases the hit rate over the pre-computed results, maximizing the energy reduction even further. Results collected on a set of ConvNets trained for computer vision and speech processing tasks reveal that the proposed associative-based hw-sw co-design achieves up to 77% in energy savings with less than 1% in accuracy loss.

scholarly journals Neural Fourier Energy Disaggregation

Sensors ◽  
2022 ◽  
Vol 22 (2) ◽  
pp. 473
Author(s):  
Christoforos Nalmpantis ◽  
Nikolaos Virtsionis Gkalinikis ◽  
Dimitris Vrakas
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Fourier Transformation ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Limited Resources ◽  
Learning Models ◽  
Neural Architecture ◽  
Energy Disaggregation ◽  
Computationally Expensive

Deploying energy disaggregation models in the real-world is a challenging task. These models are usually deep neural networks and can be costly when running on a server or prohibitive when the target device has limited resources. Deep learning models are usually computationally expensive and they have large storage requirements. Reducing the computational cost and the size of a neural network, without trading off any performance is not a trivial task. This paper suggests a novel neural architecture that has less learning parameters, smaller size and fast inference time without trading off performance. The proposed architecture performs on par with two popular strong baseline models. The key characteristic is the Fourier transformation which has no learning parameters and it can be computed efficiently.

Deep neural networks trained with heavier data augmentation learn features closer to representations in hIT

2018 ◽  
Author(s):  
Alex Hernández-García ◽  
Johannes Mehrer ◽  
Nikolaus Kriegeskorte ◽  
Peter König ◽  
Tim C. Kietzmann
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Data Augmentation

Representation of adversarial images in deep neural networks and the human brain

2018 ◽  
Author(s):  
Chi Zhang ◽  
Xiaohan Duan ◽  
Ruyuan Zhang ◽  
Li Tong
Keyword(s):  
Neural Networks ◽  
Human Brain ◽  
Deep Neural Networks

Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks

2018 ◽  
Author(s):  
Daniel Povey ◽  
Gaofeng Cheng ◽  
Yiming Wang ◽  
Ke Li ◽  
Hainan Xu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Matrix Factorization ◽  
Deep Neural Networks ◽  
Low Rank ◽  
Rank Matrix ◽  
Low Rank Matrix
Sign in / Sign up

Export Citation Format

Share Document