SWIRL: High-performance many-core CPU code generation for deep neural networks

2019 ◽  
Vol 33 (6) ◽  
pp. 1275-1289 ◽  
Author(s):  
Anand Venkat ◽  
Tharindu Rusira ◽  
Raj Barik ◽  
Mary Hall ◽  
Leonard Truong
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Code Generation ◽  
High Performance ◽  
Deep Neural Networks ◽  
Graphics Processing Unit ◽  
Processing Unit ◽  
Data Movement ◽  
Central Processing ◽  
The Status

Deep neural networks (DNNs) have demonstrated effectiveness in many domains including object recognition, speech recognition, natural language processing, and health care. Typically, the computations involved in DNN training and inferencing are time consuming and require efficient implementations. Existing frameworks such as TensorFlow, Theano, Torch, Cognitive Tool Kit (CNTK), and Caffe enable Graphics Processing Unit (GPUs) as the status quo devices for DNN execution, leaving Central Processing Unit (CPUs) behind. Moreover, existing frameworks forgo or limit cross layer optimization opportunities that have the potential to improve performance by significantly reducing data movement through the memory hierarchy. In this article, we describe an alternative approach called SWIRL, a compiler that provides high-performance CPU implementations for DNNs. SWIRL is built on top of the existing domain-specific language (DSL) for DNNs called LATTE. SWIRL separates DNN specification and its schedule using predefined transformation recipes for tensors and layers commonly found in DNN layers. These recipes synergize with DSL constructs to generate high-quality fused, vectorized, and parallelized code for CPUs. On an Intel Xeon Platinum 8180M CPU, SWIRL achieves performance comparable with Tensorflow integrated with MKL-DNN; on average 1.00× of Tensorflow inference and 0.99× of Tensorflow training. It also outperforms the original LATTE compiler on average by 1.22× and 1.30× on inference and training, respectively.

scholarly journals Optimized Compression for Implementing Convolutional Neural Networks on FPGA

Electronics ◽  
2019 ◽  
Vol 8 (3) ◽  
pp. 295 ◽  
Author(s):  
Min Zhang ◽  
Linpeng Li ◽  
Hai Wang ◽  
Yan Liu ◽  
Hongbo Qin ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Short Term Memory ◽  
Graphics Processing Unit ◽  
Processing Unit ◽  
Central Processing ◽  
Pruning Strategy ◽  
Field Programmable ◽  
Storage Format ◽  
Evaluation Board

Field programmable gate array (FPGA) is widely considered as a promising platform for convolutional neural network (CNN) acceleration. However, the large numbers of parameters of CNNs cause heavy computing and memory burdens for FPGA-based CNN implementation. To solve this problem, this paper proposes an optimized compression strategy, and realizes an accelerator based on FPGA for CNNs. Firstly, a reversed-pruning strategy is proposed which reduces the number of parameters of AlexNet by a factor of 13× without accuracy loss on the ImageNet dataset. Peak-pruning is further introduced to achieve better compressibility. Moreover, quantization gives another 4× with negligible loss of accuracy. Secondly, an efficient storage technique, which aims for the reduction of the whole overhead cache of the convolutional layer and the fully connected layer, is presented respectively. Finally, the effectiveness of the proposed strategy is verified by an accelerator implemented on a Xilinx ZCU104 evaluation board. By improving existing pruning techniques and the storage format of sparse data, we significantly reduce the size of AlexNet by 28×, from 243 MB to 8.7 MB. In addition, the overall performance of our accelerator achieves 9.73 fps for the compressed AlexNet. Compared with the central processing unit (CPU) and graphics processing unit (GPU) platforms, our implementation achieves 182.3× and 1.1× improvements in latency and throughput, respectively, on the convolutional (CONV) layers of AlexNet, with an 822.0× and 15.8× improvement for energy efficiency, separately. This novel compression strategy provides a reference for other neural network applications, including CNNs, long short-term memory (LSTM), and recurrent neural networks (RNNs).

scholarly journals Survey of deployment locations and underlying hardware architectures for contemporary deep neural networks

2019 ◽  
Vol 15 (8) ◽  
pp. 155014771986866
Author(s):  
Miloš Kotlar ◽  
Dragan Bojić ◽  
Marija Punt ◽  
Veljko Milutinović
Keyword(s):  
Neural Networks ◽  
Integrated Circuits ◽  
Data Processing ◽  
Deep Neural Networks ◽  
Multicore Processors ◽  
Two Dimensions ◽  
Processing Unit ◽  
Central Processing ◽  
Trade Offs ◽  
Dew Computing

This article overviews the emerging use of deep neural networks in data analytics and explores which type of underlying hardware and architectural approach is best used in various deployment locations when implementing deep neural networks. The locations which are discussed are in the cloud, fog, and dew computing (dew computing is performed by end devices). Covered architectural approaches include multicore processors (central processing unit), manycore processors (graphics processing unit), field programmable gate arrays, and application-specific integrated circuits. The proposed classification in this article divides the existing solutions into 12 different categories, organized in two dimensions. The proposed classification allows a comparison of existing architectures, which are predominantly cloud-based, and anticipated future architectures, which are expected to be hybrid cloud-fog-dew architectures for applications in Internet of Things and Wireless Sensor Networks. Researchers interested in studying trade-offs among data processing bandwidth, data processing latency, and processing power consumption would benefit from the classification made in this article.

scholarly journals A Framework for Simulating and Estimating the State and Functional Topology of Complex Dynamic Geometric Networks

2011 ◽  
Vol 23 (1) ◽  
pp. 183-214 ◽  
Author(s):  
Marius Buibas ◽  
Gabriel A. Silva
Keyword(s):  
Neural Networks ◽  
High Performance ◽  
Graphics Processing Unit ◽  
Signal Propagation ◽  
Cellular Neural Networks ◽  
Standard Test ◽  
The State ◽  
Processing Unit ◽  
Complex Dynamic ◽  
Functional Topology

We introduce a framework for simulating signal propagation in geometric networks (networks that can be mapped to geometric graphs in some space) and developing algorithms that estimate (i.e., map) the state and functional topology of complex dynamic geometric networks. Within the framework, we define the key features typically present in such networks and of particular relevance to biological cellular neural networks: dynamics, signaling, observation, and control. The framework is particularly well suited for estimating functional connectivity in cellular neural networks from experimentally observable data and has been implemented using graphics processing unit high-performance computing. Computationally, the framework can simulate cellular network signaling close to or faster than real time. We further propose a standard test set of networks to measure performance and compare different mapping algorithms.

scholarly journals DEVELOPING PARALLEL COMPUTING ALGORITHMS USING GPU’S TO DETERMINE OIL AND GAS RESERVES PRESENTED IN THE UPSTREAM (EXPLORATION) SECTOR

2020 ◽  
Author(s):  
Stefan Boodoo ◽  
Ajay Joshi
Keyword(s):  
High Performance ◽  
Oil And Gas ◽  
Gpu Computing ◽  
Graphics Processing Unit ◽  
Reservoir Rock ◽  
Processing Unit ◽  
Potential Wells ◽  
Central Processing ◽  
Rock Formations ◽  
Graphics Processing

Oil and Gas companies keep exploring every new possible method to increase the likelihood of finding a commercial hydrocarbon bearing prospect. Well logging generates gigabytes of data from various probes and sensors. After processing, a prospective reservoir will indicate areas of oil, gas, water and reservoir rock. Incorporating High Performance Computing (HPC) methodologies will allow for thousands of potential wells to be indicative of its hydrocarbon bearing potential. This study will present the use of the Graphics Processing Unit (GPU) computing as another method of analyzing probable reserves. Raw well log data from the Kansas Geological Society (1999-2018) forms the basis of the data analysis. Parallel algorithms are developed and make use of Nvidia’s Compute Unified Device Architecture (CUDA). The results gathered highlight a 5 times speedup using a Nvidia GeForce GT 330M GPU as compared to an Intel Core i7 740QM Central Processing Unit (CPU). The processed results display depth wise areas of shale and rock formations as well as water, oil and/or gas reserves.

Optimization of K-Means Clustering on Graphics Processing Unit Using Compute Unified Device Architecture

2017 ◽  
Vol 14 (1) ◽  
pp. 789-795
Author(s):  
V Saveetha ◽  
S Sophia
Keyword(s):  
High Performance ◽  
Programming Model ◽  
Graphics Processing Unit ◽  
Direct Access ◽  
Communication Overhead ◽  
Processing Unit ◽  
Compute Unified Device Architecture ◽  
Central Processing ◽  
Device Architecture ◽  
Graphics Processing

Parallel data clustering aims at using algorithms and methods to extract knowledge from fat databases in rational time using high performance architectures. The computational challenge faced by cluster analysis due to increasing capacity of data can be overcome by exploiting the power of these architectures. The recent development in parallel power of Graphics Processing Unit enables low cost high performance solutions for general purpose applications. The Compute Unified Device Architecture programming model provides application programming interface methods to handle data proficiently on Graphics Processing Unit for iterative clustering algorithms like K-Means. The existing Graphics Processing Unit based K-Means algorithms highly focus on improvising the speedup of the algorithms and fall short to handle the high time spent on transfer of data between the Central Processing Unit and Graphics Processing Unit. A competent K-Means algorithm is proposed in this paper to lessen the transfer time by introducing a novel approach to check the convergence of the algorithm and utilize the pinned memory for direct access. This algorithm outperforms the other algorithms by maximizing parallelism and utilizing the memory features. The relative speedups and the validity measure for the proposed algorithm is elevated when compared with K-Means on Graphics Processing Unit and K-Means using Flag on Graphics Processing Unit. Thus the planned approach proves that communication overhead can be reduced in K-Means clustering.

scholarly journals A TensorFlow Extension Framework for Optimized Generation of Hardware CNN Inference Engines

Technologies ◽  
2020 ◽  
Vol 8 (1) ◽  
pp. 6 ◽  
Author(s):  
Vasileios Leon ◽  
Spyridon Mouselinos ◽  
Konstantina Koliogeorgi ◽  
Sotirios Xydis ◽  
Dimitrios Soudris ◽  
...  
Keyword(s):  
Integrated Circuit ◽  
High Performance ◽  
Graphics Processing Unit ◽  
Inference Engine ◽  
Efficient Solutions ◽  
Processing Unit ◽  
Central Processing ◽  
Inference Engines ◽  
Field Programmable ◽  
Hardware Description

The workloads of Convolutional Neural Networks (CNNs) exhibit a streaming nature that makes them attractive for reconfigurable architectures such as the Field-Programmable Gate Arrays (FPGAs), while their increased need for low-power and speed has established Application-Specific Integrated Circuit (ASIC)-based accelerators as alternative efficient solutions. During the last five years, the development of Hardware Description Language (HDL)-based CNN accelerators, either for FPGA or ASIC, has seen huge academic interest due to their high-performance and room for optimizations. Towards this direction, we propose a library-based framework, which extends TensorFlow, the well-established machine learning framework, and automatically generates high-throughput CNN inference engines for FPGAs and ASICs. The framework allows software developers to exploit the benefits of FPGA/ASIC acceleration without requiring any expertise on HDL development and low-level design. Moreover, it provides a set of optimization knobs concerning the model architecture and the inference engine generation, allowing the developer to tune the accelerator according to the requirements of the respective use case. Our framework is evaluated by optimizing the LeNet CNN model on the MNIST dataset, and implementing FPGA- and ASIC-based accelerators using the generated inference engine. The optimal FPGA-based accelerator on Zynq-7000 delivers 93% less memory footprint and 54% less Look-Up Table (LUT) utilization, and up to 10× speedup on the inference execution vs. different Graphics Processing Unit (GPU) and Central Processing Unit (CPU) implementations of the same model, in exchange for a negligible accuracy loss, i.e., 0.89%. For the same accuracy drop, the 45 nm standard-cell-based ASIC accelerator provides an implementation which operates at 520 MHz and occupies an area of 0.059 mm 2 , while the power consumption is ∼7.5 mW.

High-performance computer graphics technologies in engineering applications

2019 ◽  
Vol 16 (2) ◽  
pp. 304-308
Author(s):  
Chao Peng
Keyword(s):  
Computer Graphics ◽  
High Performance ◽  
Graphics Processing Unit ◽  
General Purpose ◽  
Processing Unit ◽  
Data Streaming ◽  
Engineering Applications ◽  
Content Type ◽  
Central Processing ◽  
Advantages And Disadvantages

Purpose The purpose of this paper is to investigate possibilities to adopt state-of-the-art computer graphics technologies for big data visualization in engineering applications. Toward this purpose, a conceptual heterogeneous system is proposed for graphical rendering, which is established with multiple central processing unit cores and multiple graphics processing unit GPUs. Design/methodology/approach The design of the system supports both general-purpose computation and graphics-related computation. Three processing components are discussed to fulfill the execution requirements in load balancing, data streaming and display. This design fully uses computational and memory resources and enhances the performance with the support of GPU-based parallelization. Findings The advantages and disadvantages of particular technical methods for each processing component are discussed. The possible ways to integrate them are analyzed. Originality/value This work has contributions of using computer graphics technologies in engineering applications.

scholarly journals HYPERDOCK: Improving virtual screening through parallel hyperheuristics

2019 ◽  
Vol 34 (1) ◽  
pp. 30-41
Author(s):  
Baldomero Imbernón ◽  
Antonio Llanes ◽  
José-Matías Cutillas-Lozano ◽  
Domingo Giménez
Keyword(s):  
Virtual Screening ◽  
High Performance ◽  
Graphics Processing Unit ◽  
Processing Unit ◽  
Central Processing ◽  
Pharmacological Targets ◽  
Graphics Processing ◽  
Computational Systems ◽  
Performance Computing ◽  
Different Levels

Virtual screening (VS) methods aid clinical research by predicting the interaction of ligands with pharmacological targets. The computational requirements of VS, along with the size of the databases, propitiate the use of high-performance computing. METADOCK is a tool for the application of metaheuristics to VS in heterogeneous clusters of computers based on central processing unit (CPU) and graphics processing unit (GPU). HYPERDOCK represents a step forward; the exploration for satisfactory metaheuristics is systematically approached by means of hyperheuristics working on top of the metaheuristic schema of METADOCK. Multiple metaheuristics are explored, so the process is computationally demanding. HYPERDOCK exploits the parallelism of METADOCK and includes parallelism at its own level. The different levels of parallelism can be used to exploit the parallelism offered by computational systems composed of multicore CPU + multi-GPUs. The efficient exploitation of these systems enables HYPERDOCK to improve ligand–receptor binding.

scholarly journals Deep Learning Framework for Vehicle and Pedestrian Detection in Rural Roads on an Embedded GPU

Electronics ◽  
2020 ◽  
Vol 9 (4) ◽  
pp. 589 ◽  
Author(s):  
Luis Barba-Guaman ◽  
José Eugenio Naranjo ◽  
Anthony Ortiz
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Deep Learning ◽  
Embedded System ◽  
Rural Areas ◽  
High Performance ◽  
Processing Time ◽  
Graphics Processing Unit ◽  
Pedestrian Detection ◽  
Processing Unit

Object detection, one of the most fundamental and challenging problems in computer vision. Nowadays some dedicated embedded systems have emerged as a powerful strategy for deliver high processing capabilities including the NVIDIA Jetson family. The aim of the present work is the recognition of objects in complex rural areas through an embedded system, as well as the verification of accuracy and processing time. For this purpose, a low power embedded Graphics Processing Unit (Jetson Nano) has been selected, which allows multiple neural networks to be run in simultaneous and a computer vision algorithm to be applied for image recognition. As well, the performance of these deep learning neural networks such as ssd-mobilenet v1 and v2, pednet, multiped and ssd-inception v2 has been tested. Moreover, it was found that the accuracy and processing time were in some cases improved when all the models suggested in the research were applied. The pednet network model provides a high performance in pedestrian recognition, however, the sdd-mobilenet v2 and ssd-inception v2 models are better at detecting other objects such as vehicles in complex scenarios.

scholarly journals High Performance GPU-Based Fourier Volume Rendering

2015 ◽  
Vol 2015 ◽  
pp. 1-13 ◽  
Author(s):  
Marwan Abdellah ◽  
Ayman Eldeib ◽  
Amr Sharawi
Keyword(s):  
Volume Rendering ◽  
High Performance ◽  
Graphics Processing Unit ◽  
Rapid Evolution ◽  
Processing Unit ◽  
Central Processing ◽  
Slice Theorem ◽  
Cuda Technology ◽  
Gpu Architectures ◽  
3D Volume

Fourier volume rendering (FVR) is a significant visualization technique that has been used widely in digital radiography. As a result of itsO(N2log⁡N)time complexity, it provides a faster alternative to spatial domain volume rendering algorithms that areO(N3)computationally complex. Relying on theFourier projection-slice theorem, this technique operates on the spectral representation of a 3D volume instead of processing its spatial representation to generate attenuation-only projections that look likeX-ray radiographs. Due to the rapid evolution of its underlying architecture, the graphics processing unit (GPU) became an attractive competent platform that can deliver giant computational raw power compared to the central processing unit (CPU) on a per-dollar-basis. The introduction of the compute unified device architecture (CUDA) technology enables embarrassingly-parallel algorithms to run efficiently on CUDA-capable GPU architectures. In this work, a high performance GPU-accelerated implementation of the FVR pipeline on CUDA-enabled GPUs is presented. This proposed implementation can achieve a speed-up of 117x compared to a single-threaded hybrid implementation that uses the CPU and GPU together by taking advantage of executing the rendering pipeline entirely on recent GPU architectures.

Sign in / Sign up

Export Citation Format

Share Document