scholarly journals Accelerating Neural Network Inference on FPGA-Based Platforms—A Survey

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1025
Author(s):  
Ran Wu ◽  
Xinmin Guo ◽  
Jian Du ◽  
Junbao Li
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Network Inference ◽  
Semantic Segmentation ◽  
Object Identification ◽  
Data Reuse ◽  
Practical Applications ◽  
Resource Limited ◽  
Video Recognition ◽  
Future Work

The breakthrough of deep learning has started a technological revolution in various areas such as object identification, image/video recognition and semantic segmentation. Neural network, which is one of representative applications of deep learning, has been widely used and developed many efficient models. However, the edge implementation of neural network inference is restricted because of conflicts between the high computation and storage complexity and resource-limited hardware platforms in applications scenarios. In this paper, we research neural networks which are involved in the acceleration on FPGA-based platforms. The architecture of networks and characteristics of FPGA are analyzed, compared and summarized, as well as their influence on acceleration tasks. Based on the analysis, we generalize the acceleration strategies into five aspects—computing complexity, computing parallelism, data reuse, pruning and quantization. Then previous works on neural network acceleration are introduced following these topics. We summarize how to design a technical route for practical applications based on these strategies. Challenges in the path are discussed to provide guidance for future work.

scholarly journals Orchard Mapping with Deep Learning Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3813
Author(s):  
Athanasios Anagnostis ◽  
Aristotelis C. Tagarakis ◽  
Dimitrios Kateris ◽  
Vasileios Moysiadis ◽  
Claus Grøn Sørensen ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Automated Detection ◽  
Aerial Images ◽  
Training Dataset ◽  
Field Boundary ◽  
Different Seasons ◽  
Detection And Localization ◽  
Different Levels

This study aimed to propose an approach for orchard trees segmentation using aerial images based on a deep learning convolutional neural network variant, namely the U-net network. The purpose was the automated detection and localization of the canopy of orchard trees under various conditions (i.e., different seasons, different tree ages, different levels of weed coverage). The implemented dataset was composed of images from three different walnut orchards. The achieved variability of the dataset resulted in obtaining images that fell under seven different use cases. The best-trained model achieved 91%, 90%, and 87% accuracy for training, validation, and testing, respectively. The trained model was also tested on never-before-seen orthomosaic images or orchards based on two methods (oversampling and undersampling) in order to tackle issues with out-of-the-field boundary transparent pixels from the image. Even though the training dataset did not contain orthomosaic images, it achieved performance levels that reached up to 99%, demonstrating the robustness of the proposed approach.

scholarly journals Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

2021 ◽  
Vol 26 (1) ◽  
pp. 200-215
Author(s):  
Muhammad Alam ◽  
Jian-Feng Wang ◽  
Cong Guangpei ◽  
LV Yunrong ◽  
Yuanfang Chen
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Natural Scene ◽  
Remote Sensing Images ◽  
Advantages And Disadvantages ◽  
Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

scholarly journals Real-time deep learning semantic segmentation during intra-operative surgery for 3D augmented reality assistance

2021 ◽  
Author(s):  
Leonardo Tanzi ◽  
Pietro Piazzolla ◽  
Francesco Porpiglia ◽  
Enrico Vezzetti
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Augmented Reality ◽  
Ad Hoc ◽  
Geodesic Distance ◽  
Semantic Segmentation ◽  
Endoscopic Image ◽  
Operative Surgery ◽  
Endoscopic Videos

Abstract Purpose The current study aimed to propose a Deep Learning (DL) and Augmented Reality (AR) based solution for a in-vivo robot-assisted radical prostatectomy (RARP), to improve the precision of a published work from our group. We implemented a two-steps automatic system to align a 3D virtual ad-hoc model of a patient’s organ with its 2D endoscopic image, to assist surgeons during the procedure. Methods This approach was carried out using a Convolutional Neural Network (CNN) based structure for semantic segmentation and a subsequent elaboration of the obtained output, which produced the needed parameters for attaching the 3D model. We used a dataset obtained from 5 endoscopic videos (A, B, C, D, E), selected and tagged by our team’s specialists. We then evaluated the most performing couple of segmentation architecture and neural network and tested the overlay performances. Results U-Net stood out as the most effecting architectures for segmentation. ResNet and MobileNet obtained similar Intersection over Unit (IoU) results but MobileNet was able to elaborate almost twice operations per seconds. This segmentation technique outperformed the results from the former work, obtaining an average IoU for the catheter of 0.894 (σ = 0.076) compared to 0.339 (σ = 0.195). This modifications lead to an improvement also in the 3D overlay performances, in particular in the Euclidean Distance between the predicted and actual model’s anchor point, from 12.569 (σ= 4.456) to 4.160 (σ = 1.448) and in the Geodesic Distance between the predicted and actual model’s rotations, from 0.266 (σ = 0.131) to 0.169 (σ = 0.073). Conclusion This work is a further step through the adoption of DL and AR in the surgery domain. In future works, we will overcome the limits of this approach and finally improve every step of the surgical procedure.

scholarly journals Collaborative Intelligence: Accelerating Deep Neural Network Inference via Device-Edge Synergy

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Nanliang Shan ◽  
Zecong Ye ◽  
Xiaolong Cui
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
Network Inference ◽  
Partition Scheme ◽  
Model Inference ◽  
Cloud Server ◽  
Hardware Configuration ◽  
Collaborative Intelligence ◽  
Service Latency

With the development of mobile edge computing (MEC), more and more intelligent services and applications based on deep neural networks are deployed on mobile devices to meet the diverse and personalized needs of users. Unfortunately, deploying and inferencing deep learning models on resource-constrained devices are challenging. The traditional cloud-based method usually runs the deep learning model on the cloud server. Since a large amount of input data needs to be transmitted to the server through WAN, it will cause a large service latency. This is unacceptable for most current latency-sensitive and computation-intensive applications. In this paper, we propose Cogent, an execution framework that accelerates deep neural network inference through device-edge synergy. In the Cogent framework, it is divided into two operation stages, including the automatic pruning and partition stage and the containerized deployment stage. Cogent uses reinforcement learning (RL) to automatically predict pruning and partition strategies based on feedback from the hardware configuration and system conditions so that the pruned and partitioned model can better adapt to the system environment and user hardware configuration. Then through containerized deployment to the device and the edge server to accelerate model inference, experiments show that the learning-based hardware-aware automatic pruning and partition scheme can significantly reduce the service latency, and it accelerates the overall model inference process while maintaining accuracy. Using this method can accelerate up to 8.89× without loss of accuracy of more than 7%.

Deep learning Q inversion from reflection seismic data with strong attenuation using an encoder-decoder convolutional neural network: an example from South China Sea

2020 ◽  
Author(s):  
Hao Zhang ◽  
Jianguang Han ◽  
Heng Zhang ◽  
Yi Zhang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
High Resolution ◽  
Seismic Data ◽  
Semantic Segmentation ◽  
Attenuation Effect ◽  
Attenuation Model ◽  
Reflection Seismic ◽  
Q Model ◽  
Segmentation Task

<p>The seismic waves exhibit various types of attenuation while propagating through the subsurface, which is strongly related to the complexity of the earth. Anelasticity of the subsurface medium, which is quantified by the quality factor Q, causes dissipation of seismic energy. Attenuation distorts the phase of the seismic data and decays the higher frequencies in the data more than lower frequencies. Strong attenuation effect resulting from geology such as gas pocket is a notoriously challenging problem for high resolution imaging because it strongly reduces the amplitude and downgrade the imaging quality of deeper events. To compensate this attenuation effect, first we need to accurately estimate the attenuation model (Q). However, it is challenging to directly derive a laterally and vertically varying attenuation model in depth domain from the surface reflection seismic data. This research paper proposes a method to derive the anomalous Q model corresponding to strong attenuative media from marine reflection seismic data using a deep-learning approach, the convolutional neural network (CNN). We treat Q anomaly detection problem as a semantic segmentation task and train an encoder-decoder CNN (U-Net) to perform a pixel-by-pixel prediction on the seismic section to invert a pixel group belongs to different level of attenuation probability which can help to build up the attenuation model. The proposed method in this paper uses a volume of marine 3D reflection seismic data for network training and validation, which needs only a very small amount of data as the training set due to the feature of U-Net, a specific encoder-decoder CNN architecture in semantic segmentation task. Finally, in order to evaluate the attenuation model result predicted by the proposed method, we validate the predicted heterogeneous Q model using de-absorption pre-stack depth migration (Q-PSDM), a high-resolution depth imaging result with reasonable compensation is obtained.</p>

scholarly journals Automated segmentation of articular disc of the temporomandibular joint on magnetic resonance images using deep learning

2022 ◽  
Vol 12 (1) ◽  
Author(s):  
Shota Ito ◽  
Yuichi Mine ◽  
Yuki Yoshimi ◽  
Saori Takeda ◽  
Akari Tanaka ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Magnetic Resonance ◽  
Convolutional Neural Network ◽  
Temporomandibular Joint ◽  
Temporomandibular Disorders ◽  
Temporomandibular Disorder ◽  
Semantic Segmentation ◽  
Magnetic Resonance Images ◽  
Articular Disc

AbstractTemporomandibular disorders are typically accompanied by a number of clinical manifestations that involve pain and dysfunction of the masticatory muscles and temporomandibular joint. The most important subgroup of articular abnormalities in patients with temporomandibular disorders includes patients with different forms of articular disc displacement and deformation. Here, we propose a fully automated articular disc detection and segmentation system to support the diagnosis of temporomandibular disorder on magnetic resonance imaging. This system uses deep learning-based semantic segmentation approaches. The study included a total of 217 magnetic resonance images from 10 patients with anterior displacement of the articular disc and 10 healthy control subjects with normal articular discs. These images were used to evaluate three deep learning-based semantic segmentation approaches: our proposed convolutional neural network encoder-decoder named 3DiscNet (Detection for Displaced articular DISC using convolutional neural NETwork), U-Net, and SegNet-Basic. Of the three algorithms, 3DiscNet and SegNet-Basic showed comparably good metrics (Dice coefficient, sensitivity, and positive predictive value). This study provides a proof-of-concept for a fully automated deep learning-based segmentation methodology for articular discs on magnetic resonance images, and obtained promising initial results, indicating that the method could potentially be used in clinical practice for the assessment of temporomandibular disorders.

Weapon and Object Detection Using Mobile-Net SSD Model in Deep Neural Network

2021 ◽  
Vol 9 (8) ◽  
pp. 1573-1582
Author(s):  
S Gopi Naik
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Deep Learning ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Optimization Algorithms ◽  
Object Identification ◽  
Integrated System ◽  
Raspberry Pi ◽  
High Level

Abstract: The plan is to establish an integrated system that can manage high-quality visual information and also detect weapons quickly and efficiently. It is obtained by integrating ARM-based computer vision and optimization algorithms with deep neural networks able to detect the presence of a threat. The whole system is connected to a Raspberry Pi module, which will capture live broadcasting and evaluate it using a deep convolutional neural network. Due to the intimate interaction between object identification and video and image analysis in real-time objects, By generating sophisticated ensembles that incorporate various low-level picture features with high-level information from object detection and scenario classifiers, their performance can quickly plateau. Deep learning models, which can learn semantic, high-level, deeper features, have been developed to overcome the issues that are present in optimization algorithms. It presents a review of deep learning based object detection frameworks that use Convolutional Neural Network layers for better understanding of object detection. The Mobile-Net SSD model behaves differently in network design, training methods, and optimization functions, among other things. The crime rate in suspicious areas has been reduced as a consequence of weapon detection. However, security is always a major concern in human life. The Raspberry Pi module, or computer vision, has been extensively used in the detection and monitoring of weapons. Due to the growing rate of human safety protection, privacy and the integration of live broadcasting systems which can detect and analyse images, suspicious areas are becoming indispensable in intelligence. This process uses a Mobile-Net SSD algorithm to achieve automatic weapons and object detection. Keywords: Computer Vision, Weapon and Object Detection, Raspberry Pi Camera, RTSP, SMTP, Mobile-Net SSD, CNN, Artificial Intelligence.

scholarly journals 3D Reconstruction of a Complex Grid Structure Combining UAS Images and Deep Learning

2020 ◽  
Vol 12 (19) ◽  
pp. 3128
Author(s):  
Vladimir A. Knyaz ◽  
Vladimir V. Kniaz ◽  
Fabio Remondino ◽  
Sergey Y. Zheltov ◽  
Armin Gruen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
3D Reconstruction ◽  
Network Model ◽  
Neural Network Model ◽  
Feature Matching ◽  
Semantic Segmentation ◽  
Unmanned Aerial Systems ◽  
Grid Structure ◽  
Large Size

The latest advances in technical characteristics of unmanned aerial systems (UAS) and their onboard sensors opened the way for smart flying vehicles exploiting new application areas and allowing to perform missions seemed to be impossible before. One of these complicated tasks is the 3D reconstruction and monitoring of large-size, complex, grid-like structures as radio or television towers. Although image-based 3D survey contains a lot of visual and geometrical information useful for making preliminary conclusions on construction health, standard photogrammetric processing fails to perform dense and robust 3D reconstruction of complex large-size mesh structures. The main problem of such objects is repeated and self-occlusive similar elements resulting in false feature matching. This paper presents a method developed for an accurate Multi-View Stereo (MVS) dense 3D reconstruction of the Shukhov Radio Tower in Moscow (Russia) based on UAS photogrammetric survey. A key element for the successful image-based 3D reconstruction is the developed WireNetV2 neural network model for robust automatic semantic segmentation of wire structures. The proposed neural network provides high matching quality due to an accurate masking of the tower elements. The main contributions of the paper are: (1) a deep learning WireNetV2 convolutional neural network model that outperforms the state-of-the-art results of semantic segmentation on a dataset containing images of grid structures of complicated topology with repeated elements, holes, self-occlusions, thus providing robust grid structure masking and, as a result, accurate 3D reconstruction, (2) an advanced image-based pipeline aided by a neural network for the accurate 3D reconstruction of the large-size and complex grid structured, evaluated on UAS imagery of Shukhov radio tower in Moscow.

scholarly journals Object Identification and Localization Using Grad-CAM++ with Mask Regional Convolution Neural Network

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1541
Author(s):  
Xavier Alphonse Inbaraj ◽  
Charlyn Villavicencio ◽  
Julio Jerison Macrohon ◽  
Jyh-Horng Jeng ◽  
Jer-Guang Hsieh
Keyword(s):  
Neural Network ◽  
Real Time ◽  
Semantic Segmentation ◽  
Object Identification ◽  
Convolution Neural Network ◽  
Object Localization ◽  
Localization Method ◽  
Wide Range ◽  
Detection Of Objects ◽  
Real Time Applications

One of the fundamental advancements in the deployment of object detectors in real-time applications is to improve object recognition against obstruction, obscurity, and noises in images. In addition, object detection is a challenging task since it needs the correct detection of objects from images. Semantic segmentation and localization are an important module to recognizing an object in an image. The object localization method (Grad-CAM++) is mostly used by researchers for object localization, which uses the gradient with a convolution layer to build a localization map for important regions on the image. This paper proposes a method called Combined Grad-CAM++ with the Mask Regional Convolution Neural Network (GC-MRCNN) in order to detect objects in the image and also localization. The major advantage of proposed method is that they outperform all the counterpart methods in the domain and can also be used in unsupervised environments. The proposed detector based on GC-MRCNN provides a robust and feasible ability in detecting and classifying objects exist and their shapes in real time. It is found that the proposed method is able to perform highly effectively and efficiently in a wide range of images and provides higher resolution visual representation than existing methods (Grad-CAM, Grad-CAM++), which was proven by comparing various algorithms.

scholarly journals Multiorgan Detection: Deep Learning Based Techniques and Research Directions

2019 ◽  
Vol 9 (1) ◽  
pp. 4590-4594
Keyword(s):  
Deep Learning ◽  
Semantic Segmentation ◽  
Diagnostic Systems ◽  
Abdominal Ct ◽  
Learning Methods ◽  
Research Directions ◽  
Detection Techniques ◽  
Clinical Procedures ◽  
Future Work ◽  
Segmentation Problem

Automatic organ segmentation plays an important role in clinical procedures such as planning of radiation therapies and in computer-aided diagnostic systems. Several state-of –art techniques are available for multiorgan segmentation, however deep learning methods are doing exceptionally well and become the methodology of choice to analyze medical images. This intensively carried out work is conducted for deep learning methods applied on various organs in abdominal CT images. Firstly, this paper formulates segmentation, semantic segmentation problem and their methods. Secondly, multiorgan detection techniques based on deep learning along with their contributions, chosen datasets and gaps are discussed. It presents the metrics used to evaluate these methods. Finally, interesting conclusions has been drawn which will add to do future work using deep learning.

Sign in / Sign up

Export Citation Format

Share Document