scholarly journals Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

2018 ◽  
Author(s):  
Alexey A. Shvets ◽  
Alexander Rakhlin ◽  
Alexandr A. Kalinin ◽  
Vladimir I. Iglovikov
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Robot Assisted ◽  
Automatic Instrument ◽  
Binary Segmentation ◽  
Video Feed ◽  
Class Segmentation ◽  
Segmentation Problem ◽  
Robot Assisted Surgery

AbstractSemantic segmentation of robotic instruments is an important problem for the robot-assisted surgery. One of the main challenges is to correctly detect an instrument’s position for the tracking and pose estimation in the vicinity of surgical scenes. Accurate pixel-wise instrument segmentation is needed to address this challenge. In this paper we describe our deep learning-based approach for robotic instrument segmentation. Our approach demonstrates an improvement over the state-of-the-art results using several novel deep neural network architectures. It addressed the binary segmentation problem, where every pixel in an image is labeled as an instrument or background from the surgery video feed. In addition, we solve a multi-class segmentation problem, in which we distinguish between different instruments or different parts of an instrument from the background. In this setting, our approach outperforms other methods for automatic instrument segmentation thereby providing state-of-the-art results for these problems. The source code for our solution is made publicly available.

scholarly journals Towards Scalable Economic Photovoltaic Potential Analysis Using Aerial Images and Deep Learning

Energies ◽  
2021 ◽  
Vol 14 (13) ◽  
pp. 3800
Author(s):  
Sebastian Krapf ◽  
Nils Kemmerzell ◽  
Syed Khawaja Haseeb Khawaja Haseeb Uddin ◽  
Manuel Hack Hack Vázquez ◽  
Fabian Netzler ◽  
...  
Keyword(s):  
Deep Learning ◽  
System Analysis ◽  
State Of The Art ◽  
Critical Role ◽  
Semantic Segmentation ◽  
Energy System ◽  
Aerial Images ◽  
Potential Analysis ◽  
3D Data ◽  
Challenges And Opportunities

Roof-mounted photovoltaic systems play a critical role in the global transition to renewable energy generation. An analysis of roof photovoltaic potential is an important tool for supporting decision-making and for accelerating new installations. State of the art uses 3D data to conduct potential analyses with high spatial resolution, limiting the study area to places with available 3D data. Recent advances in deep learning allow the required roof information from aerial images to be extracted. Furthermore, most publications consider the technical photovoltaic potential, and only a few publications determine the photovoltaic economic potential. Therefore, this paper extends state of the art by proposing and applying a methodology for scalable economic photovoltaic potential analysis using aerial images and deep learning. Two convolutional neural networks are trained for semantic segmentation of roof segments and superstructures and achieve an Intersection over Union values of 0.84 and 0.64, respectively. We calculated the internal rate of return of each roof segment for 71 buildings in a small study area. A comparison of this paper’s methodology with a 3D-based analysis discusses its benefits and disadvantages. The proposed methodology uses only publicly available data and is potentially scalable to the global level. However, this poses a variety of research challenges and opportunities, which are summarized with a focus on the application of deep learning, economic photovoltaic potential analysis, and energy system analysis.

scholarly journals InstantDL: an easy-to-use deep learning pipeline for image segmentation and classification

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Dominik Jens Elias Waibel ◽  
Sayedali Shetab Boushehri ◽  
Carsten Marr
Keyword(s):  
Image Processing ◽  
Deep Learning ◽  
Specific Problem ◽  
State Of The Art ◽  
Image Data ◽  
Semantic Segmentation ◽  
Parameter Tuning ◽  
Cellular Processes ◽  
Minimal Effort ◽  
Instance Segmentation

Abstract Background Deep learning contributes to uncovering molecular and cellular processes with highly performant algorithms. Convolutional neural networks have become the state-of-the-art tool to provide accurate and fast image data processing. However, published algorithms mostly solve only one specific problem and they typically require a considerable coding effort and machine learning background for their application. Results We have thus developed InstantDL, a deep learning pipeline for four common image processing tasks: semantic segmentation, instance segmentation, pixel-wise regression and classification. InstantDL enables researchers with a basic computational background to apply debugged and benchmarked state-of-the-art deep learning algorithms to their own data with minimal effort. To make the pipeline robust, we have automated and standardized workflows and extensively tested it in different scenarios. Moreover, it allows assessing the uncertainty of predictions. We have benchmarked InstantDL on seven publicly available datasets achieving competitive performance without any parameter tuning. For customization of the pipeline to specific tasks, all code is easily accessible and well documented. Conclusions With InstantDL, we hope to empower biomedical researchers to conduct reproducible image processing with a convenient and easy-to-use pipeline.

scholarly journals A Novel Object-Based Deep Learning Framework for Semantic Segmentation of Very High-Resolution Remote Sensing Data: Comparison with Convolutional and Fully Convolutional Networks

2019 ◽  
Vol 11 (6) ◽  
pp. 684 ◽  
Author(s):  
Maria Papadomanolaki ◽  
Maria Vakalopoulou ◽  
Konstantinos Karantzalos
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Novel Object ◽  
Convolutional Networks ◽  
Learning Framework ◽  
Fully Convolutional Networks ◽  
Object Based ◽  
Deep Networks ◽  
Very High

Deep learning architectures have received much attention in recent years demonstrating state-of-the-art performance in several segmentation, classification and other computer vision tasks. Most of these deep networks are based on either convolutional or fully convolutional architectures. In this paper, we propose a novel object-based deep-learning framework for semantic segmentation in very high-resolution satellite data. In particular, we exploit object-based priors integrated into a fully convolutional neural network by incorporating an anisotropic diffusion data preprocessing step and an additional loss term during the training process. Under this constrained framework, the goal is to enforce pixels that belong to the same object to be classified at the same semantic category. We compared thoroughly the novel object-based framework with the currently dominating convolutional and fully convolutional deep networks. In particular, numerous experiments were conducted on the publicly available ISPRS WGII/4 benchmark datasets, namely Vaihingen and Potsdam, for validation and inter-comparison based on a variety of metrics. Quantitatively, experimental results indicate that, overall, the proposed object-based framework slightly outperformed the current state-of-the-art fully convolutional networks by more than 1% in terms of overall accuracy, while intersection over union results are improved for all semantic categories. Qualitatively, man-made classes with more strict geometry such as buildings were the ones that benefit most from our method, especially along object boundaries, highlighting the great potential of the developed approach.

Robot-assisted surgery for lung cancer: State of the art and perspectives

Lung Cancer ◽  
2016 ◽  
Vol 101 ◽  
pp. 28-34 ◽  
Author(s):  
Giulia Veronesi ◽  
Pierluigi Novellis ◽  
Emanuele Voulaz ◽  
Marco Alloisio
Keyword(s):  
Lung Cancer ◽  
State Of The Art ◽  
Robot Assisted ◽  
Robot Assisted Surgery

scholarly journals Transformer-Based Decoder Designs for Semantic Segmentation on Remotely Sensed Images

2021 ◽  
Vol 13 (24) ◽  
pp. 5100
Author(s):  
Teerapong Panboonyuen ◽  
Kulsawasd Jitkajornwanich ◽  
Siam Lawawirojwong ◽  
Panu Srestasathiern ◽  
Peerapon Vateekul
Keyword(s):  
Image Processing ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Language Processing ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Landsat 8 ◽  
Convolutional Network ◽  
Image Labeling ◽  
Feature Pyramid

Transformers have demonstrated remarkable accomplishments in several natural language processing (NLP) tasks as well as image processing tasks. Herein, we present a deep-learning (DL) model that is capable of improving the semantic segmentation network in two ways. First, utilizing the pre-training Swin Transformer (SwinTF) under Vision Transformer (ViT) as a backbone, the model weights downstream tasks by joining task layers upon the pretrained encoder. Secondly, decoder designs are applied to our DL network with three decoder designs, U-Net, pyramid scene parsing (PSP) network, and feature pyramid network (FPN), to perform pixel-level segmentation. The results are compared with other image labeling state of the art (SOTA) methods, such as global convolutional network (GCN) and ViT. Extensive experiments show that our Swin Transformer (SwinTF) with decoder designs reached a new state of the art on the Thailand Isan Landsat-8 corpus (89.8% F1 score), Thailand North Landsat-8 corpus (63.12% F1 score), and competitive results on ISPRS Vaihingen. Moreover, both our best-proposed methods (SwinTF-PSP and SwinTF-FPN) even outperformed SwinTF with supervised pre-training ViT on the ImageNet-1K in the Thailand, Landsat-8, and ISPRS Vaihingen corpora.

Comparative Study of Neural Network-Based Approaches for QRS Segmentation

2020 ◽  
Vol 11 (4) ◽  
pp. 80-103
Author(s):  
George Kolokolnikov ◽  
Anna Borde ◽  
Victor Skuratov ◽  
Roman Gaponov ◽  
Anastasiya Rumyantseva
Keyword(s):  
Deep Learning ◽  
Signal Analysis ◽  
State Of The Art ◽  
Learning Approach ◽  
Review Section ◽  
Advantages And Disadvantages ◽  
Holter Monitor ◽  
Life Threatening ◽  
Deep Learning Model ◽  
Segmentation Problem

The paper is devoted to the development of QRS segmentation system based on deep learning approach. The considered segmentation problem plays an important role in the automatic analysis of heart rhythms, which makes it possible to identify life-threatening pathologies. The main goal of the research is to choose the best segmentation pipeline in terms of accuracy and time-efficiency. Process of ECG-signal analysis is described, and the problem of QRS segmentation is discussed. State-of-the-art algorithms are analyzed in literature review section and the most prominent are chosen for further research. In the course of the research, four hypotheses about appropriate deep learning model are checked: LSTM-based model, 2-input 1-dimensional CNN model, “signal-to-picture” approach based on 2-dimensional CNN, and the simplest 1-dimensional CNN model. All the architectures are tested, and their advantages and disadvantages are discussed. The proposed ECG segmentation pipeline is developed for Holter monitor software.

Review the state-of-the-art technologies of semantic segmentation based on deep learning

2022 ◽  
Author(s):  
Yujian Mo ◽  
Yan Wu ◽  
Xinneng Yang ◽  
Feilin Liu ◽  
Yujun Liao
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
The State

Quantify pixel-level detection of dam surface crack using deep learning

2022 ◽  
Author(s):  
Bo Chen ◽  
Hua Zhang ◽  
Yonglong Li ◽  
Shuang Wang ◽  
Huaifang Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Surface Crack ◽  
Crack Detection ◽  
State Of The Art ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Quantitative Information ◽  
Cross Entropy ◽  
Detection Methods ◽  
Water Conservancy

Abstract An increasing number of detection methods based on computer vision are applied to detect cracks in water conservancy infrastructure. However, most studies directly use existing feature extraction networks to extract cracks information, which are proposed for open-source datasets. As the cracks distribution and pixel features are different from these data, the extracted cracks information is incomplete. In this paper, a deep learning-based network for dam surface crack detection is proposed, which mainly addresses the semantic segmentation of cracks on the dam surface. Particularly, we design a shallow encoding network to extract features of crack images based on the statistical analysis of cracks. Further, to enhance the relevance of contextual information, we introduce an attention module into the decoding network. During the training, we use the sum of Cross-Entropy and Dice Loss as the loss function to overcome data imbalance. The quantitative information of cracks is extracted by the imaging principle after using morphological algorithms to extract the morphological features of the predicted result. We built a manual annotation dataset containing 1577 images to verify the effectiveness of the proposed method. This method achieves the state-of-the-art performance on our dataset. Specifically, the precision, recall, IoU, F1_measure, and accuracy achieve 90.81%, 81.54%, 75.23%, 85.93%, 99.76%, respectively. And the quantization error of cracks is less than 4%.

scholarly journals Multiorgan Detection: Deep Learning Based Techniques and Research Directions

2019 ◽  
Vol 9 (1) ◽  
pp. 4590-4594
Keyword(s):  
Deep Learning ◽  
Semantic Segmentation ◽  
Diagnostic Systems ◽  
Abdominal Ct ◽  
Learning Methods ◽  
Research Directions ◽  
Detection Techniques ◽  
Clinical Procedures ◽  
Future Work ◽  
Segmentation Problem

Automatic organ segmentation plays an important role in clinical procedures such as planning of radiation therapies and in computer-aided diagnostic systems. Several state-of –art techniques are available for multiorgan segmentation, however deep learning methods are doing exceptionally well and become the methodology of choice to analyze medical images. This intensively carried out work is conducted for deep learning methods applied on various organs in abdominal CT images. Firstly, this paper formulates segmentation, semantic segmentation problem and their methods. Secondly, multiorgan detection techniques based on deep learning along with their contributions, chosen datasets and gaps are discussed. It presents the metrics used to evaluate these methods. Finally, interesting conclusions has been drawn which will add to do future work using deep learning.

Sign in / Sign up

Export Citation Format

Share Document