scholarly journals Mask then classify: multi-instance segmentation for surgical instruments

2021 ◽  
Author(s):  
Thomas Kurmann ◽  
Pablo Márquez-Neila ◽  
Max Allan ◽  
Sebastian Wolf ◽  
Raphael Sznitman
Keyword(s):  
Semantic Segmentation ◽  
Surgical Instruments ◽  
Segmentation Method ◽  
Surgical Robots ◽  
Minimally Invasive Surgical ◽  
Level Information ◽  
Previous State ◽  
Respective Type ◽  
Ablation Study ◽  
Instance Segmentation

Abstract Purpose The detection and segmentation of surgical instruments has been a vital step for many applications in minimally invasive surgical robotics. Previously, the problem was tackled from a semantic segmentation perspective, yet these methods fail to provide good segmentation maps of instrument types and do not contain any information on the instance affiliation of each pixel. We propose to overcome this limitation by using a novel instance segmentation method which first masks instruments and then classifies them into their respective type. Methods We introduce a novel method for instance segmentation where a pixel-wise mask of each instance is found prior to classification. An encoder–decoder network is used to extract instrument instances, which are then separately classified using the features of the previous stages. Furthermore, we present a method to incorporate instrument priors from surgical robots. Results Experiments are performed on the robotic instrument segmentation dataset of the 2017 endoscopic vision challenge. We perform a fourfold cross-validation and show an improvement of over 18% to the previous state-of-the-art. Furthermore, we perform an ablation study which highlights the importance of certain design choices and observe an increase of 10% over semantic segmentation methods. Conclusions We have presented a novel instance segmentation method for surgical instruments which outperforms previous semantic segmentation-based methods. Our method further provides a more informative output of instance level information, while retaining a precise segmentation mask. Finally, we have shown that robotic instrument priors can be used to further increase the performance.

scholarly journals Real-Time Instance Segmentation of Traffic Videos for Embedded Devices

Sensors ◽  
2021 ◽  
Vol 21 (1) ◽  
pp. 275
Author(s):  
Ruben Panero Martinez ◽  
Ionut Schiopu ◽  
Bruno Cornelis ◽  
Adrian Munteanu
Keyword(s):  
Real Time ◽  
Network Architecture ◽  
Training Procedure ◽  
Segmentation Method ◽  
Embedded Devices ◽  
Network Training ◽  
Assignment Algorithm ◽  
Ablation Study ◽  
Reduced Rate ◽  
Instance Segmentation

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.

AI with robotics in surgery extreme environments

2021 ◽  
pp. 1-10
Author(s):  
Wenji Li ◽  
Lihong Xie ◽  
C.B. Sivaparthipan ◽  
C. Chandru Vignesh
Keyword(s):  
Robotic Surgery ◽  
Human Error ◽  
Simulation Analysis ◽  
Extreme Environments ◽  
Semantic Segmentation ◽  
Robotic System ◽  
Surgical Instruments ◽  
Surgical Robots ◽  
And Control ◽  
Surgical System

Robotic surgery offers surgeons a greater degree of accuracy, versatility, and control than with standard techniques for other kinds of complicated procedures. The robotic surgery technology offers numerous advantages for patients and leads to unforeseen effects that are easier to predict when such a complex interactive device is used for treatment. The challenging complications that are occurred during robotic surgery include, risk of human error while operating the robotic system and the possibility for mechanical failure. The paper proposes Robot Assisted - Remote Center Surgical System (RA-RCSS) to improve mechanical malfunction threat and practical skills of surgeons through intra practice feedback and demonstration from human experts. A mask region-based supervised learning model is trained to conduct semantic segmentation of surgical instruments and targets to improve surgical coordinates further and to facilitate self-oriented practice. Furthermore, the master-slave bilateral technique is integrated with RA-RCSS to analyze the mechanical failures and malfunctions of the robotic system. The emerging safety standard environment is presented as a key enabling factor in the commercialization of autonomous surgical robots. The simulation analysis is performed based on accuracy, security, performance, and cost factor proves the reliability of the proposed framework.

An adaptive coarse-fine semantic segmentation method for the attachment recognition on marine current turbines

2021 ◽  
Vol 93 ◽  
pp. 107182
Author(s):  
Haiyang Peng ◽  
Dingding Yang ◽  
Tianzhen Wang ◽  
Shreya Pandey ◽  
Lisu Chen ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Segmentation Method ◽  
Marine Current

scholarly journals InstantDL: an easy-to-use deep learning pipeline for image segmentation and classification

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Dominik Jens Elias Waibel ◽  
Sayedali Shetab Boushehri ◽  
Carsten Marr
Keyword(s):  
Image Processing ◽  
Deep Learning ◽  
Specific Problem ◽  
State Of The Art ◽  
Image Data ◽  
Semantic Segmentation ◽  
Parameter Tuning ◽  
Cellular Processes ◽  
Minimal Effort ◽  
Instance Segmentation

Abstract Background Deep learning contributes to uncovering molecular and cellular processes with highly performant algorithms. Convolutional neural networks have become the state-of-the-art tool to provide accurate and fast image data processing. However, published algorithms mostly solve only one specific problem and they typically require a considerable coding effort and machine learning background for their application. Results We have thus developed InstantDL, a deep learning pipeline for four common image processing tasks: semantic segmentation, instance segmentation, pixel-wise regression and classification. InstantDL enables researchers with a basic computational background to apply debugged and benchmarked state-of-the-art deep learning algorithms to their own data with minimal effort. To make the pipeline robust, we have automated and standardized workflows and extensively tested it in different scenarios. Moreover, it allows assessing the uncertainty of predictions. We have benchmarked InstantDL on seven publicly available datasets achieving competitive performance without any parameter tuning. For customization of the pipeline to specific tasks, all code is easily accessible and well documented. Conclusions With InstantDL, we hope to empower biomedical researchers to conduct reproducible image processing with a convenient and easy-to-use pipeline.

scholarly journals A semantic segmentation method for exposed rebar on dam concrete based on Unet

2020 ◽  
Vol 1651 ◽  
pp. 012169
Author(s):  
Jun Zhang ◽  
Jian Mo ◽  
Huajun Xu ◽  
Zixing Liu
Keyword(s):  
Semantic Segmentation ◽  
Segmentation Method ◽  
Dam Concrete

scholarly journals An Efficient Module for Instance Segmentation Based on Multi-Level Features and Attention Mechanisms

2021 ◽  
Vol 11 (3) ◽  
pp. 968
Author(s):  
Yingchun Sun ◽  
Wang Gao ◽  
Shuguo Pan ◽  
Tao Zhao ◽  
Yahui Peng
Keyword(s):  
Feature Extraction ◽  
Spatial Structure ◽  
Semantic Feature ◽  
Semantic Features ◽  
Segmentation Method ◽  
Spatial Dimensions ◽  
Feature Pyramid ◽  
Multi Level ◽  
High Level ◽  
Instance Segmentation

Recently, multi-level feature networks have been extensively used in instance segmentation. However, because not all features are beneficial to instance segmentation tasks, the performance of networks cannot be adequately improved by synthesizing multi-level convolutional features indiscriminately. In order to solve the problem, an attention-based feature pyramid module (AFPM) is proposed, which integrates the attention mechanism on the basis of a multi-level feature pyramid network to efficiently and pertinently extract the high-level semantic features and low-level spatial structure features; for instance, segmentation. Firstly, we adopt a convolutional block attention module (CBAM) into feature extraction, and sequentially generate attention maps which focus on instance-related features along the channel and spatial dimensions. Secondly, we build inter-dimensional dependencies through a convolutional triplet attention module (CTAM) in lateral attention connections, which is used to propagate a helpful semantic feature map and filter redundant informative features irrelevant to instance objects. Finally, we construct branches for feature enhancement to strengthen detailed information to boost the entire feature hierarchy of the network. The experimental results on the Cityscapes dataset manifest that the proposed module outperforms other excellent methods under different evaluation metrics and effectively upgrades the performance of the instance segmentation method.

Sign in / Sign up

Export Citation Format

Share Document