Adaptive kernel selection network with attention constraint for surgical instrument classification

AbstractComputer vision (CV) technologies are assisting the health care industry in many respects, i.e., disease diagnosis. However, as a pivotal procedure before and after surgery, the inventory work of surgical instruments has not been researched with the CV-powered technologies. To reduce the risk and hazard of surgical tools’ loss, we propose a study of systematic surgical instrument classification and introduce a novel attention-based deep neural network called SKA-ResNet which is mainly composed of: (a) A feature extractor with selective kernel attention module to automatically adjust the receptive fields of neurons and enhance the learnt expression and (b) A multi-scale regularizer with KL-divergence as the constraint to exploit the relationships between feature maps. Our method is easily trained end-to-end in only one stage with few additional calculation burdens. Moreover, to facilitate our study, we create a new surgical instrument dataset called SID19 (with 19 kinds of surgical tools consisting of 3800 images) for the first time. Experimental results show the superiority of SKA-ResNet for the classification of surgical tools on SID19 when compared with state-of-the-art models. The classification accuracy of our method reaches up to 97.703%, which is well supportive for the inventory and recognition study of surgical tools. Also, our method can achieve state-of-the-art performance on four challenging fine-grained visual classification datasets.

Download Full-text

The Effectiveness of Hot Bead Sterilization in Maintaining Sterile Surgical Instrument Tips Across Sequential Mouse Surgeries

Journal of the American Association for Laboratory Animal Science ◽

10.30802/aalas-jaalas-21-000047 ◽

2021 ◽

Author(s):

Julie A Holdridge ◽

Madison S Nichols ◽

William D Dupont ◽

Carissa P Jones ◽

Katherine A Shuster

Keyword(s):

Risk Assessment ◽

Gastrointestinal Tract ◽

Bacterial Contamination ◽

Surgical Instruments ◽

Surgical Instrument ◽

Experimental Conditions ◽

Before And After ◽

In Series ◽

Surgical Field ◽

The Common

One strategy commonly employed for rodent surgeries is a “tips-only” surgical technique, which restricts the surgeon to using only the sterile working ends of the surgical instruments to manipulate the surgical field and sterilizes instrument tipswith a hot bead sterilizer between consecutive rodents. Despite the common use of the “tips-only” technique, research is lacking on the number of sequential surgeries for which the same set of hot bead-sterilized instruments can be used before introducing bacterial contamination. We performed serial mouse surgeries using the “tips-only” technique under 3 different conditions (aseptic, fur contamination, or cecal contamination) and assessed aerobic bacterial growth before and after each round of hot bead sterilization. Instrument tips showed an increasing probability of contamination of at least one instrument in a series of consecutive surgeries. The probability that all surgical instrument tips in the series were sterile after hotbead sterilization fell by 4% for each surgery involving inadvertent or fur contamination and by 11.5% for each surgery with contamination for all surgical types combined (including entering the gastrointestinal tract). Based on our results, hot bead sterilization is not adequate for surgeries associated with gross contamination. Under our experimental conditions and assuming independence of outcomes between consecutive surgeries, up to 5 surgeries associated with minor or inadvertent contamination could be performed in series with a probability higher than 80% that all instrument tips were sterile for allsurgeries. A case-by-case risk assessment should be conducted to derive institutional guidelines for the maximal number ofsurgeries that can be performed in sequence using the “tips-only” technique with hot bead sterilization of the same set ofsurgical instruments between surgeries. Full sterilization of instruments after every surgery provides the greatest confidencein maintaining sterility.

Download Full-text

A Novel Multi-Scale Attention PFE-UNet for Forest Image Segmentation

Forests ◽

10.3390/f12070937 ◽

2021 ◽

Vol 12 (7) ◽

pp. 937

Author(s):

Boyang Zhang ◽

Hongbo Mu ◽

Mingyu Gao ◽

Haiming Ni ◽

Jianfeng Chen ◽

...

Keyword(s):

Feature Extraction ◽

Image Segmentation ◽

State Of The Art ◽

Computational Cost ◽

Receptive Fields ◽

Feature Maps ◽

Multi Scale ◽

Extensive Evaluation ◽

Network Transition ◽

Segmentation Task

The precise segmentation of forest areas is essential for monitoring tasks related to forest exploration, extraction, and statistics. However, the effective and accurate segmentation of forest images will be affected by factors such as blurring and discontinuity of forest boundaries. Therefore, a Pyramid Feature Extraction-UNet network (PFE-UNet) based on traditional UNet is proposed to be applied to end-to-end forest image segmentation. Among them, the Pyramid Feature Extraction module (PFE) is introduced in the network transition layer, which obtains multi-scale forest image information through different receptive fields. The spatial attention module (SA) and the channel-wise attention module (CA) are applied to low-level feature maps and PFE feature maps, respectively, to highlight specific segmentation task features while fusing context information and suppressing irrelevant regions. The standard convolution block is replaced by a novel depthwise separable convolutional unit (DSC Unit), which not only reduces the computational cost but also prevents overfitting. This paper presents an extensive evaluation with the DeepGlobe dataset and a comparative analysis with several state-of-the-art networks. The experimental results show that the PFE-UNet network obtains an accuracy of 94.23% in handling the real-time forest image segmentation, which is significantly higher than other advanced networks. This means that the proposed PFE-UNet also provides a valuable reference for the precise segmentation of forest images.

Download Full-text

BARNet: Bilinear Attention Network with Adaptive Receptive Fields for Surgical Instrument Segmentation

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/116 ◽

2020 ◽

Cited By ~ 1

Author(s):

Zhen-Liang Ni ◽

Gui-Bin Bian ◽

Guan-An Wang ◽

Xiao-Hu Zhou ◽

Zeng-Guang Hou ◽

...

Keyword(s):

Receptive Field ◽

Receptive Fields ◽

Computer Assisted Surgery ◽

Computer Assisted ◽

Semantic Features ◽

Surgical Instrument ◽

Feature Maps ◽

Illumination Variation ◽

Attention Network ◽

Scale Variation

Surgical instrument segmentation is crucial for computer-assisted surgery. Different from common object segmentation, it is more challenging due to the large illumination variation and scale variation in the surgical scenes. In this paper, we propose a bilinear attention network with adaptive receptive fields to address these two issues. To deal with the illumination variation, the bilinear attention module models global contexts and semantic dependencies between pixels by capturing second-order statistics. With them, semantic features in challenging areas can be inferred from their neighbors, and the distinction of various semantics can be boosted. To adapt to the scale variation, our adaptive receptive field module aggregates multi-scale features and selects receptive fields adaptively. Specifically, it models the semantic relationships between channels to choose feature maps with appropriate scales, changing the receptive field of subsequent convolutions. The proposed network achieves the best performance 97.47% mean IoU on Cata7. It also takes the first place on EndoVis 2017, exceeding the second place by 10.10% mean IoU.

Download Full-text

Learning Visual Words for Weakly-Supervised Semantic Segmentation

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/136 ◽

2021 ◽

Author(s):

Lixiang Ru ◽

Bo Du ◽

Chen Wu

Keyword(s):

State Of The Art ◽

Local Maximum ◽

Semantic Segmentation ◽

Input Image ◽

Feature Maps ◽

Visual Words ◽

Fine Grained ◽

Spatial Pyramid Pooling ◽

Weakly Supervised ◽

Spatial Pyramid

Current weakly-supervised semantic segmentation (WSSS) methods with image-level labels mainly adopt class activation maps (CAM) to generate the initial pseudo labels. However, CAM usually only identifies the most discriminative object extents, which is attributed to the fact that the network doesn't need to discover the integral object to recognize image-level labels. In this work, to tackle this problem, we proposed to simultaneously learn the image-level labels and local visual word labels. Specifically, in each forward propagation, the feature maps of the input image will be encoded to visual words with a learnable codebook. By enforcing the network to classify the encoded fine-grained visual words, the generated CAM could cover more semantic regions. Besides, we also proposed a hybrid spatial pyramid pooling module that could preserve local maximum and global average values of feature maps, so that more object details and less background were considered. Based on the proposed methods, we conducted experiments on the PASCAL VOC 2012 dataset. Our proposed method achieved 67.2% mIoU on the val set and 67.3% mIoU on the test set, which outperformed recent state-of-the-art methods.

Download Full-text

Improve YOLOv3 using dilated spatial pyramid module for multi-scale object detection

International Journal of Advanced Robotic Systems ◽

10.1177/1729881420936062 ◽

2020 ◽

Vol 17 (4) ◽

pp. 172988142093606

Author(s):

Xiaoguo Zhang ◽

Ye Gao ◽

Huiqing Wang ◽

Qing Wang

Keyword(s):

Object Detection ◽

State Of The Art ◽

Receptive Fields ◽

Mean Average Precision ◽

Feature Maps ◽

Average Precision ◽

Multi Scale ◽

Art Object ◽

Spatial Pyramid ◽

Scale Variation

Effectively and efficiently recognizing multi-scale objects is one of the key challenges of utilizing deep convolutional neural network to the object detection field. YOLOv3 (You only look once v3) is the state-of-the-art object detector with good performance in both aspects of accuracy and speed; however, the scale variation is still the challenging problem which needs to be improved. Considering that the detection performances of multi-scale objects are related to the receptive fields of the network, in this work, we propose a novel dilated spatial pyramid module to integrate multi-scale information to effectively deal with scale variation problem. Firstly, the input of dilated spatial pyramid is fed into multiple parallel branches with different dilation rates to generate feature maps with different receptive fields. Then, the input of dilated spatial pyramid and outputs of different branches are concatenated to integrate multi-scale information. Moreover, dilated spatial pyramid is integrated with YOLOv3 in front of the first detection header to present dilated spatial pyramid-You only look once model. Experiment results on PASCAL VOC2007 demonstrate that dilated spatial pyramid-You only look once model outperforms other state-of-the-art methods in mean average precision, while it still keeps a satisfying real-time detection speed. For 416 × 416 input, dilated spatial pyramid-You only look once model achieves 82.2% mean average precision at 56 frames per second, 3.9% higher than YOLOv3 with only slight speed drops.

Download Full-text

Behavioral Genetics: Concepts for Research and Practice in Language Development and Disorders

Journal of Speech Language and Hearing Research ◽

10.1044/jshr.3805.1126 ◽

1995 ◽

Vol 38 (5) ◽

pp. 1126-1142 ◽

Cited By ~ 14

Author(s):

Jeffrey W. Gilger

Keyword(s):

Language Development ◽

Behavioral Genetics ◽

State Of The Art ◽

Genetic Research ◽

Great Promise ◽

Behavioral Genetic ◽

Fine Grained ◽

Future Goals ◽

Current State ◽

Research Designs

This paper is an introduction to behavioral genetics for researchers and practioners in language development and disorders. The specific aims are to illustrate some essential concepts and to show how behavioral genetic research can be applied to the language sciences. Past genetic research on language-related traits has tended to focus on simple etiology (i.e., the heritability or familiality of language skills). The current state of the art, however, suggests that great promise lies in addressing more complex questions through behavioral genetic paradigms. In terms of future goals it is suggested that: (a) more behavioral genetic work of all types should be done—including replications and expansions of preliminary studies already in print; (b) work should focus on fine-grained, theory-based phenotypes with research designs that can address complex questions in language development; and (c) work in this area should utilize a variety of samples and methods (e.g., twin and family samples, heritability and segregation analyses, linkage and association tests, etc.).

Download Full-text

Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition

Sensors ◽

10.3390/s21124233 ◽

2021 ◽

Vol 21 (12) ◽

pp. 4233

Author(s):

Bogdan Mocanu ◽

Ruxandra Tapu ◽

Titus Zaharia

Keyword(s):

Emotion Recognition ◽

Loss Function ◽

State Of The Art ◽

Disease Diagnosis ◽

Data Representation ◽

Speech Emotion Recognition ◽

Audio Features ◽

Global Accuracy ◽

Space Data ◽

Art Techniques

Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%.

Download Full-text

Representation Learning for Fine-Grained Change Detection

Sensors ◽

10.3390/s21134486 ◽

2021 ◽

Vol 21 (13) ◽

pp. 4486

Author(s):

Niall O’Mahony ◽

Sean Campbell ◽

Lenka Krpalkova ◽

Anderson Carvalho ◽

Joseph Walsh ◽

...

Keyword(s):

Deep Learning ◽

Change Detection ◽

Model Calibration ◽

State Of The Art ◽

Representation Learning ◽

Machine Intelligence ◽

The State ◽

Sensor Data ◽

Fine Grained ◽

Learning Techniques

Fine-grained change detection in sensor data is very challenging for artificial intelligence though it is critically important in practice. It is the process of identifying differences in the state of an object or phenomenon where the differences are class-specific and are difficult to generalise. As a result, many recent technologies that leverage big data and deep learning struggle with this task. This review focuses on the state-of-the-art methods, applications, and challenges of representation learning for fine-grained change detection. Our research focuses on methods of harnessing the latent metric space of representation learning techniques as an interim output for hybrid human-machine intelligence. We review methods for transforming and projecting embedding space such that significant changes can be communicated more effectively and a more comprehensive interpretation of underlying relationships in sensor data is facilitated. We conduct this research in our work towards developing a method for aligning the axes of latent embedding space with meaningful real-world metrics so that the reasoning behind the detection of change in relation to past observations may be revealed and adjusted. This is an important topic in many fields concerned with producing more meaningful and explainable outputs from deep learning and also for providing means for knowledge injection and model calibration in order to maintain user confidence.

Download Full-text

Non-Local and Multi-Scale Mechanisms for Image Inpainting

Sensors ◽

10.3390/s21093281 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3281

Author(s):

Xu He ◽

Yong Yin

Keyword(s):

Markov Random Fields ◽

Receptive Fields ◽

Image Inpainting ◽

Long Distance ◽

Visual Appearance ◽

Fine Grained ◽

Multi Scale ◽

Non Local ◽

Relationship Of

Recently, deep learning-based techniques have shown great power in image inpainting especially dealing with squared holes. However, they fail to generate plausible results inside the missing regions for irregular and large holes as there is a lack of understanding between missing regions and existing counterparts. To overcome this limitation, we combine two non-local mechanisms including a contextual attention module (CAM) and an implicit diversified Markov random fields (ID-MRF) loss with a multi-scale architecture which uses several dense fusion blocks (DFB) based on the dense combination of dilated convolution to guide the generative network to restore discontinuous and continuous large masked areas. To prevent color discrepancies and grid-like artifacts, we apply the ID-MRF loss to improve the visual appearance by comparing similarities of long-distance feature patches. To further capture the long-term relationship of different regions in large missing regions, we introduce the CAM. Although CAM has the ability to create plausible results via reconstructing refined features, it depends on initial predicted results. Hence, we employ the DFB to obtain larger and more effective receptive fields, which benefits to predict more precise and fine-grained information for CAM. Extensive experiments on two widely-used datasets demonstrate that our proposed framework significantly outperforms the state-of-the-art approaches both in quantity and quality.

Download Full-text

ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition

International Journal of Computer Vision ◽

10.1007/s11263-021-01477-5 ◽

2021 ◽

Author(s):

Anil S. Baslamisli ◽

Partha Das ◽

Hoang-An Le ◽

Sezer Karaoglu ◽

Theo Gevers

Keyword(s):

Neural Network ◽

Large Scale ◽

State Of The Art ◽

Image Decomposition ◽

Natural Environments ◽

Decomposition Algorithms ◽

Ambient Light ◽

Fine Grained ◽

Large Scale Dataset ◽

Direct Illumination

AbstractIn general, intrinsic image decomposition algorithms interpret shading as one unified component including all photometric effects. As shading transitions are generally smoother than reflectance (albedo) changes, these methods may fail in distinguishing strong photometric effects from reflectance variations. Therefore, in this paper, we propose to decompose the shading component into direct (illumination) and indirect shading (ambient light and shadows) subcomponents. The aim is to distinguish strong photometric effects from reflectance variations. An end-to-end deep convolutional neural network (ShadingNet) is proposed that operates in a fine-to-coarse manner with a specialized fusion and refinement unit exploiting the fine-grained shading model. It is designed to learn specific reflectance cues separated from specific photometric effects to analyze the disentanglement capability. A large-scale dataset of scene-level synthetic images of outdoor natural environments is provided with fine-grained intrinsic image ground-truths. Large scale experiments show that our approach using fine-grained shading decompositions outperforms state-of-the-art algorithms utilizing unified shading on NED, MPI Sintel, GTA V, IIW, MIT Intrinsic Images, 3DRMS and SRD datasets.

Download Full-text