Semantic segmentation of reflectance confocal microscopy mosaics of pigmented lesions using weak labels

AbstractReflectance confocal microscopy (RCM) is a non-invasive imaging tool that reduces the need for invasive histopathology for skin cancer diagnoses by providing high-resolution mosaics showing the architectural patterns of skin, which are used to identify malignancies in-vivo. RCM mosaics are similar to dermatopathology sections, both requiring extensive training to interpret. However, these modalities differ in orientation, as RCM mosaics are horizontal (parallel to the skin surface) while histopathology sections are vertical, and contrast mechanism, RCM with a single (reflectance) mechanism resulting in grayscale images and histopathology with multi-factor color-stained contrast. Image analysis and machine learning methods can potentially provide a diagnostic aid to clinicians to interpret RCM mosaics, eventually helping to ease the adoption and more efficiently utilizing RCM in routine clinical practice. However standard supervised machine learning may require a prohibitive volume of hand-labeled training data. In this paper, we present a weakly supervised machine learning model to perform semantic segmentation of architectural patterns encountered in RCM mosaics. Unlike more widely used fully supervised segmentation models that require pixel-level annotations, which are very labor-demanding and error-prone to obtain, here we focus on training models using only patch-level labels (e.g. a single field of view within an entire mosaic). We segment RCM mosaics into “benign” and “aspecific (nonspecific)” regions, where aspecific regions represent the loss of regular architecture due to injury and/or inflammation, pre-malignancy, or malignancy. We adopt Efficientnet, a deep neural network (DNN) proven to accurately accomplish classification tasks, to generate class activation maps, and use a Gaussian weighting kernel to stitch smaller images back into larger fields of view. The trained DNN achieved an average area under the curve of 0.969, and Dice coefficient of 0.778 showing the feasibility of spatial localization of aspecific regions in RCM images, and making the diagnostics decision model more interpretable to the clinicians.

Download Full-text

Semi-Automated Semantic Segmentation of Arctic Shorelines Using Very High-Resolution Airborne Imagery, Spectral Indices and Weakly Supervised Machine Learning Approaches

Remote Sensing ◽

10.3390/rs13224572 ◽

2021 ◽

Vol 13 (22) ◽

pp. 4572

Author(s):

Bibek Aryal ◽

Stephen M. Escarzaga ◽

Sergio A. Vargas Vargas Zesati ◽

Miguel Velez-Reyes ◽

Olac Fuentes ◽

...

Keyword(s):

Machine Learning ◽

Semantic Segmentation ◽

Training Data ◽

Supervised Machine Learning ◽

The Arctic ◽

Ecosystem Structure ◽

Learning Approaches ◽

Water Index ◽

Shoreline Mapping ◽

And Function

Precise coastal shoreline mapping is essential for monitoring changes in erosion rates, surface hydrology, and ecosystem structure and function. Monitoring water bodies in the Arctic National Wildlife Refuge (ANWR) is of high importance, especially considering the potential for oil and natural gas exploration in the region. In this work, we propose a modified variant of the Deep Neural Network based U-Net Architecture for the automated mapping of 4 Band Orthorectified NOAA Airborne Imagery using sparsely labeled training data and compare it to the performance of traditional Machine Learning (ML) based approaches—namely, random forest, xgboost—and spectral water indices—Normalized Difference Water Index (NDWI), and Normalized Difference Surface Water Index (NDSWI)—to support shoreline mapping of Arctic coastlines. We conclude that it is possible to modify the U-Net model to accept sparse labels as input and the results are comparable to other ML methods (an Intersection-over-Union (IoU) of 94.86% using U-Net vs. an IoU of 95.05% using the best performing method).

Download Full-text

A machine learning method for identifying morphological patterns in reflectance confocal microscopy mosaics of melanocytic skin lesions in-vivo

10.1117/12.2212978 ◽

2016 ◽

Cited By ~ 2

Author(s):

Kivanc Kose ◽

Christi Alessi-Fox ◽

Melissa Gill ◽

Jennifer G. Dy ◽

Dana H. Brooks ◽

...

Keyword(s):

Machine Learning ◽

Confocal Microscopy ◽

Skin Lesions ◽

Reflectance Confocal Microscopy ◽

Machine Learning Method ◽

Learning Method ◽

Morphological Patterns

Download Full-text

Artificial Intelligence-Based Approaches to Reflectance Confocal Microscopy Image Analysis in Dermatology

Journal of Clinical Medicine ◽

10.3390/jcm11020429 ◽

2022 ◽

Vol 11 (2) ◽

pp. 429

Author(s):

Ana Maria Malciu ◽

Mihai Lupu ◽

Vlad Mihai Voiculescu

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Image Analysis ◽

Confocal Microscopy ◽

Image Quality Assessment ◽

Skin Diseases ◽

Learning Technologies ◽

Reflectance Confocal Microscopy ◽

Imaging Method

Reflectance confocal microscopy (RCM) is a non-invasive imaging method designed to identify various skin diseases. Confocal based diagnosis may be subjective due to the learning curve of the method, the scarcity of training programs available for RCM, and the lack of clearly defined diagnostic criteria for all skin conditions. Given that in vivo RCM is becoming more widely used in dermatology, numerous deep learning technologies have been developed in recent years to provide a more objective approach to RCM image analysis. Machine learning-based algorithms are used in RCM image quality assessment to reduce the number of artifacts the operator has to view, shorten evaluation times, and decrease the number of patient visits to the clinic. However, the current visual method for identifying the dermal-epidermal junction (DEJ) in RCM images is subjective, and there is a lot of variation. The delineation of DEJ on RCM images could be automated through artificial intelligence, saving time and assisting novice RCM users in studying the key DEJ morphological structure. The purpose of this paper is to supply a current summary of machine learning and artificial intelligence’s impact on the quality control of RCM images, key morphological structures identification, and detection of different skin lesion types on static RCM images.

Download Full-text

An unsupervised machine learning method for delineating stratum corneum in reflectance confocal microscopy stacks of human skin in vivo

10.1117/12.2213036 ◽

2016 ◽

Author(s):

Alican Bozkurt ◽

Kivanc Kose ◽

Christi A. Fox ◽

Jennifer Dy ◽

Dana H. Brooks ◽

...

Keyword(s):

Machine Learning ◽

Confocal Microscopy ◽

Stratum Corneum ◽

Human Skin ◽

Reflectance Confocal Microscopy ◽

Machine Learning Method ◽

Learning Method ◽

Unsupervised Machine Learning

Download Full-text

Usefulness of in vivo reflectance confocal microscopy in Morbihan syndrome

Skin Research and Technology ◽

10.1111/srt.13002 ◽

2021 ◽

Author(s):

Arianna Rizzo ◽

Diletta Fiorani ◽

Laura Lazzeri ◽

Paolo Taddeucci ◽

Pietro Rubegni ◽

...

Keyword(s):

Confocal Microscopy ◽

Reflectance Confocal Microscopy

Download Full-text

Supervised Machine Learning Methods and Hyperspectral Imaging Techniques Jointly Applied for Brain Cancer Classification

Sensors ◽

10.3390/s21113827 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3827

Author(s):

Gemma Urbanos ◽

Alberto Martín ◽

Guillermo Vázquez ◽

Marta Villanueva ◽

Manuel Villa ◽

...

Keyword(s):

Machine Learning ◽

Blood Vessel ◽

Hyperspectral Imaging ◽

Imaging Techniques ◽

Venous Blood ◽

Healthy Tissue ◽

Supervised Machine Learning ◽

Support Vector ◽

Arterial Blood

Hyperspectral imaging techniques (HSI) do not require contact with patients and are non-ionizing as well as non-invasive. As a consequence, they have been extensively applied in the medical field. HSI is being combined with machine learning (ML) processes to obtain models to assist in diagnosis. In particular, the combination of these techniques has proven to be a reliable aid in the differentiation of healthy and tumor tissue during brain tumor surgery. ML algorithms such as support vector machine (SVM), random forest (RF) and convolutional neural networks (CNN) are used to make predictions and provide in-vivo visualizations that may assist neurosurgeons in being more precise, hence reducing damages to healthy tissue. In this work, thirteen in-vivo hyperspectral images from twelve different patients with high-grade gliomas (grade III and IV) have been selected to train SVM, RF and CNN classifiers. Five different classes have been defined during the experiments: healthy tissue, tumor, venous blood vessel, arterial blood vessel and dura mater. Overall accuracy (OACC) results vary from 60% to 95% depending on the training conditions. Finally, as far as the contribution of each band to the OACC is concerned, the results obtained in this work are 3.81 times greater than those reported in the literature.

Download Full-text

In vivo reflectance confocal microscopy for the diagnosis of scabies

BMJ Case Reports ◽

10.1136/bcr-2020-240507 ◽

2021 ◽

Vol 14 (1) ◽

pp. e240507

Author(s):

Mihai Lupu ◽

Vlad Mihai Voiculescu ◽

Cristina Vajaitu ◽

Olguta Anca Orzan

Keyword(s):

Confocal Microscopy ◽

Reflectance Confocal Microscopy

Download Full-text

Effects of Training Set Size on Supervised Machine-Learning Land-Cover Classification of Large-Area High-Resolution Remotely Sensed Data

Remote Sensing ◽

10.3390/rs13030368 ◽

2021 ◽

Vol 13 (3) ◽

pp. 368

Author(s):

Christopher A. Ramezan ◽

Timothy A. Warner ◽

Aaron E. Maxwell ◽

Bradley S. Price

Keyword(s):

Machine Learning ◽

Sample Size ◽

Remotely Sensed ◽

Training Data ◽

Supervised Machine Learning ◽

Sample Sizes ◽

Remotely Sensed Data ◽

Large Area ◽

Training Set ◽

Set Size

The size of the training data set is a major determinant of classification accuracy. Nevertheless, the collection of a large training data set for supervised classifiers can be a challenge, especially for studies covering a large area, which may be typical of many real-world applied projects. This work investigates how variations in training set size, ranging from a large sample size (n = 10,000) to a very small sample size (n = 40), affect the performance of six supervised machine-learning algorithms applied to classify large-area high-spatial-resolution (HR) (1–5 m) remotely sensed data within the context of a geographic object-based image analysis (GEOBIA) approach. GEOBIA, in which adjacent similar pixels are grouped into image-objects that form the unit of the classification, offers the potential benefit of allowing multiple additional variables, such as measures of object geometry and texture, thus increasing the dimensionality of the classification input data. The six supervised machine-learning algorithms are support vector machines (SVM), random forests (RF), k-nearest neighbors (k-NN), single-layer perceptron neural networks (NEU), learning vector quantization (LVQ), and gradient-boosted trees (GBM). RF, the algorithm with the highest overall accuracy, was notable for its negligible decrease in overall accuracy, 1.0%, when training sample size decreased from 10,000 to 315 samples. GBM provided similar overall accuracy to RF; however, the algorithm was very expensive in terms of training time and computational resources, especially with large training sets. In contrast to RF and GBM, NEU, and SVM were particularly sensitive to decreasing sample size, with NEU classifications generally producing overall accuracies that were on average slightly higher than SVM classifications for larger sample sizes, but lower than SVM for the smallest sample sizes. NEU however required a longer processing time. The k-NN classifier saw less of a drop in overall accuracy than NEU and SVM as training set size decreased; however, the overall accuracies of k-NN were typically less than RF, NEU, and SVM classifiers. LVQ generally had the lowest overall accuracy of all six methods, but was relatively insensitive to sample size, down to the smallest sample sizes. Overall, due to its relatively high accuracy with small training sample sets, and minimal variations in overall accuracy between very large and small sample sets, as well as relatively short processing time, RF was a good classifier for large-area land-cover classifications of HR remotely sensed data, especially when training data are scarce. However, as performance of different supervised classifiers varies in response to training set size, investigating multiple classification algorithms is recommended to achieve optimal accuracy for a project.

Download Full-text