Transferability of Deep Learning Models for Focus Quality Assessment in Digital Pathology

Abstract Out-of-focus sections of whole slide images are a significant source of false positives and other systematic errors in clinical diagnoses. As a result, focus quality assessment (FQA) methods must be able to quickly and accurately differentiate between focus levels in a scan. Recently, deep learning methods using convolutional neural networks (CNNs) have been adopted for FQA. However, the biggest obstacles impeding their wide usage in clinical workflows are their generalizability across different test conditions and their potentially high computational cost. In this study, we focus on the transferability and scalability of CNN-based FQA approaches. We carry out an investigation on ten architecturally diverse networks using five datasets with stain and tissue diversity. We evaluate the computational complexity of each network and scale this to realistic applications involving hundreds of whole slide images. We assess how well each full model transfers to a separate, unseen dataset without fine-tuning. We show that shallower networks transfer well when used on small input patch sizes, while deeper networks work more effectively on larger inputs. Furthermore, we introduce neural architecture search (NAS) to the field and learn an automatically designed low-complexity CNN architecture using differentiable architecture search which achieved competitive performance relative to established CNNs.

Download Full-text

Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification

Journal of Imaging ◽

10.3390/jimaging7030051 ◽

2021 ◽

Vol 7 (3) ◽

pp. 51

Author(s):

Emanuela Paladini ◽

Edoardo Vantaggiato ◽

Fares Bougourzi ◽

Cosimo Distante ◽

Abdenour Hadid ◽

...

Keyword(s):

Colorectal Cancer ◽

Deep Learning ◽

Digital Pathology ◽

Texture Features ◽

Tissue Type ◽

Cancer Tissue ◽

Learning Methods ◽

Feature Based ◽

Type Classification ◽

Whole Slide Images

In recent years, automatic tissue phenotyping has attracted increasing interest in the Digital Pathology (DP) field. For Colorectal Cancer (CRC), tissue phenotyping can diagnose the cancer and differentiate between different cancer grades. The development of Whole Slide Images (WSIs) has provided the required data for creating automatic tissue phenotyping systems. In this paper, we study different hand-crafted feature-based and deep learning methods using two popular multi-classes CRC-tissue-type databases: Kather-CRC-2016 and CRC-TP. For the hand-crafted features, we use two texture descriptors (LPQ and BSIF) and their combination. In addition, two classifiers are used (SVM and NN) to classify the texture features into distinct CRC tissue types. For the deep learning methods, we evaluate four Convolutional Neural Network (CNN) architectures (ResNet-101, ResNeXt-50, Inception-v3, and DenseNet-161). Moreover, we propose two Ensemble CNN approaches: Mean-Ensemble-CNN and NN-Ensemble-CNN. The experimental results show that the proposed approaches outperformed the hand-crafted feature-based methods, CNN architectures and the state-of-the-art methods in both databases.

Download Full-text

A deep-learning-based prognostic nomogram integrating microscopic digital pathology and macroscopic magnetic resonance images in nasopharyngeal carcinoma: a multi-cohort study

Therapeutic Advances in Medical Oncology ◽

10.1177/1758835920971416 ◽

2020 ◽

Vol 12 ◽

pp. 175883592097141

Author(s):

Fan Zhang ◽

Lian-Zhen Zhong ◽

Xun Zhao ◽

Di Dong ◽

Ji-Jin Yao ◽

...

Keyword(s):

Deep Learning ◽

Magnetic Resonance ◽

Nasopharyngeal Carcinoma ◽

Treatment Failure ◽

Digital Pathology ◽

Imaging Features ◽

Multi Scale ◽

External Test ◽

Whole Slide Images ◽

Independent Cohort

Background: To explore the prognostic value of radiomics-based and digital pathology-based imaging biomarkers from macroscopic magnetic resonance imaging (MRI) and microscopic whole-slide images for patients with nasopharyngeal carcinoma (NPC). Methods: We recruited 220 NPC patients and divided them into training ( n = 132), internal test ( n = 44), and external test ( n = 44) cohorts. The primary endpoint was failure-free survival (FFS). Radiomic features were extracted from pretreatment MRI and selected and integrated into a radiomic signature. The histopathological signature was extracted from whole-slide images of biopsy specimens using an end-to-end deep-learning method. Incorporating two signatures and independent clinical factors, a multi-scale nomogram was constructed. We also tested the correlation between the key imaging features and genetic alternations in an independent cohort of 16 patients (biological test cohort). Results: Both radiomic and histopathologic signatures presented significant associations with treatment failure in the three cohorts (C-index: 0.689–0.779, all p < 0.050). The multi-scale nomogram showed a consistent significant improvement for predicting treatment failure compared with the clinical model in the training (C-index: 0.817 versus 0.730, p < 0.050), internal test (C-index: 0.828 versus 0.602, p < 0.050) and external test (C-index: 0.834 versus 0.679, p < 0.050) cohorts. Furthermore, patients were stratified successfully into two groups with distinguishable prognosis (log-rank p < 0.0010) using our nomogram. We also found that two texture features were related to the genetic alternations of chromatin remodeling pathways in another independent cohort. Conclusion: The multi-scale imaging features showed a complementary value in prognostic prediction and may improve individualized treatment in NPC.

Download Full-text

CP-NAS: Child-Parent Neural Architecture Search for 1-bit CNNs

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/144 ◽

2020 ◽

Author(s):

Li'an Zhuo ◽

Baochang Zhang ◽

Hanlin Chen ◽

Linlin Yang ◽

Chen Chen ◽

...

Keyword(s):

Computational Cost ◽

Unified Framework ◽

Embedded Devices ◽

Natural Approach ◽

Neural Architecture ◽

Resource Limited ◽

Training Stage ◽

Comparable Accuracy ◽

Kernel Level ◽

High Computational Cost

Neural architecture search (NAS) proves to be among the best approaches for many tasks by generating an application-adaptive neural architectures, which are still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binarized weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a uniﬁed framework. To this end, a Child-Parent model is introduced to a differentiable NAS to search the binarized architecture(Child) under the supervision of a full-precision model (Parent). In the search stage, the Child-Parent model uses an indicator generated by the parent and child model accuracy to evaluate the performance and abandon operations with less potential. In the training stage, a kernel level CP loss is introduced to optimize the binarized network. Extensive experiments demonstrate that the proposed CP-NAS achieves a comparable accuracy with traditional NAS on both the CIFAR and ImageNet databases. It achieves an accuracy of 95.27% on CIFAR-10, 64.3% on ImageNet with binarized weights and activations, and a 30% faster search than prior arts.

Download Full-text

Iterative machine learning applied to annotation of text datasets

10.5753/eniac.2021.18268 ◽

2021 ◽

Author(s):

Thiago Abdo ◽

Fabiano Silva

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Learning Algorithms ◽

Computational Cost ◽

Machine Learning Techniques ◽

Learning Approaches ◽

Learning Techniques ◽

The Creation ◽

The Impact ◽

High Computational Cost

The purpose of this paper is to analyze the use of different machine learning approaches and algorithms to be integrated as an automated assistance on a tool to aid the creation of new annotated datasets. We evaluate how they scale in an environment without dedicated machine learning hardware. In particular, we study the impact over a dataset with few examples and one that is being constructed. We experiment using deep learning algorithms (Bert) and classical learning algorithms with a lower computational cost (W2V and Glove combined with RF and SVM). Our experiments show that deep learning algorithms have a performance advantage over classical techniques. However, deep learning algorithms have a high computational cost, making them inadequate to an environment with reduced hardware resources. Simulations using Active and Iterative machine learning techniques to assist the creation of new datasets are conducted. For these simulations, we use the classical learning algorithms because of their computational cost. The knowledge gathered with our experimental evaluation aims to support the creation of a tool for building new text datasets.

Download Full-text

Transfer Learning for Inference of Metastatic Origin from Whole Slide Histology

10.1101/2021.04.21.440864 ◽

2021 ◽

Author(s):

Geoffrey F. Schau ◽

Hassan Ghani ◽

Erik A. Burlingame ◽

Guillaume Thibault ◽

Joe W. Gray ◽

...

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Primary Tumor ◽

Large Scale ◽

Metastatic Cancer ◽

Clinical Diagnostics ◽

Training Data ◽

Fine Tuning ◽

Learning Approach ◽

Whole Slide Images

AbstractAccurate diagnosis of metastatic cancer is essential for prescribing optimal control strategies to halt further spread of metastasizing disease. While pathological inspection aided by immunohistochemistry staining provides a valuable gold standard for clinical diagnostics, deep learning methods have emerged as powerful tools for identifying clinically relevant features of whole slide histology relevant to a tumor’s metastatic origin. Although deep learning models require significant training data to learn effectively, transfer learning paradigms provide mechanisms to circumvent limited training data by first training a model on related data prior to fine-tuning on smaller data sets of interest. In this work we propose a transfer learning approach that trains a convolutional neural network to infer the metastatic origin of tumor tissue from whole slide images of hematoxylin and eosin (H&E) stained tissue sections and illustrate the advantages of pre-training network on whole slide images of primary tumor morphology. We further characterize statistical dissimilarity between primary and metastatic tumors of various indications on patch-level images to highlight limitations of our indication-specific transfer learning approach. Using a primary-to-metastatic transfer learning approach, we achieved mean class-specific areas under receiver operator characteristics curve (AUROC) of 0.779, which outperformed comparable models trained on only images of primary tumor (mean AUROC of 0.691) or trained on only images of metastatic tumor (mean AUROC of 0.675), supporting the use of large scale primary tumor imaging data in developing computer vision models to characterize metastatic origin of tumor lesions.

Download Full-text

Challenges Developing Deep Learning Algorithms in Cytology

Acta Cytologica ◽

10.1159/000510991 ◽

2020 ◽

pp. 1-9

Author(s):

Ewen David McAlpine ◽

Liron Pantanowitz ◽

Pamela M. Michelow

Keyword(s):

Deep Learning ◽

Ethical Issues ◽

Digital Pathology ◽

Algorithm Development ◽

Regulatory Frameworks ◽

Skilled Personnel ◽

And Storage ◽

Computer Resources ◽

Whole Slide Images ◽

Broad Understanding

Background: The incorporation of digital pathology into routine pathology practice is becoming more widespread. Definite advantages exist with respect to the implementation of artificial intelligence (AI) and deep learning in pathology, including cytopathology. However, there are also unique challenges in this regard. Summary: This review discusses cytology-specific challenges, including the need to implement digital cytology prior to AI; the large file sizes and increased acquisition times for whole slide images in cytology; the routine use of multiple stains, such as Papanicolaou and Romanowsky stains; the lack of high-quality annotated datasets on which to train algorithms; and the considerable computer resources required, in terms of both computer infrastructure and skilled personnel, for computing and storage of data. Global concerns regarding AI that are certainly applicable to cytology include the need for model validation and continued quality assurance, ethical issues such as the use of patient data in developing algorithms, the need to develop regulatory frameworks regarding what type of data can be utilized and ensuring cybersecurity during data collection and storage, and algorithm development. Key Messages: While AI will likely play a role in cytology practice in the future, applying this technology to cytology poses a unique set of challenges. A broad understanding of digital pathology and algorithm development is desirable to guide the development of algorithms, as well as the need to be cognizant of potential pitfalls to avoid when incorporating the technology in practice.

Download Full-text

Developing image analysis pipelines of whole-slide images: Pre- and post-processing

Journal of Clinical and Translational Science ◽

10.1017/cts.2020.531 ◽

2020 ◽

pp. 1-11

Author(s):

Byron Smith ◽

Meyke Hermsen ◽

Elizabeth Lesser ◽

Deepak Ravichandar ◽

Walter Kremers

Keyword(s):

Image Analysis ◽

Deep Learning ◽

Digital Pathology ◽

Disruptive Technology ◽

Post Processing ◽

Artifact Detection ◽

The Press ◽

Slide Image ◽

Processing Order ◽

Whole Slide Images

Abstract Deep learning has pushed the scope of digital pathology beyond simple digitization and telemedicine. The incorporation of these algorithms in routine workflow is on the horizon and maybe a disruptive technology, reducing processing time, and increasing detection of anomalies. While the newest computational methods enjoy much of the press, incorporating deep learning into standard laboratory workflow requires many more steps than simply training and testing a model. Image analysis using deep learning methods often requires substantial pre- and post-processing order to improve interpretation and prediction. Similar to any data processing pipeline, images must be prepared for modeling and the resultant predictions need further processing for interpretation. Examples include artifact detection, color normalization, image subsampling or tiling, removal of errant predictions, etc. Once processed, predictions are complicated by image file size – typically several gigabytes when unpacked. This forces images to be tiled, meaning that a series of subsamples from the whole-slide image (WSI) are used in modeling. Herein, we review many of these methods as they pertain to the analysis of biopsy slides and discuss the multitude of unique issues that are part of the analysis of very large images.

Download Full-text

A Comparison of Parametric and Sample-Based Message Representation in Cooperative Localization

International Journal of Navigation and Observation ◽

10.1155/2012/281592 ◽

2012 ◽

Vol 2012 ◽

pp. 1-10 ◽

Cited By ~ 45

Author(s):

Jaime Lien ◽

Ulric J. Ferner ◽

Warakorn Srichavengsup ◽

Henk Wymeersch ◽

Moe Z. Win

Keyword(s):

Message Passing ◽

Ad Hoc ◽

Computational Cost ◽

Low Complexity ◽

Ultra Wideband ◽

Cooperative Localization ◽

Location Awareness ◽

Excellent Performance ◽

Hoc Networks ◽

High Computational Cost

Location awareness is a key enabling feature and fundamental challenge in present and future wireless networks. Most existing localization methods rely on existing infrastructure and thus lack the flexibility and robustness necessary for large ad hoc networks. In this paper, we build upon SPAWN (sum-product algorithm over a wireless network), which determines node locations through iterative message passing, but does so at a high computational cost. We compare different message representations for SPAWN in terms of performance and complexity and investigate several types of cooperation based on censoring. Our results, based on experimental data with ultra-wideband (UWB) nodes, indicate that parametric message representation combined with simple censoring can give excellent performance at relatively low complexity.

Download Full-text

Video Based Person Re-Identification Through Selective Knowledge Distillation

International Journal of Scientific Research in Computer Science Engineering and Information Technology ◽

10.32628/cseit1952179 ◽

2019 ◽

pp. 699-703

Author(s):

Gudavalli Sai Abhilash ◽

Kantheti Rajesh ◽

Jangam Dileep Shaleem ◽

Grandi Sai Sarath ◽

Palli R Krishna Prasad

Keyword(s):

Computational Cost ◽

Low Complexity ◽

Fine Tuning ◽

Low Resolution ◽

Facial Cues ◽

Significant Performance ◽

Knowledge Distillation ◽

Computational Resources ◽

The Cost ◽

Low Computational Cost

The creation and deployment of face recognition models need to identify low-resolution faces with extremely low computational cost. To address this problem, a feasible solution is compressing a complex face model to achieve higher speed and lower memory at the cost of minimal performance drop. Inspired by that, this paper proposes a learning approach to recognize low-resolution faces via selective knowledge distillation in live video. In this approach, a two-stream convolution neural network (CNN) is first initialized to recognize high-resolution faces and resolution-degraded faces with a teacher stream and a student stream, respectively. The teacher stream is represented by a complex CNN for high-accuracy recognition, and the student stream is represented by a much simpler CNN for low-complexity recognition. To avoid significant performance drop at the student stream, we then selectively distil the most informative facial features from the teacher stream by solving a sparse graph optimization problem, which are then used to regularize the fine- tuning process of the student stream. In this way, the student stream is actually trained by simultaneously handling two tasks with limited computational resources approximating the most informative facial cues via feature regression, and recovering the missing facial cues via low-resolution face classification.

Download Full-text

Artificial Intelligence and Digital Microscopy Applications in Diagnostic Hematopathology

Cancers ◽

10.3390/cancers12040797 ◽

2020 ◽

Vol 12 (4) ◽

pp. 797 ◽

Cited By ~ 2

Author(s):

Hanadi El El Achi ◽

Joseph D. Khoury

Keyword(s):

Artificial Intelligence ◽

Deep Learning ◽

Digital Images ◽

Digital Pathology ◽

Digital Microscopy ◽

Clinical Environment ◽

Lymphoproliferative Diseases ◽

Glass Slides ◽

Pathology Data ◽

Whole Slide Images

Digital Pathology is the process of converting histology glass slides to digital images using sophisticated computerized technology to facilitate acquisition, evaluation, storage, and portability of histologic information. By its nature, digitization of analog histology data renders it amenable to analysis using deep learning/artificial intelligence (DL/AI) techniques. The application of DL/AI to digital pathology data holds promise, even if the scope of use cases and regulatory framework for deploying such applications in the clinical environment remains in the early stages. Recent studies using whole-slide images and DL/AI to detect histologic abnormalities in general and cancer in particular have shown encouraging results. In this review, we focus on these emerging technologies intended for use in diagnostic hematology and the evaluation of lymphoproliferative diseases.

Download Full-text