Ensembling Neural Networks for Digital Pathology Images Classification and Segmentation

Thanks to their capability to learn generalizable descriptors directly from images, deep Convolutional Neural Networks (CNNs) seem the ideal solution to most pattern recognition problems. On the other hand, to learn the image representation, CNNs need huge sets of annotated samples that are unfeasible in many every-day scenarios. This is the case, for example, of Computer-Aided Diagnosis (CAD) systems for digital pathology, where additional challenges are posed by the high variability of the cancerous tissue characteristics. In our experiments, state-of-the-art CNNs trained from scratch on histological images were less accurate and less robust to variability than a traditional machine learning framework, highlighting all the issues of fully training deep networks with limited data from real patients. To solve this problem, we designed and compared three transfer learning frameworks, leveraging CNNs pre-trained on non-medical images. This approach obtained very high accuracy, requiring much less computational resource for the training. Our findings demonstrate that transfer learning is a solution to the automated classification of histological samples and solves the problem of designing accurate and computationally-efficient CAD systems with limited training data.

Download Full-text

Hidden Variables in Deep Learning Digital Pathology and Their Potential to Cause Batch Effects: Prediction Model Study (Preprint)

10.2196/preprints.23436 ◽

2020 ◽

Author(s):

Max Schmitt ◽

Roman Christoph Maron ◽

Achim Hekler ◽

Albrecht Stenzinger ◽

Axel Hauschild ◽

...

Keyword(s):

Neural Networks ◽

Lower Bound ◽

Confidence Interval ◽

Hidden Variables ◽

Digital Pathology ◽

Classification Systems ◽

Data Sets ◽

Batch Effects ◽

Data Set ◽

Slide Preparation

BACKGROUND An increasing number of studies within digital pathology show the potential of artificial intelligence (AI) to diagnose cancer using histological whole slide images, which requires large and diverse data sets. While diversification may result in more generalizable AI-based systems, it can also introduce hidden variables. If neural networks are able to distinguish/learn hidden variables, these variables can introduce batch effects that compromise the accuracy of classification systems. OBJECTIVE The objective of the study was to analyze the learnability of an exemplary selection of hidden variables (patient age, slide preparation date, slide origin, and scanner type) that are commonly found in whole slide image data sets in digital pathology and could create batch effects. METHODS We trained four separate convolutional neural networks (CNNs) to learn four variables using a data set of digitized whole slide melanoma images from five different institutes. For robustness, each CNN training and evaluation run was repeated multiple times, and a variable was only considered learnable if the lower bound of the 95% confidence interval of its mean balanced accuracy was above 50.0%. RESULTS A mean balanced accuracy above 50.0% was achieved for all four tasks, even when considering the lower bound of the 95% confidence interval. Performance between tasks showed wide variation, ranging from 56.1% (slide preparation date) to 100% (slide origin). CONCLUSIONS Because all of the analyzed hidden variables are learnable, they have the potential to create batch effects in dermatopathology data sets, which negatively affect AI-based classification systems. Practitioners should be aware of these and similar pitfalls when developing and evaluating such systems and address these and potentially other batch effect variables in their data sets through sufficient data set stratification.

Download Full-text

Fragile neural networks: the importance of image standardization for deep learning in digital pathology

Medical Imaging 2019: Digital Pathology ◽

10.1117/12.2512992 ◽

2019 ◽

Cited By ~ 1

Author(s):

Jonathan Folmsbee ◽

Starr Johnson ◽

Xulei Liu ◽

Margaret Brandwein-Weber ◽

Scott Doyle

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Digital Pathology

Download Full-text

The Histopathological Diagnosis of Adenocarcinoma & Squamous Cells Carcinoma of Lungs by Artificial intelligence: A comparative study of convolutional neural networks

10.1101/2020.05.02.20044602 ◽

2020 ◽

Author(s):

Mohammad Ali Abbas ◽

Syed Usama Khalid Bukhari ◽

Asmara Syed ◽

Syed Sajid Hussain Shah

Keyword(s):

Lung Cancer ◽

Neural Networks ◽

Squamous Cell Carcinoma ◽

Cell Carcinoma ◽

Convolutional Neural Networks ◽

Squamous Cell ◽

Digital Pathology ◽

Histopathological Diagnosis ◽

Data Set ◽

Diagnosis Of Lung Cancer

AbstractIntroductionMalignant tumors of the lung are the most important cause of morbidity and mortality due to cancer all over the world. A rising trend in the incidence of lung cancer has been observed. Histopathological diagnosis of lung cancer is a vital component of patient care. The use of artificial intelligence in the histopathological diagnosis of lung cancer may be a very useful technology in the near future.AimThe aim of the present research project is to determine the effectiveness of convolutional neural networks for the diagnosis of squamous cell carcinoma and adenocarcinoma of the lung by evaluating the digital pathology images of these cancers.Materials & MethodsA total of 15000 digital images of histopathological slides were acquired from the LC2500 dataset. The digital pathology images from lungs are comprised of three classes; class I contains 5000 images of benign lung tissue, class II contains 5,000 images of squamous cell carcinoma of lungs while Class III contains 5,000 images of adenocarcinoma of lungs. Six state of the art off the shelf convolutional neural network architectures, VGG-19, Alex Net, ResNet: ResNet-18, ResNet-34, ResNet-50, and ResNet-101, are used to assess the data, in this comparison study. The dataset was divided into a train set, 55% of the entire data, validation set 20%, and 25% into the test data set.ResultsA number of off the shelf pre-trained (on ImageNet data set) convolutional neural networks are used to classify the histopathological slides into three classes, benign lung tissue, squamous cell carcinoma-lung and adenocarcinoma - lung. The F-1 scores of AlexNet, VGG-19, ResNet-18, ResNet-34, ResNet-50 and ResNet-101, on the test dataset show the result of 0.973, 0.997, 0.986, 0.992, 0.999 and 0.999 respectively.DiscussionThe diagnostic accuracy of more 97% has been achieved for the diagnosis of squamous cell carcinoma and adenocarcinoma of the lungs in the present study. A similar finding has been reported in other studies for the diagnosis of metastasis of breast carcinoma in lymph nodes, basal cell carcinoma, and prostatic cancer.ConclusionThe development of algorithms for the recognition of a specific pattern of the particular malignant tumor by analyzing the digital images will reduce the chance of human errors and improve the efficiency of the laboratory for the rapid and accurate diagnosis of cancer.

Download Full-text

Hidden Variables in Deep Learning Digital Pathology and Their Potential to Cause Batch Effects: Prediction Model Study

Journal of Medical Internet Research ◽

10.2196/23436 ◽

2021 ◽

Vol 23 (2) ◽

pp. e23436

Author(s):

Max Schmitt ◽

Roman Christoph Maron ◽

Achim Hekler ◽

Albrecht Stenzinger ◽

Axel Hauschild ◽

...

Keyword(s):

Neural Networks ◽

Lower Bound ◽

Confidence Interval ◽

Hidden Variables ◽

Digital Pathology ◽

Classification Systems ◽

Data Sets ◽

Batch Effects ◽

Data Set ◽

Slide Preparation

Background An increasing number of studies within digital pathology show the potential of artificial intelligence (AI) to diagnose cancer using histological whole slide images, which requires large and diverse data sets. While diversification may result in more generalizable AI-based systems, it can also introduce hidden variables. If neural networks are able to distinguish/learn hidden variables, these variables can introduce batch effects that compromise the accuracy of classification systems. Objective The objective of the study was to analyze the learnability of an exemplary selection of hidden variables (patient age, slide preparation date, slide origin, and scanner type) that are commonly found in whole slide image data sets in digital pathology and could create batch effects. Methods We trained four separate convolutional neural networks (CNNs) to learn four variables using a data set of digitized whole slide melanoma images from five different institutes. For robustness, each CNN training and evaluation run was repeated multiple times, and a variable was only considered learnable if the lower bound of the 95% confidence interval of its mean balanced accuracy was above 50.0%. Results A mean balanced accuracy above 50.0% was achieved for all four tasks, even when considering the lower bound of the 95% confidence interval. Performance between tasks showed wide variation, ranging from 56.1% (slide preparation date) to 100% (slide origin). Conclusions Because all of the analyzed hidden variables are learnable, they have the potential to create batch effects in dermatopathology data sets, which negatively affect AI-based classification systems. Practitioners should be aware of these and similar pitfalls when developing and evaluating such systems and address these and potentially other batch effect variables in their data sets through sufficient data set stratification.

Download Full-text

Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images

10.1101/197517 ◽

2017 ◽

Cited By ~ 3

Author(s):

Pegah Khosravi ◽

Ehsan Kazemi ◽

Marcin Imielinski ◽

Olivier Elemento ◽

Iman Hajirasouliha

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Human Error ◽

Digital Pathology ◽

Automated Image Analysis ◽

Learning Approaches ◽

Breast Cancers ◽

Deep Convolutional Neural Networks ◽

Cancer Subtypes ◽

Pathological Evaluation

Pathological evaluation of tumor tissue is pivotal for diagnosis in cancer patients and automated image analysis approaches have great potential to increase precision of diagnosis and help reduce human error. In this study, we utilize various computational methods based on convolutional neural networks (CNN) and build a stand-alone pipeline to effectively classify different histopathology images across different types of cancer. In particular, we demonstrate the utility of our pipeline to discriminate between two subtypes of lung cancer, four biomarkers of bladder cancer, and five biomarkers of breast cancer. In addition, we apply our pipeline to discriminate among four immunohistochemistry (IHC) staining scores of bladder and breast cancers. Our classification pipeline utilizes a basic architecture of CNN, Google's Inceptions within three training strategies, and an ensemble of two state-of-the-art algorithms, Inception and ResNet. These strategies include training the last layer of Google's Inceptions, training the network from scratch, and fine-tunning the parameters for our data using two pre-trained version of Google's Inception architectures, Inception-V1 and Inception-V3. We demonstrate the power of deep learning approaches for identifying cancer subtypes, and the robustness of Google's Inceptions even in presence of extensive tumor heterogeneity. Our pipeline on average achieved accuracies of 100% , 92%, 95%, and 69% for discrimination of various cancer types, subtypes, biomarkers, and scores, respectively. Our pipeline and related documentation is freely available at https://github.com/ih-lab/CNN_Smoothie

Download Full-text