A deep learning model for breast ductal carcinoma in situ classification in whole slide images

The pathological differential diagnosis between breast ductal carcinoma in situ (DCIS) and invasive ductal carcinoma (IDC) is of pivotal importance for determining optimum cancer treatment(s) and clinical outcomes. Since conventional diagnosis by pathologists using microscopes is limited in terms of human resources, it is necessary to develop new techniques that can rapidly and accurately diagnose large numbers of histopathological specimens. Computational pathology tools which can assist pathologists in detecting and classifying DCIS and IDC from whole slide images (WSIs) would be of great benefit for routine pathological diagnosis. In this paper, we trained deep learning models capable of classifying biopsy and surgical histopathological WSIs into DCIS, IDC, and benign. We evaluated the models on two independent test sets (n=1,382, n=548), achieving ROC areas under the curves (AUCs) up to 0.960 and 0.977 for DCIS and IDC, respectively.

TIAToolbox: An End-to-End Toolbox for Advanced Tissue Image Analytics

Computational Pathology (CPath) has seen rapid growth in recent years, driven by advanced deep learning (DL) algorithms. These algorithms typically share the same sequence of steps. However, due to the sheer size and complexity of handling large multi-gigapixel whole-slide images, there is no open-source software library that provides a generic end-to-end API for pathology image analysis using best practices for CPath. Most researchers have designed custom pipelines from the bottom-up, restricting the development of advanced CPath algorithms to specialist users. To help overcome this bottleneck, we present TIAToolbox, a Python toolbox designed to make CPath more accessible to new and advanced CPath scientists and pathologists alike. We provide a usable and adaptable library with efficient, cutting-edge and unit-tested tools for data loading, pre-processing, model inference, post-processing and visualization. This enables all kinds of users to easily build upon recent DL developments in the CPath literature. TIAToolbox provides a user-friendly modular API to enable seamless integration of advanced DL algorithms. We show with the help of examples how state-of-the-art DL algorithms can be streamlined using TIAToolbox.

The future of artificial intelligence in digital pathology - results of a survey across stakeholder groups

Artificial intelligence (AI) provides a powerful tool to extract information from digitized histopathology whole slide images. In the last five years, academic and commercial actors have developed new technical solutions for a diverse set of tasks, including tissue segmentation, cell detection, mutation prediction, prognostication and prediction of treatment response. In the light of limited overall resources, it is presently unclear for researchers, practitioners and policymakers which of these topics are stable enough for clinical use in the near future and which topics are still experimental, but worth investing time and effort into. To identify potentially promising applications of AI in pathology, we performed an anonymous online survey of 75 computational pathology domain experts from academia and industry. Participants enrolled in 2021 were queried about their subjective opinion on promising and appealing sub-fields of computational pathology with a focus on solid tumors. The results of this survey indicate that the prediction of treatment response directly from routine pathology slides is regarded as the most promising future application. This item was ranked highest in the overall analysis and in sub-groups by age and professional background. Furthermore, prediction of genetic alterations, gene expression and survival directly from routine pathology images scored consistently high across subgroups. Together, these data demonstrate a possible direction for the development of computational pathology systems in clinical, academic and industrial research in the near future.

A Framework for Falsifiable Explanations of Machine Learning Models with an Application in Computational Pathology

In recent years, deep learning has been the key driver of breakthrough developments in computational pathology and other image based approaches that support medical diagnosis and treatment. The underlying neural networks as inherent black boxes lack transparency, and are often accompanied by approaches to explain their output. However, formally defining explainability has been a notorious unsolved riddle. Here, we introduce a hypothesis-based framework for falsifiable explanations of machine learning models. A falsifiable explanation is a hypothesis that connects an intermediate space induced by the model with the sample from which the data originate. We instantiate this framework in a computational pathology setting using label-free infrared microscopy. The intermediate space is an activation map, which is trained with an inductive bias to localize tumor. An explanation is constituted by hypothesizing that activation corresponds to tumor and associated structures, which we validate by histological staining as an independent secondary experiment.

A Hybrid High Performance Intelligent Computing Approach of CACNN and RNN for Skin Cancer Image Grading

Skin cancer is characterized as the uncontrollable growth of skin cells caused by unrepairable DNA damage. Melanoma is the deadliest form of skin cancers caused by melanocyte and early diagnosis supports therapists in curing it. Computational pathology offers a one-of-a-kind ability to spatially dissect certain interfaces on digitized histology images. A hybrid context-aware convolutional neural networks with recurrent neural network (CA-CNN-RNN) based on skin cancer histological images is proposed in this research. The proposed model encodes a histology image's local representation into higher-dimensional features first, then aggregated the feature by consider their spatial arrangement to enable the final predictions. In this research, H&E-stained sectioned images from the Cancer Genome Atlas are used as the dataset for assessment. From 58 images, 37 images were used for training and 21 images are used for testing. The process on histology images of melanoma skin cancer was analyzed and validated with various classifiers such as VGG-19, Inception, ResNet50, and DarkNet-53 using the hybrid CA-CNN-RNN model. The dataset is used to generate the results, which are then analyzed based on criteria such as accuracy, recall, precision, and F-score. The performance analysis shows that the proposed CA-CNN-RNN with different classifiers has performed better and among the classifiers the DarkNet-53 model has the better performance in all the parameters.

Structural Knowledge Transfer of Panoptic Kidney Segmentation to Other Stains, Organs, and Species

Panoptic segmentation networks are a newer class of image segmentation algorithms that are constrained to understand the difference between instance-type objects (objects that are discrete countable entities, such as renal tubules) and group-type objects (uncountable, amorphous regions of texture such as renal interstitium). This class of deep networks has unique advantages for biological datasets, particularly in computational pathology. We collected 126 periodic acid Schiff whole slide images of native diabetic nephropathy, lupus nephritis, and transplant surveillance kidney biopsies, and fully annotated them for the following micro-compartments: interstitium, glomeruli, globally sclerotic glomeruli, tubules, and arterial tree (arteries/arterioles). Using this data, we trained a panoptic feature pyramid network. We compared performance of the network against a renal pathologist's annotations, and the method's transferability to other computational pathology domain tasks was investigated. The panoptic feature pyramid networks showed high performance as compared to renal pathologist for all of the annotated classes in a testing set of transplant kidney biopsies. The network was not only able to generalize its object understanding across different stains and species of kidney data, but also across several organ types. We conclude panoptic networks have unique advantages for computational pathology; namely, these networks internally model structural morphology, which aids bootstrapping of annotations for new computational pathology tasks.

A deep learning model for gastric diffuse-type adenocarcinoma classification in whole slide images

Gastric diffuse-type adenocarcinoma represents a disproportionately high percentage of cases of gastric cancers occurring in the young, and its relative incidence seems to be on the rise. Usually it affects the body of the stomach, and it presents shorter duration and worse prognosis compared with the differentiated (intestinal) type adenocarcinoma. The main difficulty encountered in the differential diagnosis of gastric adenocarcinomas occurs with the diffuse-type. As the cancer cells of diffuse-type adenocarcinoma are often single and inconspicuous in a background desmoplaia and inflammation, it can often be mistaken for a wide variety of non-neoplastic lesions including gastritis or reactive endothelial cells seen in granulation tissue. In this study we trained deep learning models to classify gastric diffuse-type adenocarcinoma from WSIs. We evaluated the models on five test sets obtained from distinct sources, achieving receiver operator curve (ROC) area under the curves (AUCs) in the range of 0.95–0.99. The highly promising results demonstrate the potential of AI-based computational pathology for aiding pathologists in their diagnostic workflow system.