Experimental Assessment of Color Deconvolution and Color Normalization for Automated Classification of Histology Images Stained with Hematoxylin and Eosin

Histological evaluation plays a major role in cancer diagnosis and treatment. The appearance of H&E-stained images can vary significantly as a consequence of differences in several factors, such as reagents, staining conditions, preparation procedure and image acquisition system. Such potential sources of noise can all have negative effects on computer-assisted classification. To minimize such artefacts and their potentially negative effects several color pre-processing methods have been proposed in the literature—for instance, color augmentation, color constancy, color deconvolution and color transfer. Still, little work has been done to investigate the efficacy of these methods on a quantitative basis. In this paper, we evaluated the effects of color constancy, deconvolution and transfer on automated classification of H&E-stained images representing different types of cancers—specifically breast, prostate, colorectal cancer and malignant lymphoma. Our results indicate that in most cases color pre-processing does not improve the classification accuracy, especially when coupled with color-based image descriptors. Some pre-processing methods, however, can be beneficial when used with some texture-based methods like Gabor filters and Local Binary Patterns.

Detection of Ki67 Hot-Spots of Invasive Breast Cancer Based on Convolutional Neural Networks Applied to Mutual Information of H&E and Ki67 Whole Slide Images

Ki67 hot-spot detection and its evaluation in invasive breast cancer regions play a significant role in routine medical practice. The quantification of cellular proliferation assessed by Ki67 immunohistochemistry is an established prognostic and predictive biomarker that determines the choice of therapeutic protocols. In this paper, we present three deep learning-based approaches to automatically detect and quantify Ki67 hot-spot areas by means of the Ki67 labeling index. To this end, a dataset composed of 100 whole slide images (WSIs) belonging to 50 breast cancer cases (Ki67 and H&E WSI pairs) was used. Three methods based on CNN classification were proposed and compared to create the tumor proliferation map. The best results were obtained by applying the CNN to the mutual information acquired from the color deconvolution of both the Ki67 marker and the H&E WSIs. The overall accuracy of this approach was 95%. The agreement between the automatic Ki67 scoring and the manual analysis is promising with a Spearman’s ρ correlation of 0.92. The results illustrate the suitability of this CNN-based approach for detecting hot-spots areas of invasive breast cancer in WSI.

NIMG-23. COMPARISON OF A RADIO-PATHOMIC MODEL VERSUS A RADIOLOGY-ONLY TUMOR SEGMENTATION MODEL FOR THE DETECTION OF INFILTRATIVE TUMOR IN GLIOMA PATIENTS

This study used large format autopsy tissue samples to compare radio-pathomic maps of brain cancer to a current tumor segmentation algorithm. We hypothesized that an MRI-based machine learning model trained with actual histology rather than radiologist annotations cellularity would 1) improve delineation between tumor and treatment effect, and 2) detect abnormal pathology beyond the contrast-enhancing tumor region. Seventeen patients with pathologically confirmed glioma were included in this study. At autopsy, 43 tissue samples were collected from 17 subjects from whole brain slices sectioned to align with the last axial MRI prior to death. Cellularity was calculated using a color deconvolution on 40X digitized H&E stained slides from the tissue samples. In-house custom software was used to align tissue samples and cellularity information to the FLAIR image using manually defined control points. The DeepMedic algorithm was trained to segment tumors using the BraTs 2017 dataset, and then applied to our patients in order to create automated tumor probability maps. An MRI-based ensemble algorithm using a 5x5 voxel searchlight (input: T1, T1C, FLAIR, ADC) was used to predict cellularity at each voxel, using tissue samples from 14 subjects as ground truth. Both models were applied to 3 withheld test subjects in order to compare tumor probability and cellularity predictions to the pathological ground truth. The mutual information between tumor probability and actual cellularity was 1X10-15, relatively low compared to the rad-path predicted cellularity (=0.16), despite the tumor prediction model accurately highlighting regions of contrast enhancement. Additionally, the radio-pathomic ensemble model correctly identify regions of hypercellularity beyond the tumor segmentation model as well as regions of within the segmented tumor area. This study demonstrates the utility of training machine learning models with pathological ground truth rather than radiologist annotations for predicting localized tumor information, particularly in the post-treatment stage.

INNV-10. EFFECTS OF TTFIELDS USAGE AND DURATION OF USAGE ON CELLULARITY AND Ki67 DISTRIBUTIONS AT AUTOPSY

This study used autopsy tissue samples taken from patients with glioblastoma who had undergone Tumor-Treating Fields (TTFields) therapy to examine the effects of treatment usage and duration on cellular and mitotic activity distributions. We hypothesized that treatment duration and percent compliance would be associated with a change in cellularity. Three tissue samples were collected from 10 patients with glioblastoma at autopsy. These samples were specifically collected in regions that demonstrated clear contrast enhancement on the patients’ last clinical imaging, in order to capture regions of suspected active tumor. Samples were stained with both hematoxylin and eosin (HE) and Ki67 immuno-histochemistry (IHC) and digitized at 40X resolution using a digital microscope. A color deconvolution algorithm was used to separate stains, which allowed nuclei segmentation. Number of cells and percent of Ki67 positive cells were computed across 100x100 superpixels. Spearman’s correlations were used to examine the association between TTFields usage (measured as a percent) and duration of use (in days) and signatures of active tumor proliferation, measured as median cellularity and proportion of Ki67 positive cells across each patients’ three samples. Trending negative associations with median cellularity were observed for both TTFields usage (R=-0.60, p=0.065) and duration (R=-0.61,p=0.063). The magnitude of the treatment duration effect increased when controlling for time between diagnosis and death, despite a p-value drop due to the lost statistical degree of freedom (R=-0.62, p=0.073). No significant effects were observed for Ki67 positive staining. These results generally suggest a possible effect of TTFields therapy duration, where longer treatments and increased compliance lead to lower tumor proliferation. However, these results should be interpreted with caution, as larger, more statistically powerful studies will be better suited to assess the generalizability of the preliminary trends seen in these RESULTS:

An Automated Quantitative Method to Analyze Immunohistochemistry and Immunocytochemistry Images

Immunohistochemistry (IHC) plays an important role in target protein analysis. However, many researchers analyze IHC images by five/three-tier manual ranking methods based on stained area and density. Such manual scoring might be biased by the antibody amount, counterstaining density, overall brightness, and most importantly, researchers' ranking experience. The potential lack of reliability in manual approach drives us to develop an automatic tool to quantitatively analyze IHC, which can also be used for immunocytochemistry (ICC). We applied a “color deconvolution” method based on an red-green-blue (RGB) color vector matching the color of desired immunochemistry agent, 3,3′-diaminobenzidine (DAB) with haematoxylin in this case, to acquire pseudo-color images. Subsequently, Density, the product of integrating the single pixel staining density by area stained, is used as an index of immunostaining. We observed a strong correlation between the results by our automatic method and the manual scoring from experienced researchers, demonstrating the utility of this method in IHC and ICC. For IHC analysis, five-tier ranking based on density (n = 161) shows a high Spearman's coefficient (rho) of 0.80 (P < 0.0001) with the annotation given by two experienced scientists. However, the rho between experienced and inexperienced researchers' annotation (n = 154) is only 0.66 (P < 0.0001). In immunocytochemistry, the rho between density and experienced researchers' annotation is 0.80 (n = 44, P < 0.0001). In conclusion, our method can rank multiple protein targets in immunohistochemistry and may be also used in immunochemistry.

Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

Background Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of uniquely colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel. Methods Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and (3) ensemble methods that employ both ColorAE and U-Net, collectively referred to as ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor). Results We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect six different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net in ensemble methods outperform ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). Summary We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also utilized the ColorAE:U-Net ensemble method to analyze 3 mIHC WSIs with nearest neighbor spatial analysis. We demonstrate a proof of concept that these methods can be employed to quantitatively describe the spatial distribution of immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.

Analysis of resorbable mesh implants in short-term human muscular fascia cultures: a pilot study

Purpose Alteration in fascial tissue collagen composition represents a key factor in hernia etiology and recurrence. Both resorbable and non-resorbable meshes for hernia repair are currently used in the surgical setting. However, no study has investigated so far the role of different implant materials on collagen deposition and tissue remodeling in human fascia. The aim of the present study was to develop a novel ex vivo model of human soft tissue repair mesh implant, and to test its suitability to investigate the effects of different materials on tissue remodeling and collagen composition. Methods Resorbable poly-4-hydroxybutyrate and non-resorbable polypropylene mesh implants were embedded in human abdominal fascia samples, mimicking common surgical procedures. Calcein-AM/Propidium Iodide vital staining was used to assess tissue vitality. Tissue morphology was evaluated using Mallory trichrome and hematoxylin and eosin staining. Collagen type I and III expression was determined through immunostaining semi-quantification by color deconvolution. All analyses were performed after 54 days of culture. Results The established ex vivo model showed good viability at 54 days of culture, confirming both culture method feasibility and implants biocompatibility. Both mesh implants induced a disorganization of collagen fibers pattern. A statistically significantly higher collagen I/III ratio was detected in fascial tissue samples cultured with resorbable implants compared to either non-resorbable implants or meshes-free controls. Conclusion We developed a novel ex vivo model and provided evidence that resorbable polyhydroxybutyrate meshes display better biomechanical properties suitable for proper restoration in surgical hernia repair.