CellSpatialGraph: Integrate hierarchical phenotyping and graph modeling to characterize spatial architecture in tumor microenvironment on digital pathology

Abstract Motivation As a highly heterogeneous disease, the progression of tumor is not only achieved by unlimited growth of the tumor cells, but also supported, stimulated, and nurtured by the microenvironment around it. However, traditional qualitative and/or semi-quantitative parameters obtained by pathologist’s visual examination have very limited capability to capture this interaction between tumor and its microenvironment. With the advent of digital pathology, computerized image analysis may provide a better tumor characterization and give new insights into this problem. Results We propose a novel bioimage informatics pipeline for automatically characterizing the topological organization of different cell patterns in the tumor microenvironment. We apply this pipeline to the only publicly available large histopathology image dataset for a cohort of 190 patients with papillary renal cell carcinoma obtained from The Cancer Genome Atlas project. Experimental results show that the proposed topological features can successfully stratify early- and middle-stage patients with distinct survival, and show superior performance to traditional clinical features and cellular morphological and intensity features. The proposed features not only provide new insights into the topological organizations of cancers, but also can be integrated with genomic data in future studies to develop new integrative biomarkers. Availability and implementation https://github.com/chengjun583/KIRP-topological-features Supplementary information Supplementary data are available atBioinformatics online.

Download Full-text

Deep learning-based tumor microenvironment segmentation is predictive of tumor mutations and patient survival in non-small-cell lung cancer

10.1101/2021.10.09.462574 ◽

2021 ◽

Author(s):

Łukasz Rączkowski ◽

Iwona Paśnik ◽

Michał Kukiełka ◽

Marcin Nicoś ◽

Magdalena A Budzinska ◽

...

Keyword(s):

Lung Cancer ◽

Tumor Microenvironment ◽

Patient Survival ◽

Digital Pathology ◽

Gene Mutations ◽

The Cancer Genome Atlas ◽

Cancer Tissue ◽

Segmented Images ◽

Spatial Composition ◽

Main Determinants

Despite the fact that tumor microenvironment (TME) and gene mutations are the main determinants of progression of the deadliest cancer in the world - lung cancer - their interrelations are not well understood. Digital pathology data provide a unique insight into the spatial composition of the TME. Here, we generated 23,199 image patches from 55 hematoxylin-and-eosin (H&E)-stained lung cancer tissue sections and annotated them into 9 different tissue classes. Using this dataset, we trained a deep neural network and used it to segment 467 lung cancer H&E images downloaded from The Cancer Genome Atlas (TCGA) database. We used the segmented images to compute human interpretable features reflecting the heterogeneous composition of the TME, and successfully utilized them to predict patient survival (c-index 0.723) and cancer gene mutations (largest AUC 73.5% for PDGFRB). Our approach can be generalized to different cancer types to inform precision medicine strategies.

Download Full-text

Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

Diagnostic Pathology ◽

10.1186/s13000-020-01003-0 ◽

2020 ◽

Vol 15 (1) ◽

Cited By ~ 1

Author(s):

Danielle J. Fassler ◽

Shahira Abousamra ◽

Rajarsi Gupta ◽

Chao Chen ◽

Maozheng Zhao ◽

...

Keyword(s):

Image Analysis ◽

Deep Learning ◽

Spatial Analysis ◽

Tumor Microenvironment ◽

Nearest Neighbor ◽

Digital Pathology ◽

Ensemble Methods ◽

Ductal Adenocarcinoma ◽

Learning Methods ◽

Color Deconvolution

Abstract Background Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of uniquely colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel. Methods Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and (3) ensemble methods that employ both ColorAE and U-Net, collectively referred to as ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor). Results We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect six different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net in ensemble methods outperform ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). Summary We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also utilized the ColorAE:U-Net ensemble method to analyze 3 mIHC WSIs with nearest neighbor spatial analysis. We demonstrate a proof of concept that these methods can be employed to quantitatively describe the spatial distribution of immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.

Download Full-text

Highly Multiplexed Single Cell Spatial Analysis of the Tumor Microenvironment in Lymphoma Predicts Clinical Outcomes

Blood ◽

10.1182/blood-2019-130831 ◽

2019 ◽

Vol 134 (Supplement_1) ◽

pp. 2789-2789 ◽

Cited By ~ 1

Author(s):

Monirath Hav ◽

Anthony Colombo ◽

Erik Gerdtsson ◽

Mohan Singh ◽

Denaly Chen ◽

...

Keyword(s):

Overall Survival ◽

T Cells ◽

Spatial Analysis ◽

Tumor Microenvironment ◽

Single Cell ◽

Tumor Cells ◽

Clinical Outcomes ◽

Immune Cells ◽

Spatial Interaction ◽

Spatial Architecture

Background Diffuse large B-cell lymphoma (DLBCL) is the most common subtype of non-Hodgkin lymphoma. Although a majority of patients are cured with standard chemo-immunotherapy, up to 40% of DLBCL patients have refractory disease or develop relapse following R-CHOP regimen, warranting development of novel, more effective therapeutic strategies for this cohort[1]. The composition of immune cells in the tumor microenvironment (TME) and tumor PD-L1 expression have been reported to predict DLBCL outcomes, however PD1 inhibitors demonstrate response rates of less than 10%.[2-5] We hypothesize that a better characterization of spatial architecture of the tumour microenvironment (TME) in lymphoma will help explain why DLBCL has poor responses to PD1 inhibitors and guide future targeted immunotherapies for these patients. Methods Here we characterized the TME in DLBCL using imaging mass cytometry (IMC), which allows high-dimensional, single-cell and spatial analysis of FFPE tissues at sub-cellular resolution [6]. Using a panel of 32 antibodies, IMC was performed 41 tissue microarray cores from 33 DLBCL cases. IMC images were analyzed for relevant immunophenotypes, the spatial architecture of those phenotypes and compared to clinical outcomes to identify immune contexture based biomarkers. Results Phenograph was used to cluster tumor and immune cells based on phenotype. Immune cells represented 33% of the cells broken down to CD4 (36%), CD8 (30%), macrophages (26%) and TREG(8%) (Figure A).Immune cell infiltration in individual tumor samples ranged from 7% to 75% with marked heterogeneity between samples. Analysis of immune marker expression on tumor cells identified co-expression of PD-L1/CCR4/TIM3 to be highly prognostic for overall survival (p=0.003, Figure B-C) To characterize the patterns of spatial interaction in the TME, we developed an unsupervised multivariate model to construct spatial meta-clusters based on average distances from CD8 to the centroids of 5 nearest endothelial cells, TREG, CD4 T cells, macrophages, and tumor cells (Figure D). Spatial analysis revealed 11 meta-clusters for CD8 T cell interactions (Figure E). Each CD8 spatial interaction pattern is distinctive with case to case heterogeneity (Figures F). Risk assessment analyses of spatial clusters 1, 2 and 4 ("hazardous") had almost 3 times higher odds of being identified in refractory cases compared to clusters 3, 5 and 6 ("protective") (Figure G). In the "protective" spatial neighborhoods, we observed the presence of activated CD8, Th1-like CD4, and less suppressive TREGphenotypes, with opposite in "hazardous" areas (Figures H). TIM-3 expression was high both on T cells and tumor cells in the "hazardous" neighborhoods. Finally, we show that sub-setting our analysis of CD8 phenotypes based on their spatial location to other cells improved our ability to predict overall survival in the cohort. Conclusion These results are the first to demonstrate that spatial profiling of immune architecture in DLBCL is associated with clinical outcomes, and that spatial analysis of immune cells should be explored as a potential biomarker for patients treated with immunotherapies. References [1] Coiffier, B. et al.,Blood116, 2040-5 (2010). [2] Lenz, G. et al.,N. Engl. J. Med.359, 2313-2323 (2008). [3] Ansell, S. et al.,J. Clin. Oncol.19, 720-726 (2001). [4] Qiu, L., et al., BMC Cancer19, 273 (2019). [5] Kridel, R., et al., Haematologica100, 143-5 (2015). [6] Giesen, C. et al., Nat. methods |11, 417 (2014). Figure Disclosures Merchant: Agios: Speakers Bureau; Pfizer: Consultancy, Research Funding.

Download Full-text

Development of tumor microenvironment-oriented digital pathology methods for whole slide image segmentation and classification

10.15388/vu.thesis.200 ◽

2021 ◽

Author(s):

◽

Mindaugas Morkūnas

Keyword(s):

Image Segmentation ◽

Tumor Microenvironment ◽

Digital Pathology ◽

Whole Slide Image ◽

Slide Image

Download Full-text

Unmasking the immune microecology of ductal carcinoma in situ with deep learning

npj Breast Cancer ◽

10.1038/s41523-020-00205-5 ◽

2021 ◽

Vol 7 (1) ◽

Author(s):

Priya Lakshmi Narayanan ◽

Shan E. Ahmed Raza ◽

Allison H. Hall ◽

Jeffrey R. Marks ◽

Lorraine King ◽

...

Keyword(s):

Deep Learning ◽

Ductal Carcinoma In Situ ◽

Carcinoma In Situ ◽

Ductal Carcinoma ◽

Digital Pathology ◽

Deep Convolutional Neural Networks ◽

Technological Developments ◽

Infiltrating Lymphocytes ◽

Spatial Architecture

AbstractDespite increasing evidence supporting the clinical relevance of tumour infiltrating lymphocytes (TILs) in invasive breast cancer, TIL spatial variability within ductal carcinoma in situ (DCIS) samples and its association with progression are not well understood. To characterise tissue spatial architecture and the microenvironment of DCIS, we designed and validated a new deep learning pipeline, UNMaSk. Following automated detection of individual DCIS ducts using a new method IM-Net, we applied spatial tessellation to create virtual boundaries for each duct. To study local TIL infiltration for each duct, DRDIN was developed for mapping the distribution of TILs. In a dataset comprising grade 2–3 pure DCIS and DCIS adjacent to invasive cancer (adjacent DCIS), we found that pure DCIS cases had more TILs compared to adjacent DCIS. However, the colocalisation of TILs with DCIS ducts was significantly lower in pure DCIS compared to adjacent DCIS, which may suggest a more inflamed tissue ecology local to DCIS ducts in adjacent DCIS cases. Our study demonstrates that technological developments in deep convolutional neural networks and digital pathology can enable an automated morphological and microenvironmental analysis of DCIS, providing a new way to study differential immune ecology for individual ducts and identify new markers of progression.

Download Full-text