Prediction of biomarker status, diagnosis and outcome from histology slides using deep learning-based hypothesis free feature extraction.

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. 3140-3140
Author(s):  
Eldad Klaiman ◽  
Jacob Gildenblat ◽  
Ido Ben-Shaul ◽  
Astrid Heller ◽  
Konstanty Korski ◽  
...  
Keyword(s):  
Deep Learning ◽  
Binary Classification ◽  
Pathologic Complete Response ◽  
Area Under The Curve ◽  
Response Prediction ◽  
Complete Response ◽  
Algorithm Performance ◽  
New Approach ◽  
Predictive Algorithm ◽  
Learning Framework

3140 Background: Recently, histological pattern signatures obtained from diagnostic H&E images have been found to predict mutation, biomarker status or outcome. We report here on a novel deep learning based framework designed to identify and extract predictive histological signatures. We have applied this framework in 3 experiments, predicting specifically the microsatellite status (MSS) of colorectal cancer (CRC), breast cancer (BC) micrometastasis in Lymph nodes (LN) and Pathologic Complete Response (pCR) in BC diagnostic biopsies. Methods: Our deep learning based algorithm was trained on histology images at 20X magnification. Algorithms were trained for binary classification for each of the three cohorts. We used 75% of the images for training and test our algorithm on the remaining 25% of the images. Cohort details are as follows: MSS for CRC: 94 patients’ H&E stained tissue images from the Roche internal CRC80 dataset (MSS n =24; MSI n = 70) were used. BC LN: 270 patients’ H&E stained tissue images from the CAMELYON16 dataset ( LN(+) n = 110 ; LN(-), n =160) were used. pCR for BC: 225 patients’ H&E stained tissue images from the Tryphaena Study BO22280, neoadjuvant, Trastuzumab/Pertuzumab chemotherapy combination trial. (pCR=111, non-pCR n=114). Results: We report and assess algorithm performance on each of the cohorts by Area Under the Curve (AUC). Prediction of MSS in the CRC80 status yielded AUC 0.9. Prediction of LN invasion on CAMELYON16 dataset yielded AUC 0.85. Prediction of pCR on the Tryphaena cohort yielded an AUC of 0.8. Conclusions: We present a new approach to generate predictive signatures based on conventional diagnostic H&E images and a novel machine learning framework. The CRC80 and CAMELYON16 cohorts served as a confidence building experiments with predictive features well known by clinicians and visually confirmed. The predictive algorithm for pCR in the Tryphaena cohort yielded both response prediction and the high predictive value FOVs. These included tissue patterns which have not until now been considered to influence on the prediction of pCR.

scholarly journals Predicting treatment response from longitudinal images using multi-task deep learning

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Cheng Jin ◽  
Heng Yu ◽  
Jia Ke ◽  
Peirong Ding ◽  
Yongju Yi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Treatment Response ◽  
Pathologic Complete Response ◽  
Neoadjuvant Chemoradiotherapy ◽  
Biological Response ◽  
Response Prediction ◽  
Complete Response ◽  
Tumor Segmentation ◽  
Feature Representations ◽  
Treatment Response Evaluation

AbstractRadiographic imaging is routinely used to evaluate treatment response in solid tumors. Current imaging response metrics do not reliably predict the underlying biological response. Here, we present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction. We design two Siamese subnetworks that are joined at multiple layers, which enables integration of multi-scale feature representations and in-depth comparison of pre-treatment and post-treatment images. The network is trained using 2568 magnetic resonance imaging scans of 321 rectal cancer patients for predicting pathologic complete response after neoadjuvant chemoradiotherapy. In multi-institution validation, the imaging-based model achieves AUC of 0.95 (95% confidence interval: 0.91–0.98) and 0.92 (0.87–0.96) in two independent cohorts of 160 and 141 patients, respectively. When combined with blood-based tumor markers, the integrated model further improves prediction accuracy with AUC 0.97 (0.93–0.99). Our approach to capturing dynamic information in longitudinal images may be broadly used for screening, treatment response evaluation, disease monitoring, and surveillance.

scholarly journals 833 A scalable deep learning framework for rapid automated annotation of histologic and morphologic features from large unlabeled pan-cancer H&E datasets

2021 ◽  
Vol 9 (Suppl 3) ◽  
pp. A874-A874
Author(s):  
David Soong ◽  
David Soong ◽  
David Soong ◽  
Anantharaman Muthuswamy ◽  
Clifton Drew ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Solid Tumor ◽  
Blood Cells ◽  
White Blood Cells ◽  
Digital Pathology ◽  
Area Under The Curve ◽  
Morphological Characteristics ◽  
Learning Framework ◽  
Tumor Types

BackgroundRecent advances in machine learning and digital pathology have enabled a variety of applications including predicting tumor grade and genetic subtypes, quantifying the tumor microenvironment (TME), and identifying prognostic morphological features from H&E whole slide images (WSI). These supervised deep learning models require large quantities of images manually annotated with cellular- and tissue-level details by pathologists, which limits scale and generalizability across cancer types and imaging platforms. Here we propose a semi-supervised deep learning framework that automatically annotates biologically relevant image content from hundreds of solid tumor WSI with minimal pathologist intervention, thus improving quality and speed of analytical workflows aimed at deriving clinically relevant features.MethodsThe dataset consisted of >200 H&E images across >10 solid tumor types (e.g. breast, lung, colorectal, cervical, and urothelial cancers) from advanced disease patients. WSI were first partitioned into small tiles of 128μm for feature extraction using a 50-layer convolutional neural network pre-trained on the ImageNet database. Dimensionality reduction and unsupervised clustering were applied to the resultant embeddings and image clusters were identified with enriched histological and morphological characteristics. A random subset of representative tiles (<0.5% of whole slide tissue areas) from these distinct image clusters was manually reviewed by pathologists and assigned to eight histological and morphological categories: tumor, stroma/connective tissue, necrotic cells, lymphocytes, red blood cells, white blood cells, normal tissue and glass/background. This dataset allowed the development of a multi-label deep neural network to segment morphologically distinct regions and detect/quantify histopathological features in WSI.ResultsAs representative image tiles within each image cluster were morphologically similar, expert pathologists were able to assign annotations to multiple images in parallel, effectively at 150 images/hour. Five-fold cross-validation showed average prediction accuracy of 0.93 [0.8–1.0] and area under the curve of 0.90 [0.8–1.0] over the eight image categories. As an extension of this classifier framework, all whole slide H&E images were segmented and composite lymphocyte, stromal, and necrotic content per patient tumor was derived and correlated with estimates by pathologists (p<0.05).ConclusionsA novel and scalable deep learning framework for annotating and learning H&E features from a large unlabeled WSI dataset across tumor types was developed. This automated approach accurately identified distinct histomorphological features, with significantly reduced labeling time and effort required for pathologists. Further, this classifier framework was extended to annotate regions enriched in lymphocytes, stromal, and necrotic cells – important TME contexture with clinical relevance for patient prognosis and treatment decisions.

scholarly journals Study of the Response Prediction of a Parametric Vibration Based on Modulation Feedback

2010 ◽  
Vol 2010 ◽  
pp. 1-9 ◽  
Author(s):  
Dishan Huang
Keyword(s):  
Fourier Series ◽  
Harmonic Balance ◽  
Engineering Application ◽  
Harmonic Component ◽  
Response Prediction ◽  
Complete Response ◽  
Parametric Vibration ◽  
Theoretical Research ◽  
New Approach ◽  
Response Expression

The problem of response prediction is investigated for parametric vibration in terms of a new concept. The response solution is presented in the special form of Fourier series for signal degree freedom of parametric vibration based on modulation feedback. By applying harmonic balance and limitation operation, all coefficients of a harmonic component are fully determined with a set of series. Meanwhile, some important dynamic behaviors are exposed through mathematical deduction, and an instability phenomenon can be discussed through given frequency factors. The investigation result shows that the new approach has an advantage in the complete response expression, and it is very significant for the theoretical research and engineering application concerning parametric vibration.

Pretreatment blood biomarkers predict pathologic responses to neo-CRT in patients with locally advanced rectal cancer

2019 ◽  
Vol 15 (28) ◽  
pp. 3233-3242
Author(s):  
Aijie Li ◽  
Kewen He ◽  
Dong Guo ◽  
Chao Liu ◽  
Duoying Wang ◽  
...  
Keyword(s):  
Rectal Cancer ◽  
Locally Advanced ◽  
Pathologic Complete Response ◽  
Area Under The Curve ◽  
Complete Response ◽  
Advanced Rectal Cancer ◽  
Locally Advanced Rectal Cancer ◽  
Blood Biomarkers ◽  
Platelet To Lymphocyte Ratio ◽  
Lymphocyte Ratio

Aim: To evaluate the value of pretreatment blood biomarkers in predicting pathologic responses to neoadjuvant chemoradiotherapy (neo-CRT) in patients with locally advanced rectal cancer. Materials & methods: We conducted logistic regression analysis and receiver operating characteristic to assess the predictive value of blood biomarkers. The outcome was defined by the pathologic complete response and good response. Results: Carcinoembryonic antigen (CEA) (p < 0.001), neutrophil-to-lymphocyte ratio (p = 0.024), platelet-to-lymphocyte ratio (p = 0.006) and lymphocyte-to-monocyte ratio (LMR) (p < 0.001) were significant predictors of pathologic complete response, with area under the curve of 0.785, 0.794, 0.740 and 0.913, respectively; CEA (p = 0.007) and LMR (p < 0.001) correlated significantly with good response, with area under the curve of 0.743 and 0.771, respectively. Conclusion: Lower LMR and higher CEA, neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio before treatment could predict poorer pathologic response to neo-CRT in patients with locally advanced rectal cancer.

scholarly journals Deep Ordinal Regression Based on Data Relationship for Small Datasets

2017 ◽  
Author(s):  
Yanzhu Liu ◽  
Adams Wai Kin Kong ◽  
Chi Keong Goh
Keyword(s):  
Deep Learning ◽  
Deep Neural Networks ◽  
Color Image ◽  
Binary Classification ◽  
Ordinal Regression ◽  
Learning Problems ◽  
Classification Problems ◽  
Regression Problem ◽  
New Approach ◽  
High Level

Ordinal regression aims to classify instances into ordinal categories. As with other supervised learning problems, learning an effective deep ordinal model from a small dataset is challenging. This paper proposes a new approach which transforms the ordinal regression problem to binary classification problems and uses triplets with instances from different categories to train deep neural networks such that high-level features describing their ordinal relationship can be extracted automatically. In the testing phase, triplets are formed by a testing instance and other instances with known ranks. A decoder is designed to estimate the rank of the testing instance based on the outputs of the network. Because of the data argumentation by permutation, deep learning can work for ordinal regression even on small datasets. Experimental results on the historical color image benchmark and MSRA image search datasets demonstrate that the proposed algorithm outperforms the traditional deep learning approach and is comparable with other state-of-the-art methods, which are highly based on prior knowledge to design effective features.

scholarly journals A Two-Stage Deep Neural Network Framework for Precipitation Estimation from Bispectral Satellite Information

2018 ◽  
Vol 19 (2) ◽  
pp. 393-408 ◽  
Author(s):  
Yumeng Tao ◽  
Kuolin Hsu ◽  
Alexander Ihler ◽  
Xiaogang Gao ◽  
Soroosh Sorooshian
Keyword(s):  
Deep Learning ◽  
Binary Classification ◽  
Precipitation Amount ◽  
Skewed Distribution ◽  
Stage Model ◽  
Two Stage ◽  
Learning Framework ◽  
Second Stage ◽  
Satellite Information ◽  
Precipitation Estimation

Abstract Compared to ground precipitation measurements, satellite-based precipitation estimation products have the advantage of global coverage and high spatiotemporal resolutions. However, the accuracy of satellite-based precipitation products is still insufficient to serve many weather, climate, and hydrologic applications at high resolutions. In this paper, the authors develop a state-of-the-art deep learning framework for precipitation estimation using bispectral satellite information, infrared (IR), and water vapor (WV) channels. Specifically, a two-stage framework for precipitation estimation from bispectral information is designed, consisting of an initial rain/no-rain (R/NR) binary classification, followed by a second stage estimating the nonzero precipitation amount. In the first stage, the model aims to eliminate the large fraction of NR pixels and to delineate precipitation regions precisely. In the second stage, the model aims to estimate the pointwise precipitation amount accurately while preserving its heavily skewed distribution. Stacked denoising autoencoders (SDAEs), a commonly used deep learning method, are applied in both stages. Performance is evaluated along a number of common performance measures, including both R/NR and real-valued precipitation accuracy, and compared with an operational product, Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks–Cloud Classification System (PERSIANN-CCS). For R/NR binary classification, the proposed two-stage model outperforms PERSIANN-CCS by 32.56% in the critical success index (CSI). For real-valued precipitation estimation, the two-stage model is 23.40% lower in average bias, is 44.52% lower in average mean squared error, and has a 27.21% higher correlation coefficient. Hence, the two-stage deep learning framework has the potential to serve as a more accurate and more reliable satellite-based precipitation estimation product. The authors also provide some future directions for development of satellite-based precipitation estimation products in both incorporating auxiliary information and improving retrieval algorithms.

Sign in / Sign up

Export Citation Format

Share Document