scholarly journals Head and Neck Cancer Primary Tumor Auto Segmentation using Model Ensembling of Deep Learning in PET-CT Images

2021 ◽  
Author(s):  
Mohamed A. Naser ◽  
Kareem A. Wahid ◽  
Lisanne V. van Dijk ◽  
Renjie He ◽  
Moamen Abobakr Abdelaal ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cross Validation ◽  
External Validation ◽  
Unmet Need ◽  
Ct Images ◽  
Majority Voting ◽  
Dice Similarity Coefficient ◽  
Primary Tumors ◽  
Label Fusion ◽  
Pet Ct

Auto-segmentation of primary tumors in oropharyngeal cancer using PET/CT images is an unmet need that has the potential to improve radiation oncology workflows. In this study, we develop a series of deep learning models based on a 3D Residual Unet (ResUnet) architecture that can segment oropharyngeal tumors with high performance as demonstrated through internal and external validation of large-scale datasets (training size = 224 patients, testing size = 101 patients) as part of the 2021 HECKTOR Challenge. Specifically, we leverage ResUNet models with either 256 or 512 bottleneck layer channels that are able to demonstrate internal validation (10-fold cross-validation) mean Dice similarity coefficient (DSC) up to 0.771 and median 95% Hausdorff distance (95% HD) as low as 2.919 mm. We employ label fusion ensemble approaches, including Simultaneous Truth and Performance Level Estimation (STAPLE) and a voxel-level threshold approach based on majority voting (AVERAGE), to generate consensus segmentations on the test data by combining the segmentations produced through different trained cross-validation models. We demonstrate that our best performing ensembling approach (256 channels AVERAGE) achieves a mean DSC of 0.770 and median 95% HD of 3.143 mm through independent external validation on the test set. Concordance of internal and external validation results suggests our models are robust and can generalize well to unseen PET/CT data. We advocate that ResUNet models coupled to label fusion ensembling approaches are promising candidates for PET/CT oropharyngeal primary tumors auto-segmentation, with future investigations targeting the ideal combination of channel combinations and label fusion strategies to maximize seg-mentation performance.

scholarly journals Combining Tumor Segmentation Masks with PET/CT Images and Clinical Data in a Deep Learning Framework for Improved Prognostic Prediction in Head and Neck Squamous Cell Carcinoma

2021 ◽  
Author(s):  
Kareem A. Wahid ◽  
Renjie He ◽  
Cem Dede ◽  
Abdallah Sherif Radwan Mohamed ◽  
Moamen Abobakr Abdelaal ◽  
...  
Keyword(s):  
Squamous Cell Carcinoma ◽  
Deep Learning ◽  
Clinical Data ◽  
Cross Validation ◽  
External Validation ◽  
Ct Images ◽  
Tumor Segmentation ◽  
Learning Framework ◽  
Pet Ct ◽  
Additional Input

PET/CT images provide a rich data source for clinical prediction models in head and neck squamous cell carcinoma (HNSCC). Deep learning models often use images in an end-to-end fashion with clinical data or no additional input for predictions. However, in the context of HNSCC, the tumor region of interest may be an informative prior in the generation of improved prediction performance. In this study, we utilize a deep learning framework based on a DenseNet architecture to combine PET images, CT images, primary tumor segmentation masks, and clinical data as separate channels to predict progression-free survival (PFS) in days for HNSCC patients. Through internal validation (10-fold cross-validation) based on a large set of training data provided by the 2021 HECKTOR Challenge, we achieve a mean C-index of 0.855 +- 0.060 and 0.650 +- 0.074 when observed events are and are not included in the C-index calculation, respectively. Ensemble approaches applied to cross-validation folds yield C-index values up to 0.698 in the independent test set (external validation). Importantly, the value of the added segmentation mask is underscored in both internal and external validation by an improvement of the C-index when compared to models that do not utilize the segmentation mask. These promising results highlight the utility of including segmentation masks as additional input channels in deep learning pipelines for clinical outcome prediction in HNSCC.

scholarly journals Progression Free Survival Prediction for Head and Neck Cancer using Deep Learning based on Clinical and PET-CT Imaging Data

2021 ◽  
Author(s):  
Mohamed A. Naser ◽  
Kareem A. Wahid ◽  
Abdallah Sherif Radwan Mohamed ◽  
Moamen Abobakr Abdelaal ◽  
Renjie He ◽  
...  
Keyword(s):  
Deep Learning ◽  
Head And Neck ◽  
Cross Validation ◽  
Progression Free Survival ◽  
Ct Images ◽  
Training Data ◽  
Learning Approaches ◽  
Test Set ◽  
Free Survival ◽  
Pet Ct

Determining progression-free survival (PFS) for head and neck squamous cell carcinoma (HNSCC) patients is a challenging but pertinent task that could help stratify patients for improved overall outcomes. PET/CT images provide a rich source of anatomical and metabolic data for potential clinical biomarkers that would inform treatment decisions and could help improve PFS. In this study, we participate in the 2021 HECKTOR Challenge to predict PFS in a large dataset of HNSCC PET/CT images using deep learning approaches. We develop a series of deep learning models based on the DenseNet architecture using a negative log-likelihood loss function that utilizes PET/CT images and clinical data as separate input channels to predict PFS in days. Internal model validation based on 10-fold cross-validation using the training data (N=224) yielded C-index values up to 0.622 (without) and 0.842 (with) censoring status considered in C-index computation, respectively. We then implemented model ensembling approaches based on the training data cross-validation folds to predict the PFS of the test set patients (N=101). External validation on the test set for the best ensembling method yielded a C-index value of 0.694. Our results are a promising example of how deep learning approaches can effectively utilize imaging and clinical data for medical outcome prediction in HNSCC, but further work in optimizing these processes is needed.

scholarly journals Fully Automated Delineation of Gross Tumor Volume for Head and Neck Cancer on PET-CT Using Deep Learning: A Dual-Center Study

2018 ◽  
Vol 2018 ◽  
pp. 1-12 ◽  
Author(s):  
Bin Huang ◽  
Zhewei Chen ◽  
Po-Man Wu ◽  
Yufeng Ye ◽  
Shi-Ting Feng ◽  
...  
Keyword(s):  
Head And Neck Cancer ◽  
Deep Learning ◽  
Head And Neck ◽  
Neck Cancer ◽  
Gold Standard ◽  
Gross Tumor Volume ◽  
Tumor Volume ◽  
Ct Images ◽  
Dice Similarity Coefficient ◽  
Pet Ct

Purpose. In this study, we proposed an automated deep learning (DL) method for head and neck cancer (HNC) gross tumor volume (GTV) contouring on positron emission tomography-computed tomography (PET-CT) images. Materials and Methods. PET-CT images were collected from 22 newly diagnosed HNC patients, of whom 17 (Database 1) and 5 (Database 2) were from two centers, respectively. An oncologist and a radiologist decided the gold standard of GTV manually by consensus. We developed a deep convolutional neural network (DCNN) and trained the network based on the two-dimensional PET-CT images and the gold standard of GTV in the training dataset. We did two experiments: Experiment 1, with Database 1 only, and Experiment 2, with both Databases 1 and 2. In both Experiment 1 and Experiment 2, we evaluated the proposed method using a leave-one-out cross-validation strategy. We compared the median results in Experiment 2 (GTVa) with the performance of other methods in the literature and with the gold standard (GTVm). Results. A tumor segmentation task for a patient on coregistered PET-CT images took less than one minute. The dice similarity coefficient (DSC) of the proposed method in Experiment 1 and Experiment 2 was 0.481∼0.872 and 0.482∼0.868, respectively. The DSC of GTVa was better than that in previous studies. A high correlation was found between GTVa and GTVm (R = 0.99, P<0.001). The median volume difference (%) between GTVm and GTVa was 10.9%. The median values of DSC, sensitivity, and precision of GTVa were 0.785, 0.764, and 0.789, respectively. Conclusion. A fully automatic GTV contouring method for HNC based on DCNN and PET-CT from dual centers has been successfully proposed with high accuracy and efficiency. Our proposed method is of help to the clinicians in HNC management.

Prostate and dominant intraprostatic lesion segmentation on PET/CT using cascaded regional-net

2021 ◽  
Author(s):  
Luke A Matkovic ◽  
Tonghe Wang ◽  
Yang Lei ◽  
Oladunni O Akin-Akintayo ◽  
Olayinka A Abiodun Ojo ◽  
...  
Keyword(s):  
Radiation Therapy ◽  
Cross Validation ◽  
Ct Images ◽  
Second Step ◽  
Prostate Segmentation ◽  
Positron Emission ◽  
Pet Ct ◽  
The Mean ◽  
And Training ◽  
Fold Cross Validation

Abstract Focal dose boost to dominant intraprostatic lesions (DILs) has recently been proposed for prostate radiation therapy. Accurate and fast delineation of the prostate and DILs is thus required during treatment planning. We propose a learning-based method using positron emission tomography (PET)/computed tomography (CT) images to automatically segment the prostate and its DILs. To enable end-to-end segmentation, a deep learning-based method, called cascaded regional-Net, is utilized. The first network, referred to as dual attention network (DAN), is used to segment the prostate via extracting comprehensive features from both PET and CT images. A second network, referred to as mask scoring regional convolutional neural network (MSR-CNN), is used to segment the DILs from the PET and CT within the prostate region. Scoring strategy is used to diminish the misclassification of the DILs. For DIL segmentation, the proposed cascaded regional-Net uses two steps to remove normal tissue regions, with the first step cropping images based on prostate segmentation and the second step using MSR-CNN to further locate the DILs. The binary masks of DILs and prostates of testing patients are generated from PET/CT by the trained network. To evaluate the proposed method, we retrospectively investigated 49 PET/CT datasets. On each dataset, the prostate and DILs were delineated by physicians and set as the ground truths and training targets. The proposed method was trained and evaluated using a five-fold cross-validation and a hold-out test. The mean surface distance and DSC values were 0.666±0.696mm and 0.932±0.059 for the prostate and 1.209±1.954mm and 0.757±0.241 for the DILs among all 49 patients. The proposed method has demonstrated great potential for improving the efficiency and reducing the observer variability of prostate and DIL contouring for DIL focal boost prostate radiation therapy.

scholarly journals Prediction of 5-year Progression-Free Survival in Advanced Nasopharyngeal Carcinoma with Pretreatment PET/CT using Multi-Modality Deep Learning-based Radiomics

2021 ◽  
Author(s):  
Bingxin Gu ◽  
Mingyuan Meng ◽  
Lei Bi ◽  
Jinman Kim ◽  
David Dagan Feng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Medical Image Analysis ◽  
Progression Free Survival ◽  
Ct Images ◽  
Tnm Stage ◽  
Support Vector ◽  
Free Survival ◽  
Prognostic Performance ◽  
Pet Ct ◽  
Testing Error

Abstract Purpose Deep Learning-based Radiomics (DLR) has achieved great success on medical image analysis. In this study, we aimed to explore the capability of our proposed end-to-end multi-modality DLR model using pretreatment PET/CT images to predict 5-year Progression-Free Survival (PFS) in advanced NPC.Methods A total of 170 patients with pathological confirmed advanced NPC (TNM stage III or IVa) were enrolled in this study. A 3D Convolutional Neural Network (CNN), with two branches to process PET and CT separately, was optimized to extract deep features from pretreatment multi-modality PET/CT images and use the derived features to predict the probability of 5-year PFS. Optionally, TNM stage, as a high-level clinical feature, can be integrated into our DLR model to further improve prognostic performance. Results For a comparison between Conventional Radiomic (CR) and DLR, 1456 handcrafted features were extracted, and three top CR methods, Random Forest (RF) + RF (AUC = 0.796 ± 0.009, testing error = 0.267 ± 0.007), RF + Adaptive Boosting (AdaBoost) (AUC = 0.783 ± 0.011, testing error = 0.286 ± 0.009), and L1-Logistic Regression (L1-LOG) + Kernel Support Vector Machines (KSVM) (AUC = 0.769 ± 0.008, testing error = 0.298 ± 0.006), were selected as benchmarks from 54 combinations of 6 feature selection methods and 9 classification methods. Compared to the three CR methods, our multi-modality DLR models using both PET and CT, with or without TNM stage (named PCT or PC model), resulted in the highest prognostic performance (PCT model: AUC = 0.842 ± 0.034, testing error = 0.194 ± 0.029; PC model: AUC = 0.825 ± 0.041, testing error = 0.223 ± 0.035). Furthermore, the multi-modality PCT model outperformed single-modality DLR models using only PET and TNM stage (named PT model: AUC = 0.818 ± 0.029, testing error = 0.218 ± 0.024) or only CT and TNM stage (named CT model: AUC = 0.657 ± 0.055, testing error = 0.375 ± 0.048). Conclusion Our study identified potential radiomics-based prognostic model for survival prediction in advanced NPC, and suggests that DLR could serve as a tool for aiding in cancer management.

Application of Deep Learning as a Noninvasive Tool to Determine Pathological Diagnosis of Enlarged Cervical Lymph Nodes with PET/CT

2020 ◽  
Author(s):  
Yuhan Yang ◽  
Bo Zheng ◽  
Yixi Wang ◽  
Xuelei Ma
Keyword(s):  
Deep Learning ◽  
Lymph Nodes ◽  
Diagnostic Performance ◽  
Imaging Modality ◽  
Ct Images ◽  
Pathological Diagnosis ◽  
Therapeutic Interventions ◽  
Cross Sectional ◽  
Cervical Lymph Nodes ◽  
Pet Ct

Abstract Objective: To construct a deep-learning convolution neural network (DL-CNN) system for pathological diagnosis of cervical lymph nodes by using computed tomography (CT), fluorine-18 fluorodeoxyglucose (18F-FDG) positron emission tomography (PET), and fused PET/CT images.Materials and methods: A total of 1020 cross-sectional images for each imaging modality was obtained from 211 patients (153 patients with lymphomas and 116 patients with metastases) with enlarged cervical lymph nodes from January 2014 to June 2018. All eligible images were distributed randomly into the training, validation, and testing cohorts with ratios of 70%, 15%, and 15%. We applied eight DL-CNN algorithms with pretrained bases from ImageNet dataset on CT, PET, and fused PET/CT imaging datasets to differentiate lymphomatous nodes from metastatic nodes, respectively. Attention heatmaps of PET and fused PET/CT images generated by class activation mapping (CAM) were used in visualization of class specific regions recognized by the prediction model with best performance. Results: The accuracy of eight deep learning algorithms with pretrained base ranged from 0.650 to 0.981 on PET testing cohort, and from 0.738 to 0.981 on fused PET/CT testing cohort. The VGG16 model on PET images and DenseNet121 model on fused PET/CT images had the best diagnostic performance among all eight algorithms with sensitivity and specificity of 1.000 and 0.963. Class-specific discriminative subregions were highlighted by attention maps for clinical review.Conclusion: A DL-CNN system was developed for classifying metastatic and lymphomatous involvement with favorable diagnostic performance on PET and PET/CT images in patients with enlarged cervical lymph nodes. The further clinical practice of this system may improve quality of the following therapeutic interventions and optimize patients’ outcomes.

A coronary artery CTA segmentation approach based on deep learning

2021 ◽  
pp. 1-14
Author(s):  
Caiyun Huang ◽  
Changhua Yin
Keyword(s):  
Deep Learning ◽  
Coronary Artery ◽  
Blood Vessel ◽  
Vessel Segmentation ◽  
Majority Voting ◽  
Dice Similarity Coefficient ◽  
Fusion Method ◽  
Level Set Function ◽  
First Choice ◽  
Medical Image Diagnosis

Presence of plaque and coronary artery stenosis are the main causes of coronary heart disease. Detection of plaque and coronary artery segmentation have become the first choice in detecting coronary artery disease. The purpose of this study is to investigate a new method for plaque detection and automatic segmentation and diagnosis of coronary arteries and to test its feasibility of applying to clinical medical image diagnosis. A multi-model fusion coronary CT angiography (CTA) vessel segmentation method is proposed based on deep learning. The method includes three network layer models namely, an original 3-dimensional full convolutional network (3D FCN) and two networks that embed the attention gating (AG) model in the original 3D FCN. Then, the prediction results of the three networks are merged by using the majority voting algorithm and thus the final prediction result of the networks is obtained. In the post-processing stage, the level set function is used to further iteratively optimize the results of network fusion prediction. The JI (Jaccard index) and DSC (Dice similarity coefficient) scores are calculated to evaluate accuracy of blood vessel segmentations. Applying to a CTA dataset of 20 patients, accuracy of coronary blood vessel segmentation using FCN, FCN-AG1, FCN-AG2 network and the fusion method are tested. The average values of JI and DSC of using the first three networks are (0.7962, 0.8843), (0.8154, 0.8966) and (0.8119, 0.8936), respectively. When using new fusion method, average JI and DSC of segmentation results increase to (0.8214, 0.9005), which are better than the best result of using FCN, FCN-AG1 and FCN-AG2 model independently.

scholarly journals Non-invasive measurement of PD-L1 status and prediction of immunotherapy response using deep learning of PET/CT images

2021 ◽  
Vol 9 (6) ◽  
pp. e002118
Author(s):  
Wei Mu ◽  
Lei Jiang ◽  
Yu Shi ◽  
Ilke Tunali ◽  
Jhanelle E Gray ◽  
...  
Keyword(s):  
Checkpoint Inhibitors ◽  
Quantitative Imaging ◽  
External Validation ◽  
Progression Free Survival ◽  
Ct Images ◽  
Operating Characteristics ◽  
Convolutional Network ◽  
Nccn Guidelines ◽  
Pet Ct ◽  
Expression Status

BackgroundCurrently, only a fraction of patients with non-small cell lung cancer (NSCLC) treated with immune checkpoint inhibitors (ICIs) experience a durable clinical benefit (DCB). According to NCCN guidelines, Programmed death-ligand 1 (PD-L1) expression status determined by immunohistochemistry (IHC) of biopsies is the only clinically approved companion biomarker to trigger the use of ICI therapy. Based on prior work showing a relationship between quantitative imaging and gene expression, we hypothesize that quantitative imaging (radiomics) can provide an alternative surrogate for PD-L1 expression status in clinical decision support.Methods18F-FDG-PET/CT images and clinical data were curated from 697 patients with NSCLC from three institutions and these were analyzed using a small-residual-convolutional-network (SResCNN) to develop a deeply learned score (DLS) to predict the PD-L1 expression status. This developed model was further used to predict DCB, progression-free survival (PFS), and overall survival (OS) in two retrospective and one prospective test cohorts of ICI-treated patients with advanced stage NSCLC.ResultsThe PD-L1 DLS significantly discriminated between PD-L1 positive and negative patients (area under receiver operating characteristics curve ≥0.82 in the training, validation, and two external test cohorts). Importantly, the DLS was indistinguishable from IHC-derived PD-L1 status in predicting PFS and OS, suggesting the utility of DLS as a surrogate for IHC. A score generated by combining the DLS with clinical characteristics was able to accurately (C-indexes of 0.70–0.87) predict DCB, PFS, and OS in retrospective training, prospective testing and external validation cohorts.ConclusionHence, we propose DLS as a surrogate or substitute for IHC-determined PD-L1 measurement to guide individual pretherapy decisions pending in larger prospective trials.

Sign in / Sign up

Export Citation Format

Share Document