Combining Tumor Segmentation Masks with PET/CT Images and Clinical Data in a Deep Learning Framework for Improved Prognostic Prediction in Head and Neck Squamous Cell Carcinoma

PET/CT images provide a rich data source for clinical prediction models in head and neck squamous cell carcinoma (HNSCC). Deep learning models often use images in an end-to-end fashion with clinical data or no additional input for predictions. However, in the context of HNSCC, the tumor region of interest may be an informative prior in the generation of improved prediction performance. In this study, we utilize a deep learning framework based on a DenseNet architecture to combine PET images, CT images, primary tumor segmentation masks, and clinical data as separate channels to predict progression-free survival (PFS) in days for HNSCC patients. Through internal validation (10-fold cross-validation) based on a large set of training data provided by the 2021 HECKTOR Challenge, we achieve a mean C-index of 0.855 +- 0.060 and 0.650 +- 0.074 when observed events are and are not included in the C-index calculation, respectively. Ensemble approaches applied to cross-validation folds yield C-index values up to 0.698 in the independent test set (external validation). Importantly, the value of the added segmentation mask is underscored in both internal and external validation by an improvement of the C-index when compared to models that do not utilize the segmentation mask. These promising results highlight the utility of including segmentation masks as additional input channels in deep learning pipelines for clinical outcome prediction in HNSCC.

Download Full-text

Head and Neck Cancer Primary Tumor Auto Segmentation using Model Ensembling of Deep Learning in PET-CT Images

10.1101/2021.10.14.21264953 ◽

2021 ◽

Author(s):

Mohamed A. Naser ◽

Kareem A. Wahid ◽

Lisanne V. van Dijk ◽

Renjie He ◽

Moamen Abobakr Abdelaal ◽

...

Keyword(s):

Deep Learning ◽

Cross Validation ◽

External Validation ◽

Unmet Need ◽

Ct Images ◽

Majority Voting ◽

Dice Similarity Coefficient ◽

Primary Tumors ◽

Label Fusion ◽

Pet Ct

Auto-segmentation of primary tumors in oropharyngeal cancer using PET/CT images is an unmet need that has the potential to improve radiation oncology workflows. In this study, we develop a series of deep learning models based on a 3D Residual Unet (ResUnet) architecture that can segment oropharyngeal tumors with high performance as demonstrated through internal and external validation of large-scale datasets (training size = 224 patients, testing size = 101 patients) as part of the 2021 HECKTOR Challenge. Specifically, we leverage ResUNet models with either 256 or 512 bottleneck layer channels that are able to demonstrate internal validation (10-fold cross-validation) mean Dice similarity coefficient (DSC) up to 0.771 and median 95% Hausdorff distance (95% HD) as low as 2.919 mm. We employ label fusion ensemble approaches, including Simultaneous Truth and Performance Level Estimation (STAPLE) and a voxel-level threshold approach based on majority voting (AVERAGE), to generate consensus segmentations on the test data by combining the segmentations produced through different trained cross-validation models. We demonstrate that our best performing ensembling approach (256 channels AVERAGE) achieves a mean DSC of 0.770 and median 95% HD of 3.143 mm through independent external validation on the test set. Concordance of internal and external validation results suggests our models are robust and can generalize well to unseen PET/CT data. We advocate that ResUNet models coupled to label fusion ensembling approaches are promising candidates for PET/CT oropharyngeal primary tumors auto-segmentation, with future investigations targeting the ideal combination of channel combinations and label fusion strategies to maximize seg-mentation performance.

Download Full-text

Tumor Segmentation in Patients with Head and Neck Cancers Using Deep Learning Based-on Multi-modality PET/CT Images

Head and Neck Tumor Segmentation - Lecture Notes in Computer Science ◽

10.1007/978-3-030-67194-5_10 ◽

2021 ◽

pp. 85-98 ◽

Cited By ~ 1

Author(s):

Mohamed A. Naser ◽

Lisanne V. van Dijk ◽

Renjie He ◽

Kareem A. Wahid ◽

Clifton D. Fuller

Keyword(s):

Deep Learning ◽

Head And Neck ◽

Ct Images ◽

Head And Neck Cancers ◽

Tumor Segmentation ◽

Pet Ct

Download Full-text

Progression Free Survival Prediction for Head and Neck Cancer using Deep Learning based on Clinical and PET-CT Imaging Data

10.1101/2021.10.14.21264955 ◽

2021 ◽

Author(s):

Mohamed A. Naser ◽

Kareem A. Wahid ◽

Abdallah Sherif Radwan Mohamed ◽

Moamen Abobakr Abdelaal ◽

Renjie He ◽

...

Keyword(s):

Deep Learning ◽

Head And Neck ◽

Cross Validation ◽

Progression Free Survival ◽

Ct Images ◽

Training Data ◽

Learning Approaches ◽

Test Set ◽

Free Survival ◽

Pet Ct

Determining progression-free survival (PFS) for head and neck squamous cell carcinoma (HNSCC) patients is a challenging but pertinent task that could help stratify patients for improved overall outcomes. PET/CT images provide a rich source of anatomical and metabolic data for potential clinical biomarkers that would inform treatment decisions and could help improve PFS. In this study, we participate in the 2021 HECKTOR Challenge to predict PFS in a large dataset of HNSCC PET/CT images using deep learning approaches. We develop a series of deep learning models based on the DenseNet architecture using a negative log-likelihood loss function that utilizes PET/CT images and clinical data as separate input channels to predict PFS in days. Internal model validation based on 10-fold cross-validation using the training data (N=224) yielded C-index values up to 0.622 (without) and 0.842 (with) censoring status considered in C-index computation, respectively. We then implemented model ensembling approaches based on the training data cross-validation folds to predict the PFS of the test set patients (N=101). External validation on the test set for the best ensembling method yielded a C-index value of 0.694. Our results are a promising example of how deep learning approaches can effectively utilize imaging and clinical data for medical outcome prediction in HNSCC, but further work in optimizing these processes is needed.

Download Full-text

COVID-19 pneumonia diagnosis using a simple 2D deep learning framework with a single chest CT image (Preprint)

10.2196/preprints.19407 ◽

2020 ◽

Author(s):

Jinseok Lee

Keyword(s):

Deep Learning ◽

Diagnostic Performance ◽

Ct Images ◽

Chest Ct ◽

University Hospital ◽

Detection Accuracy ◽

Ct Image ◽

Test Dataset ◽

Learning Framework ◽

Testing Dataset

BACKGROUND The coronavirus disease (COVID-19) has explosively spread worldwide since the beginning of 2020. According to a multinational consensus statement from the Fleischner Society, computed tomography (CT) can be used as a relevant screening tool owing to its higher sensitivity for detecting early pneumonic changes. However, physicians are extremely busy fighting COVID-19 in this era of worldwide crisis. Thus, it is crucial to accelerate the development of an artificial intelligence (AI) diagnostic tool to support physicians. OBJECTIVE We aimed to quickly develop an AI technique to diagnose COVID-19 pneumonia and differentiate it from non-COVID pneumonia and non-pneumonia diseases on CT. METHODS A simple 2D deep learning framework, named fast-track COVID-19 classification network (FCONet), was developed to diagnose COVID-19 pneumonia based on a single chest CT image. FCONet was developed by transfer learning, using one of the four state-of-art pre-trained deep learning models (VGG16, ResNet50, InceptionV3, or Xception) as a backbone. For training and testing of FCONet, we collected 3,993 chest CT images of patients with COVID-19 pneumonia, other pneumonia, and non-pneumonia diseases from Wonkwang University Hospital, Chonnam National University Hospital, and the Italian Society of Medical and Interventional Radiology public database. These CT images were split into a training and a testing set at a ratio of 8:2. For the test dataset, the diagnostic performance to diagnose COVID-19 pneumonia was compared among the four pre-trained FCONet models. In addition, we tested the FCONet models on an additional external testing dataset extracted from the embedded low-quality chest CT images of COVID-19 pneumonia in recently published papers. RESULTS Of the four pre-trained models of FCONet, the ResNet50 showed excellent diagnostic performance (sensitivity 99.58%, specificity 100%, and accuracy 99.87%) and outperformed the other three pre-trained models in testing dataset. In additional external test dataset using low-quality CT images, the detection accuracy of the ResNet50 model was the highest (96.97%), followed by Xception, InceptionV3, and VGG16 (90.71%, 89.38%, and 87.12%, respectively). CONCLUSIONS The FCONet, a simple 2D deep learning framework based on a single chest CT image, provides excellent diagnostic performance in detecting COVID-19 pneumonia. Based on our testing dataset, the ResNet50-based FCONet might be the best model, as it outperformed other FCONet models based on VGG16, Xception, and InceptionV3.

Download Full-text

Textural Features of Pretreatment 18F-FDG PET/CT Images: Prognostic Significance in Patients with Advanced T-Stage Oropharyngeal Squamous Cell Carcinoma

Journal of Nuclear Medicine ◽

10.2967/jnumed.112.119289 ◽

2013 ◽

Vol 54 (10) ◽

pp. 1703-1709 ◽

Cited By ~ 91

Author(s):

N.-M. Cheng ◽

Y.-H. Dean Fang ◽

J. Tung-Chieh Chang ◽

C.-G. Huang ◽

D.-L. Tsan ◽

...

Keyword(s):

Squamous Cell Carcinoma ◽

Cell Carcinoma ◽

Squamous Cell ◽

Fdg Pet ◽

Prognostic Significance ◽

Ct Images ◽

Textural Features ◽

Pet Ct ◽

18F Fdg ◽

Fdg Pet Ct

Download Full-text

Distinguishing nontuberculous mycobacteria from Mycobacterium tuberculosis lung disease from CT images using a deep learning framework

European Journal of Nuclear Medicine and Molecular Imaging ◽

10.1007/s00259-021-05432-x ◽

2021 ◽

Author(s):

Li Wang ◽

Wenlong Ding ◽

Yan Mo ◽

Dejun Shi ◽

Shuo Zhang ◽

...

Keyword(s):

Mycobacterium Tuberculosis ◽

Deep Learning ◽

Lung Disease ◽

Nontuberculous Mycobacteria ◽

Ct Images ◽

Learning Framework

Download Full-text

ResBCDU-Net: A Deep Learning Framework for Lung CT Image Segmentation

Sensors ◽

10.3390/s21010268 ◽

2021 ◽

Vol 21 (1) ◽

pp. 268

Author(s):

Yeganeh Jalali ◽

Mansoor Fateh ◽

Mohsen Rezvani ◽

Vahid Abolghasemi ◽

Mohammad Hossein Anisi

Keyword(s):

Image Segmentation ◽

Deep Learning ◽

Ct Images ◽

Medical Image Segmentation ◽

Great Success ◽

Morphological Operations ◽

Ct Image ◽

Feature Maps ◽

Learning Framework ◽

Lung Ct

Lung CT image segmentation is a key process in many applications such as lung cancer detection. It is considered a challenging problem due to existing similar image densities in the pulmonary structures, different types of scanners, and scanning protocols. Most of the current semi-automatic segmentation methods rely on human factors therefore it might suffer from lack of accuracy. Another shortcoming of these methods is their high false-positive rate. In recent years, several approaches, based on a deep learning framework, have been effectively applied in medical image segmentation. Among existing deep neural networks, the U-Net has provided great success in this field. In this paper, we propose a deep neural network architecture to perform an automatic lung CT image segmentation process. In the proposed method, several extensive preprocessing techniques are applied to raw CT images. Then, ground truths corresponding to these images are extracted via some morphological operations and manual reforms. Finally, all the prepared images with the corresponding ground truth are fed into a modified U-Net in which the encoder is replaced with a pre-trained ResNet-34 network (referred to as Res BCDU-Net). In the architecture, we employ BConvLSTM (Bidirectional Convolutional Long Short-term Memory)as an advanced integrator module instead of simple traditional concatenators. This is to merge the extracted feature maps of the corresponding contracting path into the previous expansion of the up-convolutional layer. Finally, a densely connected convolutional layer is utilized for the contracting path. The results of our extensive experiments on lung CT images (LIDC-IDRI database) confirm the effectiveness of the proposed method where a dice coefficient index of 97.31% is achieved.

Download Full-text

Deep6mA: A deep learning framework for exploring similar patterns in DNA N6-methyladenine sites across different species

PLoS Computational Biology ◽

10.1371/journal.pcbi.1008767 ◽

2021 ◽

Vol 17 (2) ◽

pp. e1008767

Author(s):

Zutan Li ◽

Hangjin Jiang ◽

Lingpeng Kong ◽

Yuanyuan Chen ◽

Kun Lang ◽

...

Keyword(s):

Deep Learning ◽

Prediction Accuracy ◽

Cross Validation ◽

Biological Processes ◽

Biological Functions ◽

Learning Framework ◽

Wide Range ◽

Genomic Scale ◽

Downstream Gene Expression ◽

Fold Cross Validation

N6-methyladenine (6mA) is an important DNA modification form associated with a wide range of biological processes. Identifying accurately 6mA sites on a genomic scale is crucial for under-standing of 6mA’s biological functions. However, the existing experimental techniques for detecting 6mA sites are cost-ineffective, which implies the great need of developing new computational methods for this problem. In this paper, we developed, without requiring any prior knowledge of 6mA and manually crafted sequence features, a deep learning framework named Deep6mA to identify DNA 6mA sites, and its performance is superior to other DNA 6mA prediction tools. Specifically, the 5-fold cross-validation on a benchmark dataset of rice gives the sensitivity and specificity of Deep6mA as 92.96% and 95.06%, respectively, and the overall prediction accuracy is 94%. Importantly, we find that the sequences with 6mA sites share similar patterns across different species. The model trained with rice data predicts well the 6mA sites of other three species: Arabidopsis thaliana, Fragaria vesca and Rosa chinensis with a prediction accuracy over 90%. In addition, we find that (1) 6mA tends to occur at GAGG motifs, which means the sequence near the 6mA site may be conservative; (2) 6mA is enriched in the TATA box of the promoter, which may be the main source of its regulating downstream gene expression.

Download Full-text

Prostate and dominant intraprostatic lesion segmentation on PET/CT using cascaded regional-net

Physics in Medicine and Biology ◽

10.1088/1361-6560/ac3c13 ◽

2021 ◽

Author(s):

Luke A Matkovic ◽

Tonghe Wang ◽

Yang Lei ◽

Oladunni O Akin-Akintayo ◽

Olayinka A Abiodun Ojo ◽

...

Keyword(s):

Radiation Therapy ◽

Cross Validation ◽

Ct Images ◽

Second Step ◽

Prostate Segmentation ◽

Positron Emission ◽

Pet Ct ◽

The Mean ◽

And Training ◽

Fold Cross Validation

Abstract Focal dose boost to dominant intraprostatic lesions (DILs) has recently been proposed for prostate radiation therapy. Accurate and fast delineation of the prostate and DILs is thus required during treatment planning. We propose a learning-based method using positron emission tomography (PET)/computed tomography (CT) images to automatically segment the prostate and its DILs. To enable end-to-end segmentation, a deep learning-based method, called cascaded regional-Net, is utilized. The first network, referred to as dual attention network (DAN), is used to segment the prostate via extracting comprehensive features from both PET and CT images. A second network, referred to as mask scoring regional convolutional neural network (MSR-CNN), is used to segment the DILs from the PET and CT within the prostate region. Scoring strategy is used to diminish the misclassification of the DILs. For DIL segmentation, the proposed cascaded regional-Net uses two steps to remove normal tissue regions, with the first step cropping images based on prostate segmentation and the second step using MSR-CNN to further locate the DILs. The binary masks of DILs and prostates of testing patients are generated from PET/CT by the trained network. To evaluate the proposed method, we retrospectively investigated 49 PET/CT datasets. On each dataset, the prostate and DILs were delineated by physicians and set as the ground truths and training targets. The proposed method was trained and evaluated using a five-fold cross-validation and a hold-out test. The mean surface distance and DSC values were 0.666±0.696mm and 0.932±0.059 for the prostate and 1.209±1.954mm and 0.757±0.241 for the DILs among all 49 patients. The proposed method has demonstrated great potential for improving the efficiency and reducing the observer variability of prostate and DIL contouring for DIL focal boost prostate radiation therapy.

Download Full-text