Automated prostate lesion detection and PI-RADS assessment with deep learning.

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. e16605-e16605
Author(s):  
Choongheon Yoon ◽  
Jasper Van ◽  
Michelle Bardis ◽  
Param Bhatter ◽  
Alexander Ushinsky ◽  
...  
Keyword(s):  
Neural Network ◽  
Prostate Cancer ◽  
Neural Networks ◽  
Deep Learning ◽  
Lesion Detection ◽  
Dice Coefficient ◽  
Cohen’S Kappa ◽  
Transrectal Biopsy ◽  
Cohen's Kappa ◽  
Feature Pyramid

e16605 Background: Prostate Cancer is the most commonly diagnosed male cancer in the U.S. Multiparametric magnetic resonance imaging (mpMRI) is increasingly used for both prostate cancer evaluation and biopsy guidance. The PI-RADS v2 scoring paradigm was developed to stratify prostate lesions on MRI and to predict lesion grade. Prostate organ and lesion segmentation is an essential step in pre-biopsy surgical planning. Deep learning convolutional neural networks (CNN) for image recognition are becoming a more common method of machine learning. In this study, we develop a comprehensive deep learning pipeline of 3D/2D CNN based on U-Net architecture for automatic localization and segmentation of prostates, detection of prostate lesions and PI-RADS v2 lesion scoring of mpMRIs. Methods: This IRB approved retrospective review included a total of 303 prostate nodules from 217 patients who had a prostate mpMRI between September 2014 and December 2016 and an MR-guided transrectal biopsy. For each T2 weighted image, a board-certified abdominal radiologist manually segmented the prostate and each prostate lesion. The T2 weighted and ADC series were co-registered and each lesion was assigned an overall PI-RADS score, T2 weighted PI-RADS score, and ADC PI-RADS score. After a U-Net neural network segmented the prostate organ, a mask regional convolutional neural network (R-CNN) was applied. The mask R-CNN is composed of three neural networks: feature pyramid network, region proposal network, and head network. The mask R-CNN detected the prostate lesion, segmented it, and estimated its PI-RADS score. Instead, the mask R-CNN was implemented to regress along dimensions of the PI-RADS criteria. The mask R-CNN performance was assessed with AUC, Sørensen–Dice coefficient, and Cohen’s Kappa for PI-RADS scoring agreement. Results: The AUC for prostate nodule detection was 0.79. By varying detection thresholds, sensitivity/PPV were 0.94/.54 and 0.60/0.87 at either ends of the spectrum. For detected nodules, the segmentation Sørensen–Dice coefficient was 0.76 (0.72 – 0.80). Weighted Cohen’s Kappa for PI-RADS scoring agreement was 0.63, 0.71, and 0.51 for composite, T2 weighted, and ADC respectively. Conclusions: These results demonstrate the feasibility of implementing a comprehensive 3D/2D CNN-based deep learning pipeline for evaluation of prostate mpMRI. This method is highly accurate for organ segmentation. The results for lesion detection and categorization are modest; however, the PI-RADS v2 score accuracy is comparable to previously published human interobserver agreement.

scholarly journals Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks

SLEEP ◽  
2019 ◽  
Vol 42 (11) ◽  
Author(s):  
Linda Zhang ◽  
Daniel Fabbri ◽  
Raghu Upender ◽  
David Kent
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Sleep Stage ◽  
Health Study ◽  
Sleep Staging ◽  
Heart Health ◽  
Training Set ◽  
Cohen's Kappa ◽  
Sleep Stage Scoring

Abstract Study Objectives Polysomnography (PSG) scoring is labor intensive and suffers from variability in inter- and intra-rater reliability. Automated PSG scoring has the potential to reduce the human labor costs and the variability inherent to this task. Deep learning is a form of machine learning that uses neural networks to recognize data patterns by inspecting many examples rather than by following explicit programming. Methods A sleep staging classifier trained using deep learning methods scored PSG data from the Sleep Heart Health Study (SHHS). The training set was composed of 42 560 hours of PSG data from 5213 patients. To capture higher-order data, spectrograms were generated from electroencephalography, electrooculography, and electromyography data and then passed to the neural network. A holdout set of 580 PSGs not included in the training set was used to assess model accuracy and discrimination via weighted F1-score, per-stage accuracy, and Cohen’s kappa (K). Results The optimal neural network model was composed of spectrograms in the input layer feeding into convolutional neural network layers and a long short-term memory layer to achieve a weighted F1-score of 0.87 and K = 0.82. Conclusions The deep learning sleep stage classifier demonstrates excellent accuracy and agreement with expert sleep stage scoring, outperforming human agreement on sleep staging. It achieves comparable or better F1-scores, accuracy, and Cohen’s kappa compared to literature for automated sleep stage scoring of PSG epochs. Accurate automated scoring of other PSG events may eventually allow for fully automated PSG scoring.

scholarly journals Comparison of Regularized Reconstruction and Ordered Subset Expectation Maximization Reconstruction in the Diagnostics of Prostate Cancer Using Digital Time-of-Flight 68Ga-PSMA-11 PET/CT Imaging

Diagnostics ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. 630
Author(s):  
Olof Jonmarker ◽  
Rimma Axelsson ◽  
Ted Nilsson ◽  
Stefan Gabrielson
Keyword(s):  
Prostate Cancer ◽  
Bone Metastasis ◽  
Lymph Nodes ◽  
Expectation Maximization ◽  
Reconstruction Algorithm ◽  
Lesion Detection ◽  
Detection Methods ◽  
Cohen’S Kappa ◽  
Cohen's Kappa ◽  
Pet Ct

In prostate cancer, the early detection of distant spread has been shown to be of importance. Prostate-specific membrane antigen (PSMA)-binding radionuclides in positron emission tomography (PET) is a promising method for precise disease staging. PET diagnostics depend on image reconstruction techniques, and ordered subset expectation maximization (OSEM) is the established standard. Block sequential regularized expectation maximization (BSREM) is a more recent reconstruction algorithm and may produce fewer equivocal findings and better lesion detection. Methods: 68Ga PSMA-11 PET/CT scans of patients with de novo or suspected recurrent prostate cancer were retrospectively reformatted using both the OSEM and BSREM algorithms. The lesions were counted and categorized by three radiologists. The intra-class correlation (ICC) and Cohen’s kappa for the inter-rater reliability were calculated. Results: Sixty-one patients were reviewed. BSREM identified slightly fewer lesions overall and fewer equivocal findings. ICC was excellent with regards to definitive lymph nodes and bone metastasis identification and poor with regards to equivocal metastasis irrespective of the reconstruction algorithm. The median Cohen’s kappa were 0.66, 0.74, 0.61 and 0.43 for OSEM and 0.61, 0.63, 0.66 and 0.53 for BSREM, with respect to the tumor, local lymph nodes, metastatic lymph nodes and bone metastasis detection, respectively. Conclusions: BSREM in the setting of 68Ga PMSA PET staging or restaging is comparable to OSEM.

scholarly journals Expert-level automated sleep staging of long-term scalp electroencephalography recordings using deep learning

SLEEP ◽  
2020 ◽  
Vol 43 (11) ◽  
Author(s):  
Maurice Abou Jaoude ◽  
Haoqi Sun ◽  
Kyle R Pellerin ◽  
Milena Pavlova ◽  
Rani A Sarkis ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Learning Algorithm ◽  
Sleep Stages ◽  
Sleep Staging ◽  
Cohen’S Kappa ◽  
Scalp Eeg ◽  
Cohen's Kappa ◽  
Eeg Recordings

Abstract Study Objectives Develop a high-performing, automated sleep scoring algorithm that can be applied to long-term scalp electroencephalography (EEG) recordings. Methods Using a clinical dataset of polysomnograms from 6,431 patients (MGH–PSG dataset), we trained a deep neural network to classify sleep stages based on scalp EEG data. The algorithm consists of a convolutional neural network for feature extraction, followed by a recurrent neural network that extracts temporal dependencies of sleep stages. The algorithm’s inputs are four scalp EEG bipolar channels (F3-C3, C3-O1, F4-C4, and C4-O2), which can be derived from any standard PSG or scalp EEG recording. We initially trained the algorithm on the MGH–PSG dataset and used transfer learning to fine-tune it on a dataset of long-term (24–72 h) scalp EEG recordings from 112 patients (scalpEEG dataset). Results The algorithm achieved a Cohen’s kappa of 0.74 on the MGH–PSG holdout testing set and cross-validated Cohen’s kappa of 0.78 after optimization on the scalpEEG dataset. The algorithm also performed well on two publicly available PSG datasets, demonstrating high generalizability. Performance on all datasets was comparable to the inter-rater agreement of human sleep staging experts (Cohen’s kappa ~ 0.75 ± 0.11). The algorithm’s performance on long-term scalp EEGs was robust over a wide age range and across common EEG background abnormalities. Conclusion We developed a deep learning algorithm that achieves human expert level sleep staging performance on long-term scalp EEG recordings. This algorithm, which we have made publicly available, greatly facilitates the use of large long-term EEG clinical datasets for sleep-related research.

Deep learning hybrid 3D/2D convolutional neural network for prostate MRI recognition.

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. e16600-e16600 ◽  
Author(s):  
Jasper Van ◽  
Choongheon Yoon ◽  
Justin Glavis-Bloom ◽  
Michelle Bardis ◽  
Alexander Ushinsky ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Deep Learning ◽  
Pearson Correlation ◽  
The United States ◽  
Dice Coefficient ◽  
Prostate Mri ◽  
Transrectal Biopsy ◽  
Valuable Adjunct ◽  
Pearson Coefficient ◽  
Segmented Images

e16600 Background: Prostate cancer is the most common cancer of men in the United States, with over 200,000 new cases diagnosed in 2018. Multiparametric MRI of the prostate (mpMRI) has emerged as valuable adjunct for the detection and characterization of prostate cancer as well as for guidance of prostate biopsy. As mpMRI progresses towards widespread clinical use, major challenges have been identified, arising from the need to increase accuracy of mpMRI localization of prostate lesions, improve in lesion categorization, and decrease the time and technical complexity of mpMRI evaluation by radiologists or urologists. Deep learning convolutional neural networks (CNN) for image recognition are becoming a more common method of machine learning and show promise in evaluation of complex medical imaging. In this study we describe a deep learning approach for automatic localization and segmentation of prostates organ on clinically acquired mpMRIs. Methods: This IRB approved retrospective review included patients who had a prostate MRI between September 2014 and August 2018 and an MR-guided transrectal biopsy. For each mpMRI the prostate was manually segmented by a board-certified abdominal radiologist on T2 weighted sequence. A hybrid 3D/2D CNN based on U-Net architecture was developed and trained using these manually segmented images to perform automated organ segmentation. After training, the CNN was used to produce prostate segmentations autonomously on clinical mpMRI. Accuracy of the CNN was assessed by Sørensen–Dice coefficient and Pearson coefficient. Five-fold validation was performed. Results: The CNN was successfully trained and five-fold validation performed on 411 prostate mpMRIs. The Sørensen–Dice coefficient from the five-fold cross validation was 0.87 and the Pearson correlation coefficient for segmented volume was 0.99. Conclusions: These results demonstrate that a CNN can be developed and trained to automatically localize and volumetrically segment the prostate on clinical mpMRI with high accuracy. This study supports the potential for developing an automated deep learning CNN for organ segmentation to replace clinical manual segmentation. Future studies will look towards prostate lesion localization and categorization on mpMRI.

scholarly journals Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

2021 ◽  
Vol 26 (1) ◽  
pp. 200-215
Author(s):  
Muhammad Alam ◽  
Jian-Feng Wang ◽  
Cong Guangpei ◽  
LV Yunrong ◽  
Yuanfang Chen
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Natural Scene ◽  
Remote Sensing Images ◽  
Advantages And Disadvantages ◽  
Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Classification of Shoulder X-ray Images with Deep Learning Ensemble Models

2021 ◽  
Vol 11 (6) ◽  
pp. 2723
Author(s):  
Fatih Uysal ◽  
Fırat Hardalaç ◽  
Ozan Peker ◽  
Tolga Tolunay ◽  
Nil Tokgöz
Keyword(s):  
Deep Learning ◽  
Performance Test ◽  
The Body ◽  
Test Accuracy ◽  
Cohen’S Kappa ◽  
X Ray ◽  
Cohen's Kappa ◽  
Auc Value ◽  
Magnetic Resonance Imaging Mri ◽  
Fully Connected

Fractures occur in the shoulder area, which has a wider range of motion than other joints in the body, for various reasons. To diagnose these fractures, data gathered from X-radiation (X-ray), magnetic resonance imaging (MRI), or computed tomography (CT) are used. This study aims to help physicians by classifying shoulder images taken from X-ray devices as fracture/non-fracture with artificial intelligence. For this purpose, the performances of 26 deep learning-based pre-trained models in the detection of shoulder fractures were evaluated on the musculoskeletal radiographs (MURA) dataset, and two ensemble learning models (EL1 and EL2) were developed. The pre-trained models used are ResNet, ResNeXt, DenseNet, VGG, Inception, MobileNet, and their spinal fully connected (Spinal FC) versions. In the EL1 and EL2 models developed using pre-trained models with the best performance, test accuracy was 0.8455, 0.8472, Cohen’s kappa was 0.6907, 0.6942 and the area that was related with fracture class under the receiver operating characteristic (ROC) curve (AUC) was 0.8862, 0.8695. As a result of 28 different classifications in total, the highest test accuracy and Cohen’s kappa values were obtained in the EL2 model, and the highest AUC value was obtained in the EL1 model.

Deep Learning–Based Automated Thrombolysis in Cerebral Infarction Scoring: A Timely Proof-of-Principle Study

Stroke ◽  
2021 ◽  
Author(s):  
Maximilian Nielsen ◽  
Moritz Waldmann ◽  
Andreas M. Frölich ◽  
Fabian Flottmann ◽  
Evelin Hristova ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cerebral Infarction ◽  
Digital Subtraction Angiography ◽  
Information Integration ◽  
Mechanical Thrombectomy ◽  
Model Updating ◽  
Digital Subtraction ◽  
Cohen’S Kappa ◽  
Expert Rating ◽  
Cohen's Kappa

Background and Purpose: Mechanical thrombectomy is an established procedure for treatment of acute ischemic stroke. Mechanical thrombectomy success is commonly assessed by the Thrombolysis in Cerebral Infarction (TICI) score, assigned by visual inspection of X-ray digital subtraction angiography data. However, expert-based TICI scoring is highly observer-dependent. This represents a major obstacle for mechanical thrombectomy outcome comparison in, for instance, multicentric clinical studies. Focusing on occlusions of the M1 segment of the middle cerebral artery, the present study aimed to develop a deep learning (DL) solution to automated and, therefore, objective TICI scoring, to evaluate the agreement of DL- and expert-based scoring, and to compare corresponding numbers to published scoring variability of clinical experts. Methods: The study comprises 2 independent datasets. For DL system training and initial evaluation, an in-house dataset of 491 digital subtraction angiography series and modified TICI scores of 236 patients with M1 occlusions was collected. To test the model generalization capability, an independent external dataset with 95 digital subtraction angiography series was analyzed. Characteristics of the DL system were modeling TICI scoring as ordinal regression, explicit consideration of the temporal image information, integration of physiological knowledge, and modeling of inherent TICI scoring uncertainties. Results: For the in-house dataset, the DL system yields Cohen’s kappa, overall accuracy, and specific agreement values of 0.61, 71%, and 63% to 84%, respectively, compared with the gold standard: the expert rating. Values slightly drop to 0.52/64%/43% to 87% when the model is, without changes, applied to the external dataset. After model updating, they increase to 0.65/74%/60% to 90%. Literature Cohen’s kappa values for expert-based TICI scoring agreement are in the order of 0.6. Conclusions: The agreement of DL- and expert-based modified TICI scores in the range of published interobserver variability of clinical experts highlights the potential of the proposed DL solution to automated TICI scoring.

Searching for new physics: Using explainable AI to understand deep learned parameterizations of turbulent heat fluxes

2021 ◽  
Author(s):  
Andrew Bennett ◽  
Bart Nijssen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Coupled System ◽  
Hydrologic Model ◽  
Heat Fluxes ◽  
Turbulent Heat ◽  
Phase Lag ◽  
Geophysical Research ◽  
Turbulent Heat Fluxes

<p>Machine learning (ML), and particularly deep learning (DL), for geophysical research has shown dramatic successes in recent years. However, these models are primarily geared towards better predictive capabilities, and are generally treated as black box models, limiting researchers’ ability to interpret and understand how these predictions are made. As these models are incorporated into larger models and pushed to be used in more areas it will be important to build methods that allow us to reason about how these models operate. This will have implications for scientific discovery that will ensure that these models are robust and reliable for their respective applications. Recent work in explainable artificial intelligence (XAI) has been used to interpret and explain the behavior of machine learned models.</p><p>Here, we apply new tools from the field of XAI to provide physical interpretations of a system that couples a deep-learning based parameterization for turbulent heat fluxes to a process based hydrologic model. To develop this coupling we have trained a neural network to predict turbulent heat fluxes using FluxNet data from a large number of hydroclimatically diverse sites. This neural network is coupled to the SUMMA hydrologic model, taking imodel derived states as additional inputs to improve predictions. We have shown that this coupled system provides highly accurate simulations of turbulent heat fluxes at 30 minute timesteps, accurately predicts the long-term observed water balance, and reproduces other signatures such as the phase lag with shortwave radiation. Because of these features, it seems this coupled system is learning physically accurate relationships between inputs and outputs. </p><p>We probe the relative importance of which input features are used to make predictions during wet and dry conditions to better understand what the neural network has learned. Further, we conduct controlled experiments to understand how the neural networks are able to learn to regionalize between different hydroclimates. By understanding how these neural networks make their predictions as well as how they learn to make predictions we can gain scientific insights and use them to further improve our models of the Earth system.</p>

scholarly journals Semiotic Aggregation in Deep Learning

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1365
Author(s):  
Bogdan Muşat ◽  
Răzvan Andonie
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Decision Model ◽  
Deep Neural Networks ◽  
Neural Model ◽  
Network Layers ◽  
Saliency Maps ◽  
Spatial Entropy ◽  
Insight Into

Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In our experiments, we visualize the superization process and show how the obtained knowledge can be used to explain the neural decision model. In addition, we attempt to optimize the architecture of the neural model employing a semiotic greedy technique. To the extent of our knowledge, this is the first application of computational semiotics in the analysis and interpretation of deep neural networks.

Sign in / Sign up

Export Citation Format

Share Document