Deep learning approach for automatic landmark detection and alignment analysis in whole-spine lateral radiographs

AbstractHuman spinal balance assessment relies considerably on sagittal radiographic parameter measurement. Deep learning could be applied for automatic landmark detection and alignment analysis, with mild to moderate standard errors and favourable correlations with manual measurement. In this study, based on 2210 annotated images of various spinal disease aetiologies, we developed deep learning models capable of automatically locating 45 anatomic landmarks and subsequently generating 18 radiographic parameters on a whole-spine lateral radiograph. In the assessment of model performance, the localisation accuracy and learning speed were the highest for landmarks in the cervical area, followed by those in the lumbosacral, thoracic, and femoral areas. All the predicted radiographic parameters were significantly correlated with ground truth values (all p < 0.001). The human and artificial intelligence comparison revealed that the deep learning model was capable of matching the reliability of doctors for 15/18 of the parameters. The proposed automatic alignment analysis system was able to localise spinal anatomic landmarks with high accuracy and to generate various radiographic parameters with favourable correlations with manual measurements.

Download Full-text

Automatic Detection of the Aortic Annular Plane and Coronary Ostia from Multidetector Computed Tomography

Journal of Interventional Cardiology ◽

10.1155/2020/9843275 ◽

2020 ◽

Vol 2020 ◽

pp. 1-9 ◽

Cited By ~ 1

Author(s):

Patricio Astudillo ◽

Peter Mortier ◽

Johan Bosmans ◽

Ole De Backer ◽

Peter de Jaegere ◽

...

Keyword(s):

Computed Tomography ◽

Deep Learning ◽

Aortic Valve ◽

Multidetector Computed Tomography ◽

Preoperative Planning ◽

Ground Truth ◽

Anatomical Landmarks ◽

Landmark Detection ◽

Mdct Image ◽

Coronary Ostia

Anatomic landmark detection is crucial during preoperative planning of transcatheter aortic valve implantation (TAVI) to select the proper device size and assess the risk of complications. The detection is currently a time-consuming manual process influenced by the image quality and subject to operator variability. In this work, we propose a novel automatic method to detect the relevant aortic landmarks from MDCT images using deep learning techniques. We trained three convolutional neural networks (CNNs) with 344 multidetector computed tomography (MDCT) acquisitions to detect five anatomical landmarks relevant for TAVI planning: the three basal attachment points of the aortic valve leaflets and the left and right coronary ostia. The detection strategy used these three CNN models to analyse a single MDCT image and yield three segmentation volumes as output. These segmentation volumes were averaged into one final segmentation volume, and the final predicted landmarks were obtained during a postprocessing step. Finally, we constructed the aortic annular plane, defined by the three predicted hinge points, and measured the distances from this plane to the predicted coronary ostia (i.e., coronary height). The methodology was validated on 100 patients. The automatic landmark detection was able to detect all the landmarks and showed high accuracy as the median distance between the ground truth and predictions is lower than the interobserver variations (1.5 mm [1.1–2.1], 2.0 mm [1.3–2.8] with a paired difference −0.5 ± 1.3 mm and p value <0.001). Furthermore, a high correlation is observed between predicted and manually measured coronary heights (for both R2 = 0.8). The image analysis time per patient was below one second. The proposed method is accurate, fast, and reproducible. Embedding this tool based on deep learning in the preoperative planning routine may have an impact in the TAVI environments by reducing the time and cost and improving accuracy.

Download Full-text

Neural Supermodeling

10.5194/egusphere-egu2020-12601 ◽

2020 ◽

Author(s):

Wim Wiegerinck

Keyword(s):

Deep Learning ◽

Domain Knowledge ◽

Model Performance ◽

Ground Truth ◽

Training Data ◽

Combination Method ◽

Neural Network Approach ◽

Modeling Approach ◽

Promising Tool ◽

Spatially Extended

Deep learning is a modeling approach that has shown impressive results in image processing and is arguably a promising tool for dealing with spatially extended complex systems such earth atmosphere with its visually interpretable patterns. A disadvantage of the neural network approach is that it typically requires an enormous amount of training data.&#160;Another recently proposed modeling approach is supermodeling. In supermodeling it is assumed that a dynamical system &#8211; the truth &#8211; is modelled by a set of good but imperfect models. The idea is to improve model performance by dynamically combining imperfect models during the simulation. The resulting combination of models is called the supermodel. The combination strength has to be learned from data. However, since supermodels do not start from scratch, but make use of existing domain knowledge, they may learn from less data.&#160;One of the ways to combine models is to define the tendencies of the supermodel as linear (weighted) combinations of the imperfect model tendencies. Several methods including linear regression have been proposed to optimize the weights. &#160;However, the combination method might also be nonlinear. In this work we propose and explore a novel combination of deep learning and supermodeling, in which convolutional neural networks are used as tool to combine the predictions of the imperfect models. &#160;The different supermodeling strategies are applied in simulations in a controlled environment with a three-level, quasi-geostrophic spectral model that serves as ground truth and perturbed models that serve as the imperfect models.

Download Full-text

Accuracy of Trained Physicians is Inferior to Deep Learning-Based Algorithm for Determining Angles in Ultrasound of the Newborn Hip

Ultraschall in der Medizin - European Journal of Ultrasound ◽

10.1055/a-1177-0480 ◽

2020 ◽

Author(s):

David Oelen ◽

Pascal Kaiser ◽

Thomas Baumann ◽

Raoul Schmid ◽

Christof Bühler ◽

...

Keyword(s):

Deep Learning ◽

Correct Diagnosis ◽

Accurate Determination ◽

Alpha Angle ◽

Ground Truth ◽

Developmental Dysplasia ◽

Daily Routine ◽

Base Line ◽

Ultrasound Images ◽

Landmark Detection

Abstract Purpose Sonographic diagnosis of developmental dysplasia of the hip allows treatment with a flexion-abduction orthosis preventing hip luxation. Accurate determination of alpha and beta angles according to Graf is crucial for correct diagnosis. It is unclear if algorithms could predict the angles. We aimed to compare the accuracy for users and automation reporting root mean squared errors (RMSE). Materials and Methods We used 303 306 ultrasound images of newborn hips collected between 2009 and 2016 in screening consultations. Trained physicians labelled every second image with alpha and beta angles during the consultations. A random subset of images was labeled with time and precision under lab conditions as ground truth. Automation predicted the two angles using a convolutional neural network (CNN). The analysis was focused on the alpha angle. Results Three methods were implemented, each with a different abstraction of the problem: (1) CNNs that directly learn the angles without any post-processing steps; (2) CNNs that return the relevant landmarks in the image to identify the angles; (3) CNNs that return the base line, bony roof line, and the cartilage roof line which are necessary to calculate the angles. The RMSE between physicians and ground truth were found to be 7.1° for alpha. The best CNN architecture was (2) landmark detection. The RMSE between landmark detection and ground truth was 3.9° for alpha. Conclusion The accuracy of physicians in their daily routine is inferior to deep learning-based algorithms for determining angles in ultrasound of the newborn hip. Similar methods could be used to support physicians.

Download Full-text

Impact of image compression on deep learning-based mammogram classification

Scientific Reports ◽

10.1038/s41598-021-86726-w ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Yong-Yeon Jo ◽

Young Sang Choi ◽

Hyun Woo Park ◽

Jae Hyeok Lee ◽

Hyojung Jung ◽

...

Keyword(s):

Deep Learning ◽

Image Compression ◽

Prediction Models ◽

Characteristic Curve ◽

Model Performance ◽

Ground Truth ◽

Medical Intervention ◽

Cancer Center ◽

Compact Representation ◽

The Impact

AbstractImage compression is used in several clinical organizations to help address the overhead associated with medical imaging. These methods reduce file size by using a compact representation of the original image. This study aimed to analyze the impact of image compression on the performance of deep learning-based models in classifying mammograms as “malignant”—cases that lead to a cancer diagnosis and treatment—or “normal” and “benign,” non-malignant cases that do not require immediate medical intervention. In this retrospective study, 9111 unique mammograms–5672 normal, 1686 benign, and 1754 malignant cases were collected from the National Cancer Center in the Republic of Korea. Image compression was applied to mammograms with compression ratios (CRs) ranging from 15 to 11 K. Convolutional neural networks (CNNs) with three convolutional layers and three fully-connected layers were trained using these images to classify a mammogram as malignant or not malignant across a range of CRs using five-fold cross-validation. Models trained on images with maximum CRs of 5 K had an average area under the receiver operating characteristic curve (AUROC) of 0.87 and area under the precision-recall curve (AUPRC) of 0.75 across the five folds and compression ratios. For images compressed with CRs of 10 K and 11 K, model performance decreased (average 0.79 in AUROC and 0.49 in AUPRC). Upon generating saliency maps that visualize the areas each model views as significant for prediction, models trained on less compressed (CR < = 5 K) images had maps encapsulating a radiologist’s label, while models trained on images with higher amounts of compression had maps that missed the ground truth completely. In addition, base ResNet18 models pre-trained on ImageNet and trained using compressed mammograms did not show performance improvements over our CNN model, with AUROC and AUPRC values ranging from 0.77 to 0.87 and 0.52 to 0.71 respectively when trained and tested on images with maximum CRs of 5 K. This paper finds that while training models on images with increased the robustness of the models when tested on compressed data, moderate image compression did not substantially impact the classification performance of DL-based models.

Download Full-text

Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy

Scientific Reports ◽

10.1038/s41598-021-00557-3 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Yuta Kumazu ◽

Nao Kobayashi ◽

Naoki Kitamura ◽

Elleuch Rayan ◽

Paul Neculoiu ◽

...

Keyword(s):

Deep Learning ◽

Connective Tissue ◽

Cognitive Skills ◽

Model Performance ◽

Ground Truth ◽

Learning Model ◽

Robot Assisted ◽

Loose Connective Tissue ◽

Anatomical Structures ◽

Deep Learning Model

AbstractThe prediction of anatomical structures within the surgical field by artificial intelligence (AI) is expected to support surgeons’ experience and cognitive skills. We aimed to develop a deep-learning model to automatically segment loose connective tissue fibers (LCTFs) that define a safe dissection plane. The annotation was performed on video frames capturing a robot-assisted gastrectomy performed by trained surgeons. A deep-learning model based on U-net was developed to output segmentation results. Twenty randomly sampled frames were provided to evaluate model performance by comparing Recall and F1/Dice scores with a ground truth and with a two-item questionnaire on sensitivity and misrecognition that was completed by 20 surgeons. The model produced high Recall scores (mean 0.606, maximum 0.861). Mean F1/Dice scores reached 0.549 (range 0.335–0.691), showing acceptable spatial overlap of the objects. Surgeon evaluators gave a mean sensitivity score of 3.52 (with 88.0% assigning the highest score of 4; range 2.45–3.95). The mean misrecognition score was a low 0.14 (range 0–0.7), indicating very few acknowledged over-detection failures. Thus, AI can be trained to predict fine, difficult-to-discern anatomical structures at a level convincing to expert surgeons. This technology may help reduce adverse events by determining safe dissection planes.

Download Full-text

Development of a High-Performance Multiparametric MRI Oropharyngeal Primary Tumor Auto-Segmentation Deep Learning Model and Investigation of Input Channel Effects: Results from a Prospective Imaging Registry

10.1101/2021.07.27.21261114 ◽

2021 ◽

Author(s):

Kareem Wahid ◽

Sara Ahmed ◽

Renjie He ◽

Lisanne van Dijk ◽

Jonas Teuwen ◽

...

Keyword(s):

Deep Learning ◽

High Performance ◽

False Negative ◽

Model Performance ◽

Ground Truth ◽

Turing Test ◽

Multiparametric Mri ◽

Dice Similarity Coefficient ◽

Channel Effects ◽

The Impact

Background and Purpose: Oropharyngeal cancer (OPC) primary gross tumor volume (GTVp) segmentation is crucial for radiotherapy. Multiparametric MRI (mpMRI) is increasingly used for OPC adaptive radiotherapy but relies on manual segmentation. Therefore, we constructed mpMRI deep learning (DL) OPC GTVp auto-segmentation models and determined the impact of input channels on segmentation performance. Materials and Methods: GTVp ground truth segmentations were manually generated for 30 OPC patients from a clinical trial. We evaluated five mpMRI input channels (T2, T1, ADC, Ktrans, Ve). 3D Residual U-net models were developed and assessed using leave-one-out cross-validation. A baseline T2 model was compared to mpMRI models (T2+T1, T2+ADC, T2+Ktrans, T2+Ve, all 5 channels [ALL]) primarily using the Dice similarity coefficient (DSC). Sensitivity, positive predictive value, Hausdorff distance (HD), false-negative DSC (FND), false-positive DSC, surface DSC, 95% HD, and mean surface distance were also assessed. For the best model, ground truth and DL-generated segmentations were compared through a Turing test using physician observers. Results: Models yielded mean DSCs from 0.71 (ALL) to 0.73 (T2+T1). Compared to the T2 model, performance was significantly improved for HD, FND, sensitivity, surface DSC, and 95% HD for the T2+T1 model (p<0.05) and for FND for the T2+Ve and ALL models (p<0.05). There were no differences between ground truth and DL-generated segmentations for all observers (p>0.05). Conclusion: DL using mpMRI provides high-quality segmentations of OPC GTVp. Incorporating additional mpMRI channels may increase the performance of certain evaluation metrics. This pilot study is a promising step towards fully automated MR-guided OPC radiotherapy.

Download Full-text

COVLIAS 1.0: Lung Segmentation in COVID-19 Computed Tomography Scans Using Hybrid Deep Learning Artificial Intelligence Models

Diagnostics ◽

10.3390/diagnostics11081405 ◽

2021 ◽

Vol 11 (8) ◽

pp. 1405

Author(s):

Jasjit S. Suri ◽

Sushant Agarwal ◽

Rajesh Pathak ◽

Vedmanvitha Ketireddy ◽

Marta Columbu ◽

...

Keyword(s):

Computed Tomography ◽

Deep Learning ◽

Ground Truth ◽

P Value ◽

Image Analysis System ◽

Validation Data ◽

Lung Segmentation ◽

Intelligence Models ◽

Artificial Intelligence Models ◽

Analysis System

Background: COVID-19 lung segmentation using Computed Tomography (CT) scans is important for the diagnosis of lung severity. The process of automated lung segmentation is challenging due to (a) CT radiation dosage and (b) ground-glass opacities caused by COVID-19. The lung segmentation methodologies proposed in 2020 were semi- or automated but not reliable, accurate, and user-friendly. The proposed study presents a COVID Lung Image Analysis System (COVLIAS 1.0, AtheroPoint™, Roseville, CA, USA) consisting of hybrid deep learning (HDL) models for lung segmentation. Methodology: The COVLIAS 1.0 consists of three methods based on solo deep learning (SDL) or hybrid deep learning (HDL). SegNet is proposed in the SDL category while VGG-SegNet and ResNet-SegNet are designed under the HDL paradigm. The three proposed AI approaches were benchmarked against the National Institute of Health (NIH)-based conventional segmentation model using fuzzy-connectedness. A cross-validation protocol with a 40:60 ratio between training and testing was designed, with 10% validation data. The ground truth (GT) was manually traced by a radiologist trained personnel. For performance evaluation, nine different criteria were selected to perform the evaluation of SDL or HDL lung segmentation regions and lungs long axis against GT. Results: Using the database of 5000 chest CT images (from 72 patients), COVLIAS 1.0 yielded AUC of ~0.96, ~0.97, ~0.98, and ~0.96 (p-value < 0.001), respectively within 5% range of GT area, for SegNet, VGG-SegNet, ResNet-SegNet, and NIH. The mean Figure of Merit using four models (left and right lung) was above 94%. On benchmarking against the National Institute of Health (NIH) segmentation method, the proposed model demonstrated a 58% and 44% improvement in ResNet-SegNet, 52% and 36% improvement in VGG-SegNet for lung area, and lung long axis, respectively. The PE statistics performance was in the following order: ResNet-SegNet > VGG-SegNet > NIH > SegNet. The HDL runs in <1 s on test data per image. Conclusions: The COVLIAS 1.0 system can be applied in real-time for radiology-based clinical settings.

Download Full-text

A unified framework for automated person re-indentification

Transport and Communication Science Journal ◽

10.25073/tcsj.71.7.11 ◽

2020 ◽

Vol 71 (7) ◽

pp. 868-880

Author(s):

Nguyen Hong-Quan ◽

Nguyen Thuy-Binh ◽

Tran Duc-Long ◽

Le Thi-Lan

Keyword(s):

Deep Learning ◽

Video Analysis ◽

Camera Network ◽

Unified Framework ◽

Person Detection ◽

Practical Applications ◽

Detection And Tracking ◽

Analysis System ◽

Bounding Boxes

Along with the strong development of camera networks, a video analysis system has been become more and more popular and has been applied in various practical applications. In this paper, we focus on person re-identification (person ReID) task that is a crucial step of video analysis systems. The purpose of person ReID is to associate multiple images of a given person when moving in a non-overlapping camera network. Many efforts have been made to person ReID. However, most of studies on person ReID only deal with well-alignment bounding boxes which are detected manually and considered as the perfect inputs for person ReID. In fact, when building a fully automated person ReID system the quality of the two previous steps that are person detection and tracking may have a strong effect on the person ReID performance. The contribution of this paper are two-folds. First, a unified framework for person ReID based on deep learning models is proposed. In this framework, the coupling of a deep neural network for person detection and a deep-learning-based tracking method is used. Besides, features extracted from an improved ResNet architecture are proposed for person representation to achieve a higher ReID accuracy. Second, our self-built dataset is introduced and employed for evaluation of all three steps in the fully automated person ReID framework.

Download Full-text

Evaluating the accuracy and uncertainty of atmospheric and wave model hindcasts during severe events using model ensembles

Ocean Dynamics ◽

10.1007/s10236-020-01426-9 ◽

2021 ◽

Author(s):

Ali Abdolali ◽

Andre van der Westhuysen ◽

Zaizhong Ma ◽

Avichal Mehra ◽

Aron Roland ◽

...

Keyword(s):

Extreme Event ◽

Numerical Models ◽

Model Performance ◽

Ground Truth ◽

Wave Model ◽

Atmospheric Model ◽

Stationary Source ◽

Source Point ◽

Spectral Wave Model ◽

Model Ensembles

AbstractVarious uncertainties exist in a hindcast due to the inabilities of numerical models to resolve all the complicated atmosphere-sea interactions, and the lack of certain ground truth observations. Here, a comprehensive analysis of an atmospheric model performance in hindcast mode (Hurricane Weather and Research Forecasting model—HWRF) and its 40 ensembles during severe events is conducted, evaluating the model accuracy and uncertainty for hurricane track parameters, and wind speed collected along satellite altimeter tracks and at stationary source point observations. Subsequently, the downstream spectral wave model WAVEWATCH III is forced by two sets of wind field data, each includes 40 members. The first ones are randomly extracted from original HWRF simulations and the second ones are based on spread of best track parameters. The atmospheric model spread and wave model error along satellite altimeters tracks and at stationary source point observations are estimated. The study on Hurricane Irma reveals that wind and wave observations during this extreme event are within ensemble spreads. While both Models have wide spreads over areas with landmass, maximum uncertainty in the atmospheric model is at hurricane eye in contrast to the wave model.

Download Full-text

Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

Scientific Reports ◽

10.1038/s41598-020-78129-0 ◽

2020 ◽

Vol 10 (1) ◽

Author(s):

Young-Gon Kim ◽

Sungchul Kim ◽

Cristina Eunbee Cho ◽

In Hye Song ◽

Hee Jin Lee ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Frozen Section ◽

Medical Center ◽

External Validation ◽

Model Performance ◽

Classification Model ◽

Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

Download Full-text