Diagnosis of Thyroid Nodules: Performance of a Deep Learning Convolutional Neural Network Model vs. Radiologists

AbstractComputer-aided diagnosis (CAD) systems hold potential to improve the diagnostic accuracy of thyroid ultrasound (US). We aimed to develop a deep learning-based US CAD system (dCAD) for the diagnosis of thyroid nodules and compare its performance with those of a support vector machine (SVM)-based US CAD system (sCAD) and radiologists. dCAD was developed by using US images of 4919 thyroid nodules from three institutions. Its diagnostic performance was prospectively evaluated between June 2016 and February 2017 in 286 nodules, and was compared with those of sCAD and radiologists, using logistic regression with the generalized estimating equation. Subgroup analyses were performed according to experience level and separately for small thyroid nodules 1–2 cm. There was no difference in overall sensitivity, specificity, positive predictive value (PPV), negative predictive value and accuracy (all p > 0.05) between radiologists and dCAD. Radiologists and dCAD showed higher specificity, PPV, and accuracy than sCAD (all p < 0.001). In small nodules, experienced radiologists showed higher specificity, PPV and accuracy than sCAD (all p < 0.05). In conclusion, dCAD showed overall comparable diagnostic performance with radiologists and assessed thyroid nodules more effectively than sCAD, without loss of sensitivity.

Download Full-text

Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis

European Thyroid Journal ◽

10.1159/000504390 ◽

2019 ◽

Vol 9 (4) ◽

pp. 186-193

Author(s):

Lei Xu ◽

Junling Gao ◽

Quan Wang ◽

Jichao Yin ◽

Pengfei Yu ◽

...

Keyword(s):

Machine Learning ◽

Systematic Review ◽

Deep Learning ◽

Diagnostic Performance ◽

Thyroid Nodules ◽

Computer Aided Diagnosis ◽

Cad Systems ◽

Cad System ◽

Computer Aided ◽

Aided Diagnosis

Background: Computer-aided diagnosis (CAD) systems are being applied to the ultrasonographic diagnosis of malignant thyroid nodules, but it remains controversial whether the systems add any accuracy for radiologists. Objective: To determine the accuracy of CAD systems in diagnosing malignant thyroid nodules. Methods: PubMed, EMBASE, and the Cochrane Library were searched for studies on the diagnostic performance of CAD systems. The diagnostic performance was assessed by pooled sensitivity and specificity, and their accuracy was compared with that of radiologists. The present systematic review was registered in PROSPERO (CRD42019134460). Results: Nineteen studies with 4,781 thyroid nodules were included. Both the classic machine learning- and the deep learning-based CAD system had good performance in diagnosing malignant thyroid nodules (classic machine learning: sensitivity 0.86 [95% CI 0.79–0.92], specificity 0.85 [95% CI 0.77–0.91], diagnostic odds ratio (DOR) 37.41 [95% CI 24.91–56.20]; deep learning: sensitivity 0.89 [95% CI 0.81–0.93], specificity 0.84 [95% CI 0.75–0.90], DOR 40.87 [95% CI 18.13–92.13]). The diagnostic performance of the deep learning-based CAD system was comparable to that of the radiologists (sensitivity 0.87 [95% CI 0.78–0.93] vs. 0.87 [95% CI 0.85–0.89], specificity 0.85 [95% CI 0.76–0.91] vs. 0.87 [95% CI 0.81–0.91], DOR 40.12 [95% CI 15.58–103.33] vs. DOR 44.88 [95% CI 30.71–65.57]). Conclusions: The CAD systems demonstrated good performance in diagnosing malignant thyroid nodules. However, experienced radiologists may still have an advantage over CAD systems during real-time diagnosis.

Download Full-text

Development and Validation of a Deep Learning Algorithm to Automatic Detection of Pituitary Microadenoma From MRI

Frontiers in Medicine ◽

10.3389/fmed.2021.758690 ◽

2021 ◽

Vol 8 ◽

Author(s):

Qingling Li ◽

Yanhua Zhu ◽

Minglin Chen ◽

Ruomi Guo ◽

Qingyong Hu ◽

...

Keyword(s):

Deep Learning ◽

Diagnostic Accuracy ◽

Roc Curve ◽

Predictive Value ◽

Computer Aided Diagnosis ◽

Validation Dataset ◽

Cad System ◽

Pituitary Microadenoma ◽

Computer Aided ◽

Aided Diagnosis

Background: It is often difficult to diagnose pituitary microadenoma (PM) by MRI alone, due to its relatively small size, variable anatomical structure, complex clinical symptoms, and signs among individuals. We develop and validate a deep learning -based system to diagnose PM from MRI.Methods: A total of 11,935 infertility participants were initially recruited for this project. After applying the exclusion criteria, 1,520 participants (556 PM patients and 964 controls subjects) were included for further stratified into 3 non-overlapping cohorts. The data used for the training set were derived from a retrospective study, and in the validation dataset, prospective temporal and geographical validation set were adopted. A total of 780 participants were used for training, 195 participants for testing, and 545 participants were used to validate the diagnosis performance. The PM-computer-aided diagnosis (PM-CAD) system consists of two parts: pituitary region detection and PM diagnosis. The diagnosis performance of the PM-CAD system was measured using the receiver operating characteristics (ROC) curve and area under the ROC curve (AUC), calibration curve, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1-score.Results: Pituitary microadenoma-computer-aided diagnosis system showed 94.36% diagnostic accuracy and 98.13% AUC score in the testing dataset. We confirm the robustness and generalization of our PM-CAD system, the diagnostic accuracy in the internal dataset was 96.50% and in the external dataset was 92.26 and 92.36%, the AUC was 95.5, 94.7, and 93.7%, respectively. In human-computer competition, the diagnosis performance of our PM-CAD system was comparable to radiologists with >10 years of professional expertise (diagnosis accuracy of 94.0% vs. 95.0%, AUC of 95.6% vs. 95.0%). For the misdiagnosis cases from radiologists, our system showed a 100% accurate diagnosis. A browser-based software was designed to assist the PM diagnosis.Conclusions: This is the first report showing that the PM-CAD system is a viable tool for detecting PM. Our results suggest that the PM-CAD system is applicable to radiology departments, especially in primary health care institutions.

Download Full-text

The value of S-Detect in improving the diagnostic performance of radiologists for the differential diagnosis of thyroid nodules

Medical Ultrasonography ◽

10.11152/mu-2501 ◽

2020 ◽

Vol 22 (4) ◽

pp. 415

Author(s):

Qi Wei ◽

Shu-E Zeng ◽

Li-Ping Wang ◽

Yu-Jing Yan ◽

Ting Wang ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Predictive Value ◽

Diagnostic Performance ◽

Thyroid Nodules ◽

Characteristic Curve ◽

Diagnostic Value ◽

Ultrasound Images ◽

Experienced Radiologist ◽

Before And After ◽

Sensitivity Specificity

Aims: To compare the diagnostic value of S-Detect (a computer aided diagnosis system using deep learning) in differentiating thyroid nodules in radiologists with different experience and to assess if S-Detect can improve the diagnostic performance of radiologists.Materials and methods: Between February 2018 and October 2019, 204 thyroid nodules in 181 patients were included. An experienced radiologist performed ultrasound for thyroid nodules and obtained the result of S-Detect. Four radiologists with different experience on thyroid ultrasound (Radiologist 1, 2, 3, 4 with 1, 4, 9, 20 years, respectively) analyzed the conventional ultrasound images of each thyroid nodule and made a diagnosis of “benign” or “malignant” based on the TI-RADS category. After referring to S-Detect results, they re-evaluated the diagnoses. The diagnostic performance of radiologists was analyzed before and after referring to the results of S-Detect.Results: The accuracy, sensitivity, specificity, positive predictive value and negative predictive value of S-Detect were 77.0, 91.3, 65.2, 68.3 and 90.1%, respectively. In comparison with the less experienced radiologists (radiologist 1 and 2), S-Detect had a higher area under receiver operating characteristic curve (AUC), accuracy and specificity (p <0.05). In comparison with the most experienced radiologist, the diagnostic accuracy and AUC were lower (p<0.05). In the less experienced radiologists, the diagnostic accuracy, specificity and AUC were significantly improved when combined with S-Detect (p<0.05), but not for experienced radiologists (radiologist 3 and 4) (p>0.05).Conclusions: S-Detect may become an additional diagnostic method for the diagnosis of thyroid nodules and improve the diagnostic performance of less experienced radiologists.

Download Full-text

Fast Deep Learning Computer-Aided Diagnosis against the Novel COVID-19 pandemic from Digital Chest X-ray Images

10.21203/rs.3.rs-36353/v1 ◽

2020 ◽

Cited By ~ 3

Author(s):

Mugahed A. Al-antari ◽

Cam-Hao Hua ◽

Sungyoung Lee

Keyword(s):

Deep Learning ◽

Lung Disease ◽

Lung Diseases ◽

Diagnostic Performance ◽

The Novel ◽

X Ray ◽

Cad System ◽

Computer Aided ◽

Chest X Ray ◽

Aided Diagnosis

Abstract Background and Objective: The novel coronavirus 2019 (COVID-19) is a harmful lung disease that rapidly attacks people worldwide. At the end of 2019, COVID-19 was discovered as mysterious lung disease in Wuhan, Hubei province of China. World health organization (WHO) declared the coronavirus outbreak a pandemic in the second week of March 2020. Simultaneous deep learning detection and classification of COVID-19 from the entire digital X-ray images is the key to efficiently assist patients and physicians for a fast and accurate diagnosis.Methods: In this paper, a deep learning computer-aided diagnosis (CAD) based on the YOLO predictor is proposed to simultaneously detect and diagnose COVID-19 among the other eight lung diseases: Atelectasis, Infiltration, Pneumothorax, Mass, Effusion, Pneumonia, Cardiomegaly, and Nodule. The proposed CAD system is assessed via five-fold tests for multi-class prediction problem using two different databases of chest X-ray images: COVID-19 and ChestX-ray8. The proposed CAD system is trained using an annotated training set of 50,490 chest X-ray images.Results: The suspicious regions of COVID-19 from the entire X-ray images are simultaneously detected and classified end-to-end via the proposed CAD predictor achieving overall detection and classification accuracies of 96.31% and 97.40%, respectively. The most testing images of COVID-19 and other lunge diseases are correctly predicted achieving intersection over union (IoU) with their GTs greater than 90%. Applying deep learning regularizers of data balancing and augmentation improve the diagnostic performance by 6.64% and 12.17% in terms of overall accuracy and F1-score, respectively. Meanwhile, the proposed CAD system presents its feasibility to diagnose the individual chest X-ray image within 0.009 second. Thus, the presented CAD system could predict 108 frames/second (FPS) at the real-time of prediction.Conclusion: The proposed deep learning CAD system shows its capability and reliability to achieve promising COVID-19 diagnostic performance among all other lung diseases. The proposed deep learning model seems reliable to assist health care systems, patients, and physicians in their practical validations.

Download Full-text

Qualitative analysis of contrast-enhanced ultrasound in the diagnosis of small, TR3–5 benign and malignant thyroid nodules measuring ≤1 cm

British Journal of Radiology ◽

10.1259/bjr.20190923 ◽

2020 ◽

Vol 93 (1111) ◽

pp. 20190923

Author(s):

Xin Li ◽

Feng Gao ◽

Fan Li ◽

Xiao-xia Han ◽

Si-hui Shao ◽

...

Keyword(s):

Positive Predictive Value ◽

Qualitative Analysis ◽

Negative Predictive Value ◽

Predictive Value ◽

Diagnostic Performance ◽

Thyroid Nodules ◽

Area Under The Curve ◽

Contrast Enhanced Ultrasound ◽

Contrast Enhanced ◽

Sensitivity Specificity

Objective: To evaluate the performance of contrast-enhanced ultrasound in the diagnosis of small, solid, TR3–5 benign and malignant thyroid nodules (≤1 cm). Methods: From January 2016 to March 2018, 185 thyroid nodules from 154 patients who underwent contrast enhanced ultrasound (CEUS) and fine-needle aspiration or thyroidectomy in Shanghai General Hospital were included. The χ2 test was used to compare the CEUS characteristics of benign and malignant thyroid nodules, and the CEUS features of malignant nodules assigned scores. The total score of the CEUS features and the scores of the above nodules were evaluated according to the latest 2017 version of the Thyroid Imaging Reporting and Data System (TI-RADS). The diagnostic performance of the two were compared based on the receiver operating characteristic curves generated for benign and malignant thyroid nodules. Results: The degree, enhancement patterns, boundary, shape, and homogeneity of enhancement in thyroid small solid nodules were significantly different (p＜0.05). No significant differences were seen between benign and malignant thyroid nodules regarding completeness of enhancement and size of enhanced lesions (p＞0.05). The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of the TI-RADS classification TR5 in diagnosis of malignant nodules were 90.10%, 55.95%, 74.59%, 72.22%, and 82.46%, respectively (area under the curve [AUC]=0.738; 95% confidence interval[CI], 0.663–0.813). The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of the total score of CEUS qualitative analysis indicators were 86.13%, 89.29%, 87.57%, 90.63%, and 84.27% respectively (AUC = 0.916; 95% CI, 0.871–0.961). Conclusion: CEUS qualitative analysis is superior to TI-RADS in evaluating the diagnostic performance of small, solid thyroid nodules. Qualitative analysis of CEUS has a significantly higher specificity for diagnosis of malignant thyroid nodules than TI-RADS. Advances in knowledge: The 2017 version of TI-RADS has recently suggested the malignant stratification of thyroid nodules by ultrasound. In this paper we applied this system and CEUS to evaluate 185 nodules and compare the results with pathological findings to access the diagnostic performance.

Download Full-text

Novel MRI-Based CAD System for Early Detection of Thyroid Cancer Using Multi-Input CNN

Sensors ◽

10.3390/s21113878 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3878

Author(s):

Ahmed Naglah ◽

Fahmi Khalifa ◽

Reem Khaled ◽

Ahmed Abdel Khalek Abdel Razek ◽

Mohammad Ghazal ◽

...

Keyword(s):

Thyroid Cancer ◽

Deep Learning ◽

Early Detection ◽

Thyroid Nodules ◽

Texture Features ◽

Additional Advantage ◽

Cad System ◽

Apparent Diffusion ◽

Using Data ◽

Aided Diagnosis

Early detection of thyroid nodules can greatly contribute to the prediction of cancer burdening and the steering of personalized management. We propose a novel multimodal MRI-based computer-aided diagnosis (CAD) system that differentiates malignant from benign thyroid nodules. The proposed CAD is based on a novel convolutional neural network (CNN)-based texture learning architecture. The main contribution of our system is three-fold. Firstly, our system is the first of its kind to combine T2-weighted MRI and apparent diffusion coefficient (ADC) maps using a CNN to model thyroid cancer. Secondly, it learns independent texture features for each input, giving it more advanced capabilities to simultaneously extract complex texture patterns from both modalities. Finally, the proposed system uses multiple channels for each input to combine multiple scans collected into the deep learning process using different values of the configurable diffusion gradient coefficient. Accordingly, the proposed system would enable the learning of more advanced radiomics with an additional advantage of visualizing the texture patterns after learning. We evaluated the proposed system using data collected from a cohort of 49 patients with pathologically proven thyroid nodules. The accuracy of the proposed system has also been compared against recent CNN models as well as multiple machine learning (ML) frameworks that use hand-crafted features. Our system achieved the highest performance among all compared methods with a diagnostic accuracy of 0.87, specificity of 0.97, and sensitivity of 0.69. The results suggest that texture features extracted using deep learning can contribute to the protocols of cancer diagnosis and treatment and can lead to the advancement of precision medicine.

Download Full-text

A Beneficial Role of Computer-aided Diagnosis System for Less Experienced Physicians in the Diagnosis of Thyroid Nodule on Ultrasound

10.21203/rs.3.rs-568240/v1 ◽

2021 ◽

Author(s):

Sunyoung Kang ◽

Eunjung Lee ◽

Chae Won Chung ◽

Han Na Jang ◽

Joon Ho Moon ◽

...

Keyword(s):

Diagnostic Performance ◽

Thyroid Nodules ◽

Needle Aspiration ◽

Computer Aided Diagnosis ◽

Ultrasound Images ◽

Cad System ◽

Computer Aided ◽

Before And After ◽

Needle Aspiration Cytology ◽

Aided Diagnosis

Abstract Ultrasonography is the primary diagnostic tool for thyroid nodules, while the accuracy is highly operator-dependent. The aim of this study was to investigate whether ultrasonography with computer-aided diagnosis (CAD) has assisting roles to physicians in the diagnosis of thyroid nodules. 451 thyroid nodules (³ 1 cm) evaluated by fine-needle aspiration cytology following surgery were included. 300 (66.5%) of them were diagnosed as malignancy. Thirteen physicians who had 0 months (E0, n=8), 1 year (E1, n=2), or more than 5 years (E5, n=3) of experience in ultrasonography reviewed the prepared ultrasound images of thyroid nodules before and after CAD assistance. The diagnostic performance of CAD was comparable to that of the E5 group, and better than those of the E0 and E1 groups. The AUC of the CAD for conventional PTC was higher than that for FTC and follicular variant PTC (0.925 vs. 0.499), independent of tumor size. CAD assistance significantly improved diagnostic performance in E0 group, but not in the E1 and E5 groups. In conclusion, the CAD system showed good performance in the diagnosis of conventional PTC. CAD assistance improved the diagnostic performance of physicians with less experience in ultrasonography, especially in the diagnosis of conventional PTC.

Download Full-text

Diagnostic performance evaluation of different TI-RADS using ultrasound computer-aided diagnosis of thyroid nodules: An experience with adjusted settings

PLoS ONE ◽

10.1371/journal.pone.0245617 ◽

2021 ◽

Vol 16 (1) ◽

pp. e0245617

Author(s):

Nonhlanhla Chambara ◽

Shirley Y. W. Liu ◽

Xina Lo ◽

Michael Ying

Keyword(s):

Diagnostic Performance ◽

Thyroid Nodules ◽

Computer Aided Diagnosis ◽

Ultrasound Images ◽

American Thyroid Association ◽

Korean Society ◽

Cad System ◽

Computer Aided ◽

Aided Diagnosis ◽

Default Setting

Background Thyroid cancer diagnosis has evolved to include computer-aided diagnosis (CAD) approaches to overcome the limitations of human ultrasound feature assessment. This study aimed to evaluate the diagnostic performance of a CAD system in thyroid nodule differentiation using varied settings. Methods Ultrasound images of 205 thyroid nodules from 198 patients were analysed in this retrospective study. AmCAD-UT software was used at default settings and 3 adjusted settings to diagnose the nodules. Six risk-stratification systems in the software were used to classify the thyroid nodules: The American Thyroid Association (ATA), American College of Radiology Thyroid Imaging, Reporting, and Data System (ACR-TIRADS), British Thyroid Association (BTA), European Union (EU-TIRADS), Kwak (2011) and the Korean Society of Thyroid Radiology (KSThR). The diagnostic performance of CAD was determined relative to the histopathology and/or cytology diagnosis of each nodule. Results At the default setting, EU-TIRADS yielded the highest sensitivity, 82.6% and lowest specificity, 42.1% while the ATA-TIRADS yielded the highest specificity, 66.4%. Kwak had the highest AUROC (0.74) which was comparable to that of ACR, ATA, and KSThR TIRADS (0.72, 0.73, and 0.70 respectively). At a hyperechoic foci setting of 3.5 with other settings at median values; ATA had the best-balanced sensitivity, specificity and good AUROC (70.4%; 67.3% and 0.71 respectively). Conclusion The default setting achieved the best diagnostic performance with all TIRADS and was best for maximizing the sensitivity of EU-TIRADS. Adjusting the settings by only reducing the sensitivity to echogenic foci may be most helpful for improving specificity with minimal change in sensitivity.

Download Full-text

Reliability of a computer-aided system in the evaluation of indeterminate ultrasound images of thyroid nodules

European Thyroid Journal ◽

10.1530/etj-21-0023 ◽

2022 ◽

Vol 11 (1) ◽

Author(s):

J L Reverter ◽

L Ferrer-Estopiñan ◽

F Vázquez ◽

S Ballesta ◽

S Batule ◽

...

Keyword(s):

Positive Predictive Value ◽

Risk Stratification ◽

Real World ◽

Predictive Value ◽

Diagnostic Performance ◽

Thyroid Nodules ◽

Ultrasound Images ◽

Cad System ◽

The Us ◽

Computer Aided

Introduction Computer-aided diagnostic (CAD) programs for malignancy risk stratification from ultrasound (US) imaging of thyroid nodules are being validated both experimentally and in real-world practice. However, they have not been tested for reliability in analyzing difficult or unclear images. Methods US images with indeterminate characteristics were evaluated by five observers with different experience in US examination and by a commercial CAD program. The nodules, on which the observers widely agreed, were considered concordant and, if there was little agreement, not concordant or difficult to assess. The diagnostic performance of the readers and the CAD program was calculated and compared in both groups of nodule images. Results In the group of concordant thyroid nodules (n = 37), the clinicians and the CAD system obtained similar levels of accuracy (77.0% vs 74.2%, respectively; P = 0.7) and no differences were found in sensitivity (SEN) (95.0% vs 87.5%, P = 0.2), specificity (SPE) (45.5 vs 49.4, respectively; P = 0.7), positive predictive value (PPV) (75.2% vs 77.7%, respectively; P = 0.8), nor negative predictive value (NPV) (85.6 vs 77.7, respectively; P = 0.3). When analyzing the non-concordant nodules (n = 43), the CAD system presented a decrease in accuracy of 4.2%, which was significantly lower than that observed by the experts (19.9%, P = 0.02). Conclusions Clinical observers are similar to the CAD system in the US assessment of the risk of thyroid nodules. However, the AI system for thyroid nodules AmCAD-UT® showed more reliability in the analysis of unclear or misleading images.

Download Full-text

A beneficial role of computer-aided diagnosis system for less experienced physicians in the diagnosis of thyroid nodule on ultrasound

Scientific Reports ◽

10.1038/s41598-021-99983-6 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Sunyoung Kang ◽

Eunjung Lee ◽

Chae Won Chung ◽

Han Na Jang ◽

Joon Ho Moon ◽

...

Keyword(s):

Diagnostic Performance ◽

Thyroid Nodules ◽

Needle Aspiration ◽

Computer Aided Diagnosis ◽

Cad System ◽

The Us ◽

Computer Aided ◽

Needle Aspiration Cytology ◽

Aided Diagnosis

AbstractUltrasonography (US) is the primary diagnostic tool for thyroid nodules, while the accuracy is operator-dependent. It is widely used not only by radiologists but also by physicians with different levels of experience. The aim of this study was to investigate whether US with computer-aided diagnosis (CAD) has assisting roles to physicians in the diagnosis of thyroid nodules. 451 thyroid nodules evaluated by fine-needle aspiration cytology following surgery were included. 300 (66.5%) of them were diagnosed as malignancy. Physicians with US experience less than 1 year (inexperienced, n = 10), or more than 5 years (experienced, n = 3) reviewed the US images of thyroid nodules with or without CAD assistance. The diagnostic performance of CAD was comparable to that of the experienced group, and better than those of the inexperienced group. The AUC of the CAD for conventional PTC was higher than that for FTC and follicular variant PTC (0.925 vs. 0.499), independent of tumor size. CAD assistance significantly improved diagnostic performance in the inexperienced group, but not in the experienced groups. In conclusion, the CAD system showed good performance in the diagnosis of conventional PTC. CAD assistance improved the diagnostic performance of less experienced physicians in US, especially in diagnosis of conventional PTC.

Download Full-text