scholarly journals A Hematologist-Level Deep Learning Algorithm (BMSNet) for Assessing the Morphologies of Single Nuclear Balls in Bone Marrow Smears: Algorithm Development

10.2196/15963 ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. e15963 ◽  
Author(s):  
Yi-Ying Wu ◽  
Tzu-Chuan Huang ◽  
Ren-Hua Ye ◽  
Wen-Hui Fang ◽  
Shiue-Wei Lai ◽  
...  

Background Bone marrow aspiration and biopsy remain the gold standard for the diagnosis of hematological diseases despite the development of flow cytometry (FCM) and molecular and gene analyses. However, the interpretation of the results is laborious and operator dependent. Furthermore, the obtained results exhibit inter- and intravariations among specialists. Therefore, it is important to develop a more objective and automated analysis system. Several deep learning models have been developed and applied in medical image analysis but not in the field of hematological histology, especially for bone marrow smear applications. Objective The aim of this study was to develop a deep learning model (BMSNet) for assisting hematologists in the interpretation of bone marrow smears for faster diagnosis and disease monitoring. Methods From January 1, 2016, to December 31, 2018, 122 bone marrow smears were photographed and divided into a development cohort (N=42), a validation cohort (N=70), and a competition cohort (N=10). The development cohort included 17,319 annotated cells from 291 high-resolution photos. In total, 20 photos were taken for each patient in the validation cohort and the competition cohort. This study included eight annotation categories: erythroid, blasts, myeloid, lymphoid, plasma cells, monocyte, megakaryocyte, and unable to identify. BMSNet is a convolutional neural network with the YOLO v3 architecture, which detects and classifies single cells in a single model. Six visiting staff members participated in a human-machine competition, and the results from the FCM were regarded as the ground truth. Results In the development cohort, according to 6-fold cross-validation, the average precision of the bounding box prediction without consideration of the classification is 67.4%. After removing the bounding box prediction error, the precision and recall of BMSNet were similar to those of the hematologists in most categories. In detecting more than 5% of blasts in the validation cohort, the area under the curve (AUC) of BMSNet (0.948) was higher than the AUC of the hematologists (0.929) but lower than the AUC of the pathologists (0.985). In detecting more than 20% of blasts, the AUCs of the hematologists (0.981) and pathologists (0.980) were similar and were higher than the AUC of BMSNet (0.942). Further analysis showed that the performance difference could be attributed to the myelodysplastic syndrome cases. In the competition cohort, the mean value of the correlations between BMSNet and FCM was 0.960, and the mean values of the correlations between the visiting staff and FCM ranged between 0.952 and 0.990. Conclusions Our deep learning model can assist hematologists in interpreting bone marrow smears by facilitating and accelerating the detection of hematopoietic cells. However, a detailed morphological interpretation still requires trained hematologists.

Cancers ◽  
2019 ◽  
Vol 11 (10) ◽  
pp. 1579 ◽  
Author(s):  
Muyi Sun ◽  
Wei Zhou ◽  
Xingqun Qi ◽  
Guanhong Zhang ◽  
Leonard Girnita ◽  
...  

Uveal melanoma is the most common primary intraocular malignancy in adults, with nearly half of all patients eventually developing metastases, which are invariably fatal. Manual assessment of the level of expression of the tumor suppressor BRCA1-associated protein 1 (BAP1) in tumor cell nuclei can identify patients with a high risk of developing metastases, but may suffer from poor reproducibility. In this study, we verified whether artificial intelligence could predict manual assessments of BAP1 expression in 47 enucleated eyes with uveal melanoma, collected from one European and one American referral center. Digitally scanned pathology slides were divided into 8176 patches, each with a size of 256 × 256 pixels. These were in turn divided into a training cohort of 6800 patches and a validation cohort of 1376 patches. A densely-connected classification network based on deep learning was then applied to each patch. This achieved a sensitivity of 97.1%, a specificity of 98.1%, an overall diagnostic accuracy of 97.1%, and an F1-score of 97.8% for the prediction of BAP1 expression in individual high resolution patches, and slightly less with lower resolution. The area under the receiver operating characteristic (ROC) curves of the deep learning model achieved an average of 0.99. On a full tumor level, our network classified all 47 tumors identically with an ophthalmic pathologist. We conclude that this deep learning model provides an accurate and reproducible method for the prediction of BAP1 expression in uveal melanoma.


2021 ◽  
Author(s):  
Jae-Seung Yun ◽  
Jaesik Kim ◽  
Sang-Hyuk Jung ◽  
Seon-Ah Cha ◽  
Seung-Hyun Ko ◽  
...  

Objective: We aimed to develop and evaluate a non-invasive deep learning algorithm for screening type 2 diabetes in UK Biobank participants using retinal images. Research Design and Methods: The deep learning model for prediction of type 2 diabetes was trained on retinal images from 50,077 UK Biobank participants and tested on 12,185 participants. We evaluated its performance in terms of predicting traditional risk factors (TRFs) and genetic risk for diabetes. Next, we compared the performance of three models in predicting type 2 diabetes using 1) an image-only deep learning algorithm, 2) TRFs, 3) the combination of the algorithm and TRFs. Assessing net reclassification improvement (NRI) allowed quantification of the improvement afforded by adding the algorithm to the TRF model. Results: When predicting TRFs with the deep learning algorithm, the areas under the curve (AUCs) obtained with the validation set for age, sex, and HbA1c status were 0.931 (0.928-0.934), 0.933 (0.929-0.936), and 0.734 (0.715-0.752), respectively. When predicting type 2 diabetes, the AUC of the composite logistic model using non-invasive TRFs was 0.810 (0.790-0.830), and that for the deep learning model using only fundus images was 0.731 (0.707-0.756). Upon addition of TRFs to the deep learning algorithm, discriminative performance was improved to 0.844 (0.826-0.861). The addition of the algorithm to the TRFs model improved risk stratification with an overall NRI of 50.8%. Conclusions: Our results demonstrate that this deep learning algorithm can be a useful tool for stratifying individuals at high risk of type 2 diabetes in the general population.


Author(s):  
Amit Doegar ◽  
◽  
Maitreyee Dutta ◽  
Gaurav Kumar ◽  
◽  
...  

In the present scenario, one of the threats of trust on images for digital and online applications as well as on social media. Individual’s reputation can be turnish using misinformation or manipulation in the digital images. Image forgery detection is an approach for detection and localization of forged components in the image which is manipulated. For effective image forgery detection, an adequate number of features are required which can be accomplished by a deep learning model, which does not require manual feature engineering or handcraft feature approaches. In this paper we have implemented GoogleNet deep learning model to extract the image features and employ Random Forest machine learning algorithm to detect whether the image is forged or not. The proposed approach is implemented on the publicly available benchmark dataset MICC-F220 with k-fold cross validation approach to split the dataset into training and testing dataset and also compared with the state-of-the-art approaches.


2021 ◽  
Vol 11 (16) ◽  
pp. 7355
Author(s):  
Zhiheng Xu ◽  
Xiong Ding ◽  
Kun Yin ◽  
Ziyue Li ◽  
Joan A. Smyth ◽  
...  

Tick species are considered the second leading vector of human diseases. Different ticks can transmit a variety of pathogens that cause various tick-borne diseases (TBD), such as Lyme disease. Currently, it remains a challenge to diagnose Lyme disease because of its non-specific symptoms. Rapid and accurate identification of tick species plays an important role in predicting potential disease risk for tick-bitten patients, and ensuring timely and effective treatment. Here, we developed, optimized, and tested a smartphone-based deep learning algorithm (termed “TickPhone app”) for tick identification. The deep learning model was trained by more than 2000 tick images and optimized by different parameters, including normal sizes of images, deep learning architectures, image styles, and training–testing dataset distributions. The optimized deep learning model achieved a training accuracy of ~90% and a validation accuracy of ~85%. The TickPhone app was used to identify 31 independent tick species and achieved an accuracy of 95.69%. Such a simple and easy-to-use TickPhone app showed great potential to estimate epidemiology and risk of tick-borne disease, help health care providers better predict potential disease risk for tick-bitten patients, and ultimately enable timely and effective medical treatment for patients.


2018 ◽  
Vol 36 (4_suppl) ◽  
pp. 266-266
Author(s):  
Sunyoung S. Lee ◽  
Jin Cheon Kim ◽  
Jillian Dolan ◽  
Andrew Baird

266 Background: The characteristic histological feature of pancreatic adenocarcinoma (PAD) is extensive desmoplasia alongside leukocytes and cancer-associated fibroblasts. Desmoplasia is a known barrier to the absorption and penetration of therapeutic drugs. Stromal cells are key elements for a clinical response to chemotherapy and immunotherapy, but few models exist to analyze the spatial and architectural elements that compose the complex tumor microenvironment in PAD. Methods: We created a deep learning algorithm to analyze images and quantify cells and fibrotic tissue. Histopathology slides of PAD patients (pts) were then used to automate the recognition and mapping of adenocarcinoma cells, leukocytes, fibroblasts, and degree of desmoplasia, defined as the ratio of the area of fibrosis to that of the tumor gland. This information was correlated with mutational burden, defined as mutations (mts) per megabase (mb) of each pt. Results: The histopathology slides (H&E stain) of 126 pts were obtained from The Cancer Genome Atlas (TCGA) and analyzed with the deep learning model. Pt with the largest mutational burden (733 mts/mb, n = 1 pt) showed the largest number of leukocytes (585/mm2). Those with the smallest mutational burden (0 mts/mb, n = 16 pts) showed the fewest leukocytes (median, 14/mm2). Mutational burden was linearly proportional to the number of leukocytes (R2 of 0.7772). The pt with a mutational burden of 733 was excluded as an outlier. No statistically significant difference in the number of fibroblasts, degree of desmoplasia, or thickness of the first fibrotic layer (the smooth muscle actin-rich layer outside of the tumor gland), was found among pts of varying mutational burden. The median distance from a tumor gland to a leukocyte was inversely proportional to the number of leukocytes in a box of 1 mm2 with a tumor gland at the center. Conclusions: A deep learning model enabled automated quantification and mapping of desmoplasia, stromal and malignant cells, revealing the spatial and architectural relationship of these cells in PAD pts with varying mutational burdens. Further biomarker driven studies in the context of immunotherapy and anti-fibrosis are warranted.


2020 ◽  
Vol 4 (Supplement_1) ◽  
Author(s):  
Indriani Astono ◽  
Christopher W Rowe ◽  
James Welsh ◽  
Phillip Jobling

Abstract Introduction: Nerves in the cancer microenvironment have prognostic significance, and nerve-cancer crosstalk may contribute to tumour progression, but the role of nerves in thyroid cancer is not known (1). Reproducible techniques to quantify innervation are lacking, with reliance on manual counting or basic single-parameter digital quantification. Aims: To determine if a deep machine learning algorithm could objectively quantify nerves in a digital histological dataset of thyroid cancers immunostained for the specific pan-neuronal marker PGP9.5. Methods: A training dataset of 30 digitised papillary thyroid cancer immunohistochemistry slides were manually screened for PGP9.5 positive nerves, annotated using QuPath (2). 1500 true positive nerves were identified. This dataset was used to train the deep-learning algorithm. First, a colour filter identified pixels positive for PGP9.5 (Model 1). Then, a manually tuned colour filter and clustering method identified Regions of Interest (ROIs): clusters of PGP9.5 positive pixels that may represent nerves (Model 2). These ROIs were classified by the deep learning model (Model 3), based on a Convolutional Neural Network with approximately 2.7 million trainable parameters. The full model was run on a testing dataset of thyroid cancer slides (n=5), containing 7-35 manually identified nerves per slide. Model predictions were validated by human assessment of a random subset of 100 ROIs. The code was written in Python and the model was developed in Keras. Results: Model 2 (colour filter + clustering only) identified median 2247 ROIs per slide (range 349-4748), which included 94% of the manually identified nerves. However, most Model 2 ROIs were false positives (FP) (median 85% FP, range 68-95%), indicating that Model 2 was sensitive but poorly specific for nerve identification. Model 3 (deep learning) identified fewer ROIs per slide (median 1068, range 150-3091), but still correctly identified 94% of manually annotated nerves. Of the additionally detected ROIs in Model 3, median FP rate was 35%. However, in slides where higher non-specific immunostaining was present, then the number of FP ROIs was >90%. Conclusion: Simple image analysis based on colour filtration/cluster analysis does not accurately identify immunohistochemically labelled nerves in thyroid cancers. Addition of deep-learning improves sensitivity with acceptable specificity, and significantly increases the number of true positive nerves detected compared to manual counting. However, the current deep learning model lacks specificity in the setting of non-specific immunostaining, which is a basis for improving further iterations of this model to facilitate study of the significance of innervation of thyroid and other cancers. References: (1) Faulkner et al. Cancer Discovery (2019) doi: 10.1158/2159-8290.CD-18-1398. (2) Bankhead P et al. Sci Rep 2017;7(1):16878.


2021 ◽  
Author(s):  
Joshua Levy ◽  
Christopher M Navas ◽  
Joan A Chandra ◽  
Brock Christensen ◽  
Louis J Vaickus ◽  
...  

BACKGROUND AND AIMS: Evaluation for dyssynergia is the most common reason that gastroenterologists refer patients for anorectal manometry, because dyssynergia is amenable to biofeedback by physical therapists. High-definition anorectal manometry (3D-HDAM) is a promising technology to evaluate anorectal physiology, but adoption remains limited by its sheer complexity. We developed a 3D-HDAM deep learning algorithm to evaluate for dyssynergia. METHODS: Spatial-temporal data were extracted from consecutive 3D-HDAM studies performed between 2018-2020 at a tertiary institution. The technical procedure and gold standard definition of dyssynergia were based on the London consensus, adapted to the needs of 3D-HDAM technology. Three machine learning models were generated: (1) traditional machine learning informed by conventional anorectal function metrics, (2) deep learning, and (3) a hybrid approach. Diagnostic accuracy was evaluated using bootstrap sampling to calculate area-under-the-curve (AUC). To evaluate overfitting, models were validated by adding 502 simulated defecation maneuvers with diagnostic ambiguity. RESULTS: 302 3D-HDAM studies representing 1,208 simulated defecation maneuvers were included (average age 55.2 years; 80.5% women). The deep learning model had comparable diagnostic accuracy (AUC=0.91 [95% confidence interval 0.89-0.93]) to traditional (AUC=0.93[0.92-0.95]) and hybrid (AUC=0.96[0.94-0.97]) predictive models in training cohorts. However, the deep learning model handled ambiguous tests more cautiously than other models; the deep learning model was more likely to designate an ambiguous test as inconclusive (odds ratio=4.21[2.78-6.38]) versus traditional/hybrid approaches. CONCLUSIONS: By considering complex spatial-temporal information beyond conventional anorectal function metrics, deep learning on 3D-HDAM technology may enable gastroenterologists to reliably identify and manage dyssynergia in broader practice.


Blood ◽  
2020 ◽  
Vol 136 (Supplement 1) ◽  
pp. 10-12
Author(s):  
John-William Sidhom ◽  
Ingharan J Siddarthan ◽  
Bo-Shiun Lai ◽  
Adam Luo ◽  
Bryan Hambley ◽  
...  

Acute Promyelocytic Leukemia (APL) is a subtype of Acute Myeloid Leukemia (AML), classified by a translocation between chromosomes 15 and 17 [t(15;17)], that is notably distinguished clinically by a rapidly progressive and fatal course. Due to the acute nature of its presentation, prompt and accurate diagnosis is required to initiate appropriate therapy that can be curative. However, the gold standard genetic tests can take days to confirm a diagnosis and thus therapy is often initiated on high clinical suspicion based on both clinical presentation as well as direct visualization of the peripheral smear. While there are described cellular morphological features that distinguish APL, there is still considerable difficulty in diagnosing APL from direct visualization of a peripheral smear by a hematopathologist. We hypothesized that deep learning pattern recognition would have greater discriminatory power and consistency compared to humans to distinguish t(15;17) translocation positive APL from t(15;17) translocation negative AML. To best tackle the problem of diagnosing APL rapidly from a peripheral smear, study patients with APL and AML were identified via retrospective chart review from a list of confirmed FISH t(15;17)-positive (n = 34) and -negative (n = 72) patients presenting at The Johns Hopkins Hospital (JHH). Additional inclusion criteria included new disease diagnosis, no prior treatment, and availability of peripheral blood smear image uploaded to CellaVision. Patients were separated into a discovery cohort presenting prior to 1/2019 (APL, n = 22; AML, n=60) and a validation cohort presenting on or after 1/2019 (APL, n = 12; AML, n = 12). A multiple-instance deep learning model employing convolutional layers at the per-cell level (Figure 1A) was trained on the discovery cohort and then tested on the independent prospective validation cohort to assess generalizability of the model. When compared to 10 academic clinicians (denoted with red +) who consisted of leukemia-treating hematologists, oncologists, and hematopathologists, the deep learning model was equivalent or outperformed 9/10 readers (Figure 1B) with an AUC of 0.861. We further looked at the performance of using proportion of promyelocytes (per CellaVision classification) as a biomarker of APL which had an AUC of 0.611. Finally, we applied integrated gradients, a method by which to extract per-pixel importance to the classification probability to identify and understand the morphological features the model was learning and using to distinguish APL (Figure 1C). We noted that the appearance of the chromatin in the non-APL leukemias was more dispersed and focused at the edge of the cell whereas in APL, the chromatin was more condensed and focused at the center of the cell. These morphological features, taught to us by the model, have not been previously reported in the literature as being useful for distinguishing APL from non-APL. Our work presents a deep learning model capable of rapid and accurate diagnosis of APL from universally available peripheral smears. In addition, explainable artificial intelligence is provided for biological insights to facilitate clinical management and reveal morphological concepts previously unappreciated in APL. The deep learning framework we have delineated is applicable to any diagnostic pipeline that can leverage a peripheral blood smear, potentially allowing for efficient diagnosis and early treatment of disease. Figure 1. Disclosures Streiff: Bayer: Consultancy, Speakers Bureau; Dispersol: Consultancy; BristolMyersSquibb: Consultancy; Janssen: Consultancy, Research Funding; Pfizer: Consultancy, Speakers Bureau; Portola: Consultancy; Boehringer-Ingelheim: Research Funding; NHLBI: Research Funding; PCORI: Research Funding; NovoNordisk: Research Funding; Sanofi: Research Funding. Moliterno:Pharmessentia: Consultancy; MPNRF: Research Funding. DeZern:MEI: Consultancy; Abbvie: Consultancy; Astex: Research Funding; Celgene: Consultancy, Honoraria. Levis:Astellas: Honoraria, Research Funding; Menarini: Honoraria; Amgen: Honoraria; FujiFilm: Honoraria, Research Funding; Daiichi-Sankyo: Honoraria.


Blood ◽  
2020 ◽  
Vol 136 (Supplement 1) ◽  
pp. 18-18
Author(s):  
Lintao Bi ◽  
Wen Gao ◽  
Lingjun Meng ◽  
Guiying Gu ◽  
Zhangzhen Shi ◽  
...  

It has been known that neutrophils play an important role in regulating homeostasis and disease. Tumor-associated neutrophils (TANs), as an important member of the tumor microenvironment, have gradually been proved their roles in a variety of solid tumors. It is generally believed that the changes in blood cell morphology (including neutrophils) are the phenotype of hematological diseases (such as in myelodysplastic syndromes) or tumor cells themselves. However, whether there is a possibility that the accumulation of abnormal neutrophils function leads to the change of hematopoietic stem cells and this is just the reason of hematological diseases? Do neutrophils play a key role in the pathogenesis and development of hematological tumors, especially acquired or age-related blood diseases, such as most acute and chronic leukemia, multiple myeloma and other diseases? TAN also has polarization, which is similar to tumor-associated macrophages (TAM), suggesting that the function and morphology of neutrophils are closely associated. Therefore, we assumed that there are function-related morphological differences in neutrophils in different hematological diseases. Finding these differences may provide clues for the functional research of neutrophils in hematological diseases. Artificial intelligence represented by deep learning can distinguish images efficiently and accurately (such as face recognition). Here we try to apply deep learning to discovery and recognize the morphological difference among neutrophils in different hematological diseases. We obtained whole slide images (WSI) from 4 types of malignant hematological diseases, which is chronic myelogenous leukemia (CML), multiple myeloma (MM), acute myeloblastic leukemia with maturation (AML-M2), acute monocytic leukemia (AML-M5) and normal bone marrow. Neutrophils were segmented from WSI by two diagnostic physicians (one with more than 40 years of diagnostic experience and the other with 13 years of diagnostic experience) There are 6115 neutrophils, and the number of cells in each disease and normal bone marrow is 593, 1404, 2509, 850, and 759, respectively. We trained these neutrophils using the transfer learning algorithm and the ratio of training and verification groups is 80:20. We established a convolutional neural network (CNN) model based on the morphological phenotype of neutrophils to judge their disease classification and used confusion matrix and receiver operator characteristic (ROC) curve for model evaluation. We found that neutrophils from different diseases can be classified into different categories, and the deep learning model has a high accuracy rate for judging the neutrophils from different diseases. Moreover, according to the obtained mixed matrix results, it is found that some M2 and M5 neutrophils are prone to misjudgment, while M2 and M5 is rarely confused with other diseases. The reason for this may be that M2 and M5 are both acute myeloid leukemia. Neutrophils from MM and normal bone marrow are prone to misjudge each other or judged as CML neutrophils, and MM often involves the plasma cell system, so some neutrophils of MM may be similar to normal bone marrow. Compared with acute leukemia, some chronic leukemia neutrophils are close to MM or normal bone marrow. Based on these results, we can further confirm that there are morphological and phenotypic differences between different types of hematological diseases. According to the ROC curve results, it is suggested that the deep learning model constructed based on the feature extraction of the CNN model can more accurately determine different hematological diseases according to morphological phenotypes of neutrophils. These findings suggest that neutrophils in different hematological diseases have their own features. These features may provide more evidence for the diagnosis of the disease and also provide clues for further research on the function of TAN in primary hematological diseases. Disclosures No relevant conflicts of interest to declare.


Sign in / Sign up

Export Citation Format

Share Document