Evaluation of Platelet Indices in Children with Renal Scarring based on Diagnostic Accuracy Criteria and Cohen’s Kappa

Skeletal disease is common in multiple myeloma. We investigated the inter-observer agreement and diagnostic accuracy of spinal fractures diagnosed by computer tomography (CT) and magnetic resonance imaging (MRI) from 12 myeloma patients. Two radiologists independently assessed the images. CT, MRI, and other images were combined to a gold standard. The inter-observer agreement was assessed with Cohen’s kappa. Radiologist 1 diagnosed 20 malignant spinal fractures on CT and 26 on MRI, while radiologist 2 diagnosed 12 malignant spinal fractures on CT and 22 on MRI. In comparison the gold standard diagnosed 10 malignant spinal fractures. The sensitivity for malignant fractures varied from 0.5 to 1 for CT and MRI, and the specificity varied from 0.17 to 0.67. On MRI, the specificity for malignant spinal fractures was 0.17 for both radiologists. The inter-observer agreement for malignant spinal fractures on CT was -0.42 (Cohen’s kappa) and -0.13 for MRI, while for osteoporotic fractures it was -0.24 for CT and 0.53 for MRI. We conclude that malignant spinal fractures were over-diagnosed on CT and MRI. The inter-observer agreement was extremely poor.

Download Full-text

Diagnostic Accuracy of CareStart™ Malaria HRP2 and SD Bioline Pf/PAN for Malaria in Febrile Outpatients in Varying Malaria Transmission Settings in Cameroon

Diagnostics ◽

10.3390/diagnostics11091556 ◽

2021 ◽

Vol 11 (9) ◽

pp. 1556

Author(s):

Innocent Mbulli Ali ◽

Akindeh Mbuh Nji ◽

Jacob Chefor Bonkum ◽

Marcel Nyuylam Moyeh ◽

Guenang Kenfack Carole ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Malaria Transmission ◽

Diagnostic Tests ◽

Diagnostic Performance ◽

District Hospital ◽

Rapid Diagnostic Tests ◽

Cohen’S Kappa ◽

Cohen's Kappa ◽

Malaria Rapid Diagnostic Tests ◽

Catholic Hospital

Background: There was an increase in the number of malaria cases in Cameroon in 2018 that could reflect changes in provider practice, despite effective interventions. In this study, we assessed the diagnostic performance of two malaria rapid diagnostic tests (mRDTs) for diagnostic confirmation of suspected cases of malaria in public and private health facilities in two malaria transmission settings in Cameroon. Methods: We evaluated the diagnostic performance of CareStart pf and SD Bioline Pf/PAN mRDT and compared these parameters by RDT type and transmission setting. Nested PCR and blood film microscopy were used as references. The chi square test was used for independent sample comparisons, while the McNemar’s test was used to test for the dependence of categorical data in paired sample testing. A p < 0.05 was considered significant in all comparisons. The R (v.4.0.2) software was used for analyses. Results: A total of 1126 participants consented for the study in the four sites. The diagnostic accuracy of the CareStart Pf mRDT was 0.93.6% (0.911–0.961) in Yaoundé, 0.930% (0.90–0.960) in Ngounso, 0.84% (0.794–0.891) in St Vincent Catholic Hospital Dschang and 0.407 (0.345–0.468) in Dschang district hospital. For SD Bioline Pf/PAN the accuracy was 0.759 (0.738–0.846) for St Vincent Catholic Hospital Dschang and 0.426 (0.372–0.496) for the Dschang district hospital. The accuracy was slightly lower in each case but not statistically different when PCR was considered as the reference. The likelihood ratios of the positive and negative tests were high in the high transmission settings of Yaoundé (10.99 (6.24–19.35)) and Ngounso (14.40 (7.89–26.28)) compared to the low transmission settings of Dschang (0.71 (0.37–1.37)) and St Vincent Catholic hospital (7.37 (4.32−12.59)). There was a high degree of agreement between the tests in Yaoundé (Cohen’s Kappa: 0.85 ± 0.05 (0.7–0.95)) and Ngounso (Cohen’s Kappa: 0.86 ± 0.05 (0.74, 0.97)) and moderate agreement in St Vincent hospital Dschang (k: 0.58 ± 0.06 (0.44–0.71)) and poor agreement in the District Hospital Dschang (Cohen’s Kappa: −0.11 ± 0.05 (−0.21–0.01)). The diagnostic indicators of the SD Bioline Pf/PAN were slightly better than for CareStart Pf mRDT in St Vincent Catholic hospital Dschang, irrespective of the reference test. Conclusions: Publicly procured malaria rapid diagnostic tests in Cameroon have maintained high accuracy (91–94%) in the clinical diagnosis of malaria in high malaria transmission regions of Cameroon, although they failed to reach WHO standards. We observed an exception in the low transmission region of Dschang, West region, where the accuracy tended to be lower and variable between facilities located in this town. These results underscore the importance of the routine monitoring of the quality and performance of malaria RDTs in diverse settings in malaria endemic areas.

Download Full-text

The diagnostic accuracy of MRI in determining the relations between paraclinoid aneurysms and the cavernous sinus

Neuroradiology ◽

10.1007/s00234-021-02864-y ◽

2021 ◽

Author(s):

Sérgio Tadeu Fernandes ◽

Hugo Leonardo Doria-Netto ◽

Raphael Vicente Alves ◽

Renan Luiz Lapate ◽

Nelson Paes Fortes Diniz Ferreira ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Cavernous Sinus ◽

Spin Echo ◽

Fast Spin Echo ◽

Likelihood Ratios ◽

Cohen’S Kappa ◽

Cohen's Kappa ◽

Paraclinoid Aneurysms ◽

History Of ◽

Fast Spin

Abstract Purpose The location of paraclinoid aneurysms is determinant for evaluation of its intradural compartment and risk of SAH after rupture. Advanced MRI techniques have provided clear visualization of the distal dural ring (DDR) to determine whether an aneurysm is intracavernous, transitional or intradural for decision-making. We analyzed the diagnostic accuracy of MRI in predicting whether a paraclinoid aneurysm is intracavernous, transitional or intradural. Methods We conducted a prospective cohort between January 2014 and December 2018. Patients with paraclinoid aneurysms underwent 3D fast spin-echo MRI sequence before surgical treatment. The DDR was the landmark for MRI characterization of the aneurysms as follow: (i) Intradural; (ii) Transitional; and (iii) Intracavernous. The MRI sensitivity, specificity, positive and negative likelihood ratios were determined compared to the intraoperative findings. We also evaluated the intertechnique agreement using the Cohen’s kappa coefficient (κ) for dichotomous classifications (cavernous vs non-cavernous). Results Twenty patients were included in the cohort. The accuracy of MRI showed a sensitivity of 86.7% (95%CI:59.5–98.3) and specificity of 90.0% (95%CI:55.5–99.8). Analyzing only patients without history of SAH, accuracy test improved with a sensitivity of 92.3% (95%CI:63.9–99.8) and specificity reached 100% (95%CI: 63–100). Values of Cohen’s kappa (κ), intertechnique agreement was considered substantial for dichotomous classifications (κ = 0.754; p < 0.001). For patients without previous SAH, intertechnique agreement was even more coincident for the dichotomous classification (κ = 0.901; p < 0.001). Conclusion 3D fast spin-echo MRI sequence is a reliable and useful technique for determining the location of paraclinoid aneurysms in relation to the cavernous sinus, particularly for patients with no history of SAH.

Download Full-text

An update on image forming methods: structure analysis and Gestalt evaluation of images from rocket lettuce with shading, N supply, organic or mineral fertilization, and biodynamic preparations

Organic Agriculture ◽

10.1007/s13165-021-00347-1 ◽

2021 ◽

Author(s):

Miriam Athmann ◽

Roya Bornhütter ◽

Nicolaas Busscher ◽

Paul Doesburg ◽

Uwe Geier ◽

...

Keyword(s):

Copper Chloride ◽

Organic Fertilizer ◽

Structural Features ◽

Fertilizer Treatment ◽

Friedman Test ◽

Mineral Fertilization ◽

Cohen’S Kappa ◽

N Supply ◽

Cohen's Kappa ◽

Forming Methods

AbstractIn the image forming methods, copper chloride crystallization (CCCryst), capillary dynamolysis (CapDyn), and circular chromatography (CChrom), characteristic patterns emerge in response to different food extracts. These patterns reflect the resistance to decomposition as an aspect of resilience and are therefore used in product quality assessment complementary to chemical analyses. In the presented study, rocket lettuce from a field trial with different radiation intensities, nitrogen supply, biodynamic, organic and mineral fertilization, and with or without horn silica application was investigated with all three image forming methods. The main objective was to compare two different evaluation approaches, differing in the type of image forming method leading the evaluation, the amount of factors analyzed, and the deployed perceptual strategy: Firstly, image evaluation of samples from all four experimental factors simultaneously by two individual evaluators was based mainly on analyzing structural features in CapDyn (analytical perception). Secondly, a panel of eight evaluators applied a Gestalt evaluation imbued with a kinesthetic engagement of CCCryst patterns from either fertilization treatments or horn silica treatments, followed by a confirmatory analysis of individual structural features. With the analytical approach, samples from different radiation intensities and N supply levels were identified correctly in two out of two sample sets with groups of five samples per treatment each (Cohen’s kappa, p = 0.0079), and the two organic fertilizer treatments were differentiated from the mineral fertilizer treatment in eight out of eight sample sets with groups of three manure and two minerally fertilized samples each (Cohen’s kappa, p = 0.0048). With the panel approach based on Gestalt evaluation, biodynamic fertilization was differentiated from organic and mineral fertilization in two out of two exams with 16 comparisons each (Friedman test, p < 0.001), and samples with horn silica application were successfully identified in two out of two exams with 32 comparisons each (Friedman test, p < 0.001). Further research will show which properties of the food decisive for resistance to decomposition are reflected by analytical and Gestalt criteria, respectively, in CCCryst and CapDyn images.

Download Full-text

Prediction of Streptococcus uberis clinical mastitis treatment success in dairy herds by means of mass spectrometry and machine-learning

Scientific Reports ◽

10.1038/s41598-021-87300-0 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Alexandre Maciel-Guerra ◽

Necati Esener ◽

Katharina Giebel ◽

Daniel Lea ◽

Martin J. Green ◽

...

Keyword(s):

Machine Learning ◽

Mass Spectrometry ◽

Treatment Success ◽

Clinical Mastitis ◽

Supervised Machine Learning ◽

Structural Protein ◽

Cohen’S Kappa ◽

Maldi Tof ◽

Streptococcus Uberis ◽

Cohen's Kappa

AbstractStreptococcus uberis is one of the leading pathogens causing mastitis worldwide. Identification of S. uberis strains that fail to respond to treatment with antibiotics is essential for better decision making and treatment selection. We demonstrate that the combination of supervised machine learning and matrix-assisted laser desorption ionization/time of flight (MALDI-TOF) mass spectrometry can discriminate strains of S. uberis causing clinical mastitis that are likely to be responsive or unresponsive to treatment. Diagnostics prediction systems trained on 90 individuals from 26 different farms achieved up to 86.2% and 71.5% in terms of accuracy and Cohen’s kappa. The performance was further increased by adding metadata (parity, somatic cell count of previous lactation and count of positive mastitis cases) to encoded MALDI-TOF spectra, which increased accuracy and Cohen’s kappa to 92.2% and 84.1% respectively. A computational framework integrating protein–protein networks and structural protein information to the machine learning results unveiled the molecular determinants underlying the responsive and unresponsive phenotypes.

Download Full-text

The accuracy of initial diagnoses in coma: an observational study in 835 patients with non-traumatic disorder of consciousness

Scandinavian Journal of Trauma Resuscitation and Emergency Medicine ◽

10.1186/s13049-020-00822-w ◽

2021 ◽

Vol 29 (1) ◽

Author(s):

Maximilian Lutz ◽

Martin Möckel ◽

Tobias Lindner ◽

Christoph J. Ploner ◽

Mischa Braun ◽

...

Keyword(s):

Emergency Care ◽

Final Diagnosis ◽

Class I ◽

Prehospital Emergency Care ◽

Unknown Etiology ◽

Care Providers ◽

Cohen’S Kappa ◽

Disorder Of Consciousness ◽

Cohen's Kappa ◽

Suspected Diagnosis

Abstract Background Management of patients with coma of unknown etiology (CUE) is a major challenge in most emergency departments (EDs). CUE is associated with a high mortality and a wide variety of pathologies that require differential therapies. A suspected diagnosis issued by pre-hospital emergency care providers often drives the first approach to these patients. We aim to determine the accuracy and value of the initial diagnostic hypothesis in patients with CUE. Methods Consecutive ED patients presenting with CUE were prospectively enrolled. We obtained the suspected diagnoses or working hypotheses from standardized reports given by prehospital emergency care providers, both paramedics and emergency physicians. Suspected and final diagnoses were classified into I) acute primary brain lesions, II) primary brain pathologies without acute lesions and III) pathologies that affected the brain secondarily. We compared suspected and final diagnosis with percent agreement and Cohen’s Kappa including sub-group analyses for paramedics and physicians. Furthermore, we tested the value of suspected and final diagnoses as predictors for mortality with binary logistic regression models. Results Overall, suspected and final diagnoses matched in 62% of 835 enrolled patients. Cohen’s Kappa showed a value of κ = .415 (95% CI .361–.469, p < .005). There was no relevant difference in diagnostic accuracy between paramedics and physicians. Suspected diagnoses did not significantly interact with in-hospital mortality (e.g., suspected class I: OR .982, 95% CI .518–1.836) while final diagnoses interacted strongly (e.g., final class I: OR 5.425, 95% CI 3.409–8.633). Conclusion In cases of CUE, the suspected diagnosis is unreliable, regardless of different pre-hospital care providers’ qualifications. It is not an appropriate decision-making tool as it neither sufficiently predicts the final diagnosis nor detects the especially critical comatose patient. To avoid the risk of mistriage and unnecessarily delayed therapy, we advocate for a standardized diagnostic work-up for all CUE patients that should be triggered by the emergency symptom alone and not by any suspected diagnosis.

Download Full-text

Classification of Shoulder X-ray Images with Deep Learning Ensemble Models

Applied Sciences ◽

10.3390/app11062723 ◽

2021 ◽

Vol 11 (6) ◽

pp. 2723

Author(s):

Fatih Uysal ◽

Fırat Hardalaç ◽

Ozan Peker ◽

Tolga Tolunay ◽

Nil Tokgöz

Keyword(s):

Deep Learning ◽

Performance Test ◽

The Body ◽

Test Accuracy ◽

Cohen’S Kappa ◽

X Ray ◽

Cohen's Kappa ◽

Auc Value ◽

Magnetic Resonance Imaging Mri ◽

Fully Connected

Fractures occur in the shoulder area, which has a wider range of motion than other joints in the body, for various reasons. To diagnose these fractures, data gathered from X-radiation (X-ray), magnetic resonance imaging (MRI), or computed tomography (CT) are used. This study aims to help physicians by classifying shoulder images taken from X-ray devices as fracture/non-fracture with artificial intelligence. For this purpose, the performances of 26 deep learning-based pre-trained models in the detection of shoulder fractures were evaluated on the musculoskeletal radiographs (MURA) dataset, and two ensemble learning models (EL1 and EL2) were developed. The pre-trained models used are ResNet, ResNeXt, DenseNet, VGG, Inception, MobileNet, and their spinal fully connected (Spinal FC) versions. In the EL1 and EL2 models developed using pre-trained models with the best performance, test accuracy was 0.8455, 0.8472, Cohen’s kappa was 0.6907, 0.6942 and the area that was related with fracture class under the receiver operating characteristic (ROC) curve (AUC) was 0.8862, 0.8695. As a result of 28 different classifications in total, the highest test accuracy and Cohen’s kappa values were obtained in the EL2 model, and the highest AUC value was obtained in the EL1 model.

Download Full-text

The South African dysphagia screening tool (SADS): A screening tool for a developing context

South African Journal of Communication Disorders ◽

10.4102/sajcd.v63i1.117 ◽

2016 ◽

Vol 63 (1) ◽

Cited By ~ 2

Author(s):

Calli Ostrofsky ◽

Jaishika Seedat

Keyword(s):

At Risk ◽

Acute Stroke ◽

South African ◽

Screening Tool ◽

Diagnostic Assessment ◽

The South ◽

Stroke Patients ◽

Cohen’S Kappa ◽

Cohen's Kappa ◽

Government Hospitals

Background: Notwithstanding its value, there are challenges and limitations to implementing a dysphagia screening tool from a developed contexts in a developing context. The need for a reliable and valid screening tool for dysphagia that considers context, systemic rules and resources was identified to prevent further medical compromise, optimise dysphagia prognosis and ultimately hasten patients’ return to home or work.Methodology: To establish the validity and reliability of the South African dysphagia screening tool (SADS) for acute stroke patients accessing government hospital services. The study was a quantitative, non-experimental, correlational cross-sectional design with a retrospective component. Convenient sampling was used to recruit 18 speech-language therapists and 63 acute stroke patients from three South African government hospitals. The SADS consists of 20 test items and was administered by speech-language therapists. Screening was followed by a diagnostic dysphagia assessment. The administrator of the tool was not involved in completing the diagnostic assessment, to eliminate bias and prevent contamination of results from screener to diagnostic assessment. Sensitivity, validity and efficacy of the screening tool were evaluated against the results of the diagnostic dysphagia assessment. Cohen’s kappa measures determined inter-rater agreement between the results of the SADS and the diagnostic assessment.Results and conclusion: The SADS was proven to be valid and reliable. Cohen’s kappa indicated a high inter-rater reliability and showed high sensitivity and adequate specificity in detecting dysphagia amongst acute stroke patients who were at risk for dysphagia. The SADS was characterised by concurrent, content and face validity. As a first step in establishing contextual appropriateness, the SADS is a valid and reliable screening tool that is sensitive in identifying stroke patients at risk for dysphagia within government hospitals in South Africa.

Download Full-text

Deep Learning–Based Automated Thrombolysis in Cerebral Infarction Scoring: A Timely Proof-of-Principle Study

Stroke ◽

10.1161/strokeaha.120.033807 ◽

2021 ◽

Author(s):

Maximilian Nielsen ◽

Moritz Waldmann ◽

Andreas M. Frölich ◽

Fabian Flottmann ◽

Evelin Hristova ◽

...

Keyword(s):

Deep Learning ◽

Cerebral Infarction ◽

Digital Subtraction Angiography ◽

Information Integration ◽

Mechanical Thrombectomy ◽

Model Updating ◽

Digital Subtraction ◽

Cohen’S Kappa ◽

Expert Rating ◽

Cohen's Kappa

Background and Purpose: Mechanical thrombectomy is an established procedure for treatment of acute ischemic stroke. Mechanical thrombectomy success is commonly assessed by the Thrombolysis in Cerebral Infarction (TICI) score, assigned by visual inspection of X-ray digital subtraction angiography data. However, expert-based TICI scoring is highly observer-dependent. This represents a major obstacle for mechanical thrombectomy outcome comparison in, for instance, multicentric clinical studies. Focusing on occlusions of the M1 segment of the middle cerebral artery, the present study aimed to develop a deep learning (DL) solution to automated and, therefore, objective TICI scoring, to evaluate the agreement of DL- and expert-based scoring, and to compare corresponding numbers to published scoring variability of clinical experts. Methods: The study comprises 2 independent datasets. For DL system training and initial evaluation, an in-house dataset of 491 digital subtraction angiography series and modified TICI scores of 236 patients with M1 occlusions was collected. To test the model generalization capability, an independent external dataset with 95 digital subtraction angiography series was analyzed. Characteristics of the DL system were modeling TICI scoring as ordinal regression, explicit consideration of the temporal image information, integration of physiological knowledge, and modeling of inherent TICI scoring uncertainties. Results: For the in-house dataset, the DL system yields Cohen’s kappa, overall accuracy, and specific agreement values of 0.61, 71%, and 63% to 84%, respectively, compared with the gold standard: the expert rating. Values slightly drop to 0.52/64%/43% to 87% when the model is, without changes, applied to the external dataset. After model updating, they increase to 0.65/74%/60% to 90%. Literature Cohen’s kappa values for expert-based TICI scoring agreement are in the order of 0.6. Conclusions: The agreement of DL- and expert-based modified TICI scores in the range of published interobserver variability of clinical experts highlights the potential of the proposed DL solution to automated TICI scoring.

Download Full-text

Automated Large Artery Occlusion Detection in Stroke: A Single-Center Validation Study of an Artificial Intelligence Algorithm

Cerebrovascular Diseases ◽

10.1159/000519125 ◽

2021 ◽

pp. 1-6

Author(s):

Gabriel Rodrigues ◽

Clara M. Barreira ◽

Mehdi Bouslama ◽

Diogo C. Haussen ◽

Alhamza Al-Bayati ◽

...

Keyword(s):

Artificial Intelligence ◽

Secondary Analysis ◽

Large Artery ◽

Single Center ◽

Characteristic Analysis ◽

Cohen’S Kappa ◽

Accurate Identification ◽

Vessel Occlusion ◽

Test Characteristic ◽

Cohen's Kappa

Introduction: Expediting notification of lesions in acute ischemic stroke (AIS) is critical. Limited availability of experts to assess such lesions and delays in large vessel occlusion (LVO) recognition can negatively affect outcomes. Artificial intelligence (AI) may aid LVO recognition and treatment. This study aims to evaluate the performance of an AI-based algorithm for LVO detection in AIS. Methods: Retrospective analysis of a database of AIS patients admitted in a single center between 2014 and 2019. Vascular neurologists graded computed tomography angiographies (CTAs) for presence and site of LVO. Studies were analyzed by the Viz-LVO Algorithm® version 1.4 – neural network programmed to detect occlusions from the internal carotid artery terminus (ICA-T) to the Sylvian fissure. Comparisons between human versus AI-based readings were done by test characteristic analysis and Cohen’s kappa. Primary analysis included ICA-T and/or middle cerebral artery (MCA)-M1 LVOs versus non-LVOs/more distal occlusions. Secondary analysis included MCA-M2 occlusions. Results: 610 CTAs were analyzed. The AI algorithm rejected 2.5% of the CTAs due to poor quality, which were excluded from the analysis. Viz-LVO identified ICA-T and MCA-M1 LVOs with a sensitivity of 87.6%, specificity of 88.5%, and accuracy of 87.9% (AUC 0.88, 95% CI: 0.85–0.92, p < 0.001). Cohen’s kappa was 0.74. In the secondary analysis, the algorithm yielded a sensitivity of 80.3%, specificity of 88.5%, and accuracy of 82.7%. The mean run time of the algorithm was 2.78 ± 0.5 min. Conclusion: Automated AI reading allows for fast and accurate identification of LVO strokes with timely notification to emergency teams, enabling quick decision-making for reperfusion therapies or transfer to specialized centers if needed.

Download Full-text