Faculty Opinions recommendation of Inter-observer reliability of localization of recorded stridor sounds in children.

2010 ◽  
Author(s):  
Joseph L Edmonds, Jr ◽  
Julina Ongkasuwan
Keyword(s):  
Observer Reliability

The Reliability of Ultrasound in the Assessment of Hyaline Cartilage in Rheumatoid Arthritis and Healthy Metacarpal Heads

2020 ◽  
Author(s):  
Edoardo Cipolletta ◽  
Emilio Filippucci ◽  
Andrea Di Matteo ◽  
Giulia Tesei ◽  
Micaela Ana Cosatti ◽  
...  
Keyword(s):  
Rheumatoid Arthritis ◽  
Scoring System ◽  
Quantitative Assessment ◽  
Hyaline Cartilage ◽  
Scoring Systems ◽  
Thickness Measurement ◽  
Cartilage Thickness ◽  
Observer Agreement ◽  
Very High Frequency ◽  
Observer Reliability

Abstract Purpose i) To assess the inter- and intra-observer reliability of ultrasound (US) in the evaluation of the hyaline cartilage (HC) of the metacarpal head (MH) in patients with rheumatoid arthritis (RA) and in healthy subjects (HS) both qualitatively and quantitatively. ii) To calculate the smallest detectable difference (SDD) of the MH cartilage thickness measurement. iii) To correlate the qualitative scoring system and the quantitative assessment. Materials and Methods US examination was performed on 280 MHs of 20 patients with RA and 15 HS using a very high frequency probe (up to 22 MHz). HC status was evaluated both qualitatively (using a five-grade scoring system) and quantitatively (using the average value of the longitudinal and transverse measures). The HC of MHs from II to V metacarpophalangeal joint of both hands were scanned independently on the same day by two rheumatologists to assess inter-observer reliability. All subjects were re-examined using the same scanning protocol and the same US setting by one sonographer after a week to assess intra-observer reliability. Results The inter-observer agreement and intra-observer agreement were moderate to substantial (k = 0.66 and k = 0.73) for the qualitative scoring system and high (ICC = 0.93 and ICC = 0.94) for the quantitative assessment. The SDD of the MH cartilage thickness measurement was 0.09 mm. A significant correlation between the two scoring systems was found (r = –0.35; p < 0.001). Conclusion The present study describes the main methodological issues of HC assessment. Using a standardized protocol, both the qualitative and the quantitative scoring systems can be reliable.

scholarly journals Validation of iterative CT reconstruction by inter and intra observer performance assessment of artificial lung foci

2020 ◽  
Vol 6 (3) ◽  
pp. 534-537
Author(s):  
Britta König ◽  
Nika Guberina ◽  
Hilmar Kühl ◽  
Waldemar Zylka
Keyword(s):  
Low Dose ◽  
Lung Cancer Screening ◽  
Artificial Lung ◽  
Observer Performance ◽  
Subjective Perception ◽  
Ct Reconstruction ◽  
Perfect Agreement ◽  
Observer Reliability ◽  
Iterative Ct Reconstruction ◽  
Extension Ring

AbstractWe investigate the suitability of statistical and model-based iterative reconstruction (IR) algorithm strengths and their influence on image quality and diagnostic performance in low-dose computer tomography (CT) protocols for lung-cancer screening procedures. We evaluate the inter- and intra-observer performance for the assessment of iterative CT reconstruction. Artificial lung foci shaped as spheres and spicules made from material with calibrated Hounsfield units were pressed within layered granules in lung lobes of an anthropomorphic phantom. Adaptively, a soft-tissue- and fat- extension ring were attached. The phantom with foci was scanned using standard high contrast, low-dose and ultra lowdose protocols. For reconstruction the IR algorithm ADMIRE at four different strength levels were used. Two ranking tests and Friedman statistics were performed. Fleiss k and modified Cohen’s kneywere used to quantify inter- and intra-observer performance. In conjunction with the standard lung kernel BL75 radiologists evaluated medium to high IR strength, with preference to S4, as suitable for lung foci detection. When varying reconstruction kernels the ranking became more random than with varying phantom diameter. The inter-observer reliability shows poor to slight agreement expressed by k<0 and k=0-0.20 . For the intra-observer reliability non- agreement with kney=0-0.20and moderate agreement with kney=0.60-0.79 for the first ranking test, and almost perfect agreement with kney>0.90 for the second ranking test was observed. In conclusion, our validation suggests radiological preference of medium to high iteration strengths, especially S4, for lung foci detection. An investigation of the correlation between diagnostic experience and the subjective perception of IR reconstructed CT images still needs to be investigated.

Inter-observer reliability of a risk assessment model for venous thromboembolism in acutely-ill medical hospitalized patients: Results from a prospective cohort study

2021 ◽  
pp. 026835552110212
Author(s):  
Cassia RL Ferreira ◽  
Marcos de Bastos ◽  
Mirella L Diniz ◽  
Renan A Mancini ◽  
Yan S Raposo ◽  
...  
Keyword(s):  
Risk Factors ◽  
Cohort Study ◽  
Venous Thromboembolism ◽  
Prospective Cohort Study ◽  
Prospective Cohort ◽  
Risk Classification ◽  
Assessment Model ◽  
Kappa Statistics ◽  
Medical Patients ◽  
Observer Reliability

Objectives To analyze the inter-observer reliability of risk for venous thromboembolism (VTE) in a population of adult acutely-ill medical patients. Methods In this prospective cohort study, we collected risk factors and risk classification for VTE using RAM IMPROVE7. Kappa statistics was used to evaluate inter-observer reliability between lead clinicians and trained researchers. We evaluated occurrence of VTE in patients with mismatched classification. Results We included 2,380 patients, median age 70 years (interquartile range [IQR], 58-79), 56.2% female. Adjusted Kappa for VTE risk factors ranged from substantial (0.64, 95% confidence interval [CI], 0.61-0.67) for “immobilization”, to almost perfect (0.98; 95% CI 0.97-0.99) for “thrombophilia”; risk classification was 0.64 (95% CI 0.60-0.67). Divergent risk classification occurred in 434 patients (18.2%) of whom seven (1.6%) developed VTE. Conclusion Despite substantial to almost perfect reliability between observers for risk factors and risk classification, lead clinicians tended to underestimate the risk for VTE.

scholarly journals Evaluation of Inter-Observer Reliability of Animal Welfare Indicators: Which Is the Best Index to Use?

Animals ◽  
2021 ◽  
Vol 11 (5) ◽  
pp. 1445
Author(s):  
Mauro Giammarino ◽  
Silvana Mattiello ◽  
Monica Battini ◽  
Piero Quatto ◽  
Luca Maria Battaglini ◽  
...  
Keyword(s):  
Animal Welfare ◽  
Confidence Intervals ◽  
Bootstrap Method ◽  
Dairy Goat ◽  
Microsoft Excel ◽  
Bootstrap Methods ◽  
Welfare Indicators ◽  
Observer Reliability ◽  
High Concordance ◽  
Variance Estimates

This study focuses on the problem of assessing inter-observer reliability (IOR) in the case of dichotomous categorical animal-based welfare indicators and the presence of two observers. Based on observations obtained from Animal Welfare Indicators (AWIN) project surveys conducted on nine dairy goat farms, and using udder asymmetry as an indicator, we compared the performance of the most popular agreement indexes available in the literature: Scott’s π, Cohen’s k, kPABAK, Holsti’s H, Krippendorff’s α, Hubert’s Γ, Janson and Vegelius’ J, Bangdiwala’s B, Andrés and Marzo’s ∆, and Gwet’s γ(AC1). Confidence intervals were calculated using closed formulas of variance estimates for π, k, kPABAK, H, α, Γ, J, ∆, and γ(AC1), while the bootstrap and exact bootstrap methods were used for all the indexes. All the indexes and closed formulas of variance estimates were calculated using Microsoft Excel. The bootstrap method was performed with R software, while the exact bootstrap method was performed with SAS software. k, π, and α exhibited a paradoxical behavior, showing unacceptably low values even in the presence of very high concordance rates. B and γ(AC1) showed values very close to the concordance rate, independently of its value. Both bootstrap and exact bootstrap methods turned out to be simpler compared to the implementation of closed variance formulas and provided effective confidence intervals for all the considered indexes. The best approach for measuring IOR in these cases is the use of B or γ(AC1), with bootstrap or exact bootstrap methods for confidence interval calculation.

scholarly journals PD-L1 Testing and Squamous Cell Carcinoma of the Head and Neck: A Multicenter Study on the Diagnostic Reproducibility of Different Protocols

Cancers ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 292
Author(s):  
Simona Crosta ◽  
Renzo Boldorini ◽  
Francesca Bono ◽  
Virginia Brambilla ◽  
Emanuele Dainese ◽  
...  
Keyword(s):  
Squamous Cell Carcinoma ◽  
Cell Carcinoma ◽  
Head And Neck ◽  
Squamous Cell ◽  
Checkpoint Inhibitors ◽  
Interobserver Reliability ◽  
Tissue Microarrays ◽  
Observer Agreement ◽  
Alternative Methods ◽  
Observer Reliability

Immune checkpoint inhibitors for blocking the programmed cell death protein 1 (PD-1)/programmed death-ligand 1 (PD-L1) axis are now available for squamous cell carcinoma of the head and neck (HNSCC) in relapsing and/or metastatic settings. In this work, we compared the resulting combined positive score (CPS) of PD-L1 using alternative methods adopted in routine clinical practice and determined the level of diagnostic agreement and inter-observer reliability in this setting. The study applied 5 different protocols on 40 tissue microarrays from HNSCC. The error rate of the individual protocols ranged from a minimum of 7% to a maximum of 21%, the sensitivity from 79% to 96%, and the specificity from 50% to 100%. In the intermediate group (1 ≤ CPS < 20), the majority of errors consisted of an underestimation of PD-L1 expression. In strong expressors, 5 out of 14 samples (36%) were correctly evaluated by all the protocols, but no protocol was able to correctly identify all the “strong expressors”. The overall inter-observer agreement in PD-L1 CPS reached 87%. The inter-observer reliability was moderate, with an ICC of 0.774 (95% CI (0.651; 0.871)). In conclusion, our study showed moderate interobserver reliability among different protocols. In order to improve the performances, adequate specific training to evaluate PD-L1 by CPS in the HNSCC setting should be coordinated.

Sign in / Sign up

Export Citation Format

Share Document