scholarly journals An inter- and intra-rater agreement assessment of a novel classification of pyogenic spinal infections

2022 ◽  
Author(s):  
Gaston Camino-Willhuber ◽  
Byron Delgado ◽  
Nelson Astur ◽  
Alfredo Guiroy ◽  
Marcelo Valacco ◽  
...  
Keyword(s):  
Rater Agreement ◽  
Agreement Assessment ◽  
Spinal Infections

scholarly journals Interobserver agreement of the Paris and simplified classifications of superficial colonic lesions: a Western study

2021 ◽  
Vol 09 (03) ◽  
pp. E388-E394
Author(s):  
Francesco Cocomazzi ◽  
Marco Gentile ◽  
Francesco Perri ◽  
Antonio Merla ◽  
Fabrizio Bossa ◽  
...  
Keyword(s):  
Interobserver Agreement ◽  
Classification Systems ◽  
Sensitivity Analyses ◽  
Size Estimation ◽  
Rater Agreement ◽  
Staff Members ◽  
Video Clips ◽  
Paris Classification ◽  
Level Of Agreement

Abstract Background and study aims The Paris classification of superficial colonic lesions has been widely adopted, but a simplified description that subgroups the shape into pedunculated, sessile/flat and depressed lesions has been proposed recently. The aim of this study was to evaluate the accuracy and inter-rater agreement among 13 Western endoscopists for the two classification systems. Methods Seventy video clips of superficial colonic lesions were classified according to the two classifications, and their size estimated. The interobserver agreement for each classification was assessed using both Cohen k and AC1 statistics. Accuracy was taken as the concordance between the standard morphology definition and that made by participants. Sensitivity analyses investigated agreement between trainees (T) and staff members (SM), simple or mixed lesions, distinct lesion phenotypes, and for laterally spreading tumors (LSTs). Results Overall, the interobserver agreement for the Paris classification was substantial (κ = 0.61; AC1 = 0.66), with 79.3 % accuracy. Between SM and T, the values were superimposable. For size estimation, the agreement was 0.48 by the κ-value, and 0.50 by AC1. For single or mixed lesions, κ-values were 0.60 and 0.43, respectively; corresponding AC1 values were 0.68 and 0.57. Evaluating the several different polyp subtypes separately, agreement differed significantly when analyzed by the k-statistics (0.08–0.12) or the AC1 statistics (0.59–0.71). Analyses of LSTs provided a κ-value of 0.50 and an AC1 score of 0.62, with 77.6 % accuracy. The simplified classification outperformed the Paris classification: κ = 0.68, AC1 = 0.82, accuracy = 91.6 %. Conclusions Agreement is often measured with Cohen’s κ, but we documented higher levels of agreement when analyzed with the AC1 statistic. The level of agreement was substantial for the Paris classification, and almost perfect for the simplified system.

A Preliminary Classification of Usage Measures in Information System Acceptance

2013 ◽  
pp. 105-129 ◽  
Author(s):  
Muhammad Z. I. Lallmahomed ◽  
Nor Zairah Ab Rahim ◽  
Roliana Ibrahim ◽  
Azizah Abdul Rahman
Keyword(s):  
Information System ◽  
Technology Acceptance ◽  
Rater Agreement ◽  
Dimensional Measure ◽  
System Use ◽  
System Acceptance ◽  
Acceptance Model ◽  
Q Sort ◽  
Preliminary Classification

In the light of a diverse body of disorganized usage measures available and the difficulty of building a cumulative research tradition, a literature review is conducted on system use in Information Systems (IS) Acceptance through the two main theories of Technology Adoption, the Technology Acceptance Model (TAM), and The Unified Theory of Use and Acceptance of Technology (UTAUT). The authors seek to understand how usage measures are being operationalised and proposed a preliminary classification of those measures that covers system and task aspects of use. A Q-Sort approach was taken to validate the authors’ classification scheme and the result indicates high inter-rater agreement. The ensuing classification is meant to help researchers in their choice of system use measures. This review also summarises the arguments for a multi-dimensional measure of use and establishes that omnibus measure such as frequency, volume and use/non-use hold prevalence. Finally, the authors provide recommendations for further research in the area of system use.

Overview and Classification of Spinal Infections

1996 ◽  
Vol 27 (1) ◽  
pp. 1-8 ◽  
Author(s):  
Rocco R. Calderone ◽  
John M. Larsen
Keyword(s):  
Spinal Infections

scholarly journals Manual Annotation of Time in Bed Using Free-Living Recordings of Accelerometry Data

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8442
Author(s):  
Esben Lykke Skovgaard ◽  
Jesper Pedersen ◽  
Niels Christian Møller ◽  
Anders Grøntved ◽  
Jan Christian Brønd
Keyword(s):  
Correlation Coefficients ◽  
Accelerometer Data ◽  
Manual Annotation ◽  
Rater Agreement ◽  
Sleep Diaries ◽  
Annotation Method ◽  
Time In Bed ◽  
Interclass Correlation ◽  
Novel Method

With the emergence of machine learning for the classification of sleep and other human behaviors from accelerometer data, the need for correctly annotated data is higher than ever. We present and evaluate a novel method for the manual annotation of in-bed periods in accelerometer data using the open-source software Audacity®, and we compare the method to the EEG-based sleep monitoring device Zmachine® Insight+ and self-reported sleep diaries. For evaluating the manual annotation method, we calculated the inter- and intra-rater agreement and agreement with Zmachine and sleep diaries using interclass correlation coefficients and Bland–Altman analysis. Our results showed excellent inter- and intra-rater agreement and excellent agreement with Zmachine and sleep diaries. The Bland–Altman limits of agreement were generally around ±30 min for the comparison between the manual annotation and the Zmachine timestamps for the in-bed period. Moreover, the mean bias was minuscule. We conclude that the manual annotation method presented is a viable option for annotating in-bed periods in accelerometer data, which will further qualify datasets without labeling or sleep records.

Inter-rater Agreement of the Classification of Intra-operative Adverse Events (ClassIntr®) in Abdominal Surgery

2021 ◽  
Vol Publish Ahead of Print ◽  
Author(s):  
P. Krielen ◽  
L. Gawria ◽  
M. W. J. Stommel ◽  
S. Dell-Kuster ◽  
R. Rosenthal ◽  
...  
Keyword(s):  
Adverse Events ◽  
Abdominal Surgery ◽  
Rater Agreement

The Tulip classification of perinatal mortality: introduction and multidisciplinary inter-rater agreement

2006 ◽  
Vol 113 (4) ◽  
pp. 393-401 ◽  
Author(s):  
FJ Korteweg ◽  
SJ Gordijn ◽  
A Timmer ◽  
JJHM Erwich ◽  
KA Bergman ◽  
...  
Keyword(s):  
Perinatal Mortality ◽  
Rater Agreement

scholarly journals Accuracy and reliability of injury coding in the national Dutch Trauma Registry

2021 ◽  
Vol 33 (1) ◽  
Author(s):  
Eric Twiss ◽  
Pieta Krijnen ◽  
Inger Schipper
Keyword(s):  
Outcome Measures ◽  
Injury Severity ◽  
Trauma Registry ◽  
Nationwide Survey ◽  
Secondary Outcome ◽  
Trauma Patients ◽  
Rater Agreement ◽  
Level I ◽  
And Training

Abstract Objective Injury coding is well known for lack of completeness and accuracy. The objective of this study was to perform a nationwide assessment of accuracy and reliability on Abbreviated Injury Scale (AIS) coding by Dutch Trauma Registry (DTR) coders and to determine the effect on Injury Severity Score (ISS). Additionally, the coders’ characteristics were surveyed. Methods Three fictional trauma cases were presented to all Dutch trauma coders in a nationwide survey (response rate 69%). The coders were asked to extract and code the cases’ injuries according to the AIS manual (version 2005, update 2008). Reference standard was set by three highly experienced coders. Summary statistics were used to describe the registered AIS codes and ISS distribution. The primary outcome measures were accuracy of injury coding and inter-rater agreement on AIS codes. Secondary outcome measures were characteristics of coders: profession, work setting, experience in injury coding and training level in injury coding. Results The total number of different AIS codes used to describe 14 separate injuries in the three cases was 89. Mean accuracy per AIS code was 42.2% (range 2.4–92.7%). Mean accuracy on number of AIS codes was 23%. Overall inter-rater agreement per AIS code was 49.1% (range 2.4–92.7%). The number of assigned AIS codes varied between 0 and 18 per injury. Twenty-seven percentage of injuries were overlooked. ISS was correctly scored in 42.4%. In 31.7%, the AIS coding of the two more complex cases led to incorrect classification of the patient as ISS < 16 or ISS ≥ 16. Half (47%) of the coders had no (para)medical degree, 26% were working in level I trauma centers, 37% had less than 2 years of experience and 40% had no training in AIS coding. Conclusions Accuracy of and inter-rater agreement on AIS injury scoring by DTR coders is limited. This may in part be due to the heterogeneous backgrounds and training levels of the coders. As a result of the inconsistent coding, the number of major trauma patients in the DTR may be over- or underestimated. Conclusions based on DTR data should therefore be drawn with caution.

scholarly journals The “DICA” Endoscopic Classification for Diverticular Disease of the Colon Shows a Significant Interobserver Agreement among Community Endoscopists

2019 ◽  
Vol 28 (1) ◽  
pp. 23-27
Author(s):  
Antonio Tursi ◽  
Giovanni Brandimarte ◽  
Francesco Di Mario ◽  
Gerardo Nardone ◽  
Carmelo Scarpignato ◽  
...  
Keyword(s):  
Diverticular Disease ◽  
Interobserver Agreement ◽  
Expert Group ◽  
Rater Agreement ◽  
Single Item ◽  
Statistical Measures ◽  
Endoscopic Classification ◽  
Severity Of The Disease ◽  
Endoscopic Videos

Background & Aim: An endoscopic classification of Diverticular Disease (DD), called DICA (Diverticular Inflammation and Complication Assessment) is currently available. It scores severity of the disease as DICA 1, DICA 2 and DICA 3. Our aim was to assess the agreement levels for this classification among an endoscopist community setting.Methods: A total of 66 endoscopists independently scored a set of DD endoscopic videos. The percentages of overall agreement on the DICA score and a free-marginal multirater kappa (κ) coefficient were reported as statistical measures of the inter-rater agreement.Results: The overall agreement levels were: 70.2% for DICA 1, 70.5% for DICA 2, 81.3% for DICA 3. The free marginal κ was: 0.553 for DICA 1, 0.558 for DICA 2, 0.719 for DICA 3. The agreement levels among the expert group were: 78.8% for DICA 1, 80.2% for DICA 2, 88.5% for DICA 3. The free marginal κ among the expert group were: 0.682 for DICA 1, 0.712 for DICA 2, 0.828 for DICA 3. The agreement of expert raters on the single item of the DICA classification was superior to the agreement of the overall group.Conclusions: The overall inter-rater agreement for DICA score in this study ranges from moderate to good, with a significant improvement in the expert subgroup of raters. Diverticular Inflammation and Complication Assessment is a simple and reproducible endoscopic scoring system.

Sign in / Sign up

Export Citation Format

Share Document