Abstract WP276: Simplification of a Prehospital Short NIHSS Scale Does not Increase Interrater Agreement Between Emergency Medical Services and Stroke Specialists

Objective: To assess if simplifying a prehospital 8-item NIHSS scale (NIHSS-8, fig 1) to a 0 (symptom absent) – 1 (symptom present) scoring system increases interrater agreement between emergency medical services (EMS) and stroke specialists. Methods: We analysed interrater agreement between EMS and stroke specialists of a single centre on a prospectively collected cohort of 64 suspected acute ischemic stroke patients. EMS performed NIHSS-8 score upon patient arrival at the emergency department. The stroke specialist scored the full 15-item NIHSS blind to the EMS scores and within 5 minutes of patient arrival. Linear-weighted Cohen’s kappa statistics was used to assess agreement between EMS and stroke specialist on the total NIHSS-8 score and each NIHSS-8 scale item. We then simplified each item to a 0-1 score and reassessed interrater agreement for the overall NIHSS-8 scale using linear-weighted Cohen’s kappa statistics and for each NIHSS-8 item using Cohen’s kappa statistics. We used Cohen’s kappa statistics to assess agreement for original and simplified NIHSS-8 cut-off scores. Results: EMS and stroke specialist reached substantial agreement on overall NIHSS-8 scoring (linear-weighted kappa 0.69). Optimum agreement was reached for right arm weakness (linear-weighted kappa 0.79; Table 1) and a cut-off score of 2 and 5 (Cohen’s kappa 0.78; Table 2). When the score was simplified to a 0-1, overall agreement between EMS and stroke specialists was substantial (linear-weighted kappa 0.65). Optimum agreement was seen for LOC questions (Cohen’s kappa 0.78; Table 1) and a cut-off score of 2 (Cohen’s kappa 0.77; Table 2). Conclusion: Simplifying an 8-item prehospital NIHSS stroke scale does not increase interrater agreement between emergency medical services and stroke specialists.

Download Full-text

Frequency and severity of prehospital obstetric events encountered by emergency medical services in the United States

BMC Pregnancy and Childbirth ◽

10.1186/s12884-021-04129-1 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Rebecca E. Cash ◽

Robert A. Swor ◽

Margaret Samuels-Kalow ◽

David Eisenbrey ◽

Anjali J. Kaimal ◽

...

Keyword(s):

Emergency Medical Services ◽

Prehospital Care ◽

The United States ◽

Medical Services ◽

Patient Acuity ◽

Cohen’S Kappa ◽

Specific Patient ◽

Cohen's Kappa ◽

Emergency Medical ◽

High Acuity

Abstract Background Prehospital obstetric events encountered by emergency medical services (EMS) can be high-risk patient presentations for which suboptimal care can cause substantial morbidity and mortality. The frequency of prehospital obstetric events is unclear because existing descriptions have reported obstetric and gynecological conditions together, without delineating specific patient presentations. Our objective was to identify the types, frequency, and acuity of prehospital obstetric events treated by EMS personnel in the US. Methods We conducted a cross-sectional analysis of EMS patient care records in the 2018 National EMS Information System dataset (n=22,532,890). We focused on EMS activations (i.e., calls for service) for an emergency scene response for patients aged 12-50 years with evidence of an obstetric event. Type of obstetric event was determined by examining patient symptoms, the treating EMS provider’s impression (i.e., field diagnosis), and procedures performed. High patient acuity was ascertained by EMS documentation of patient status and application of the modified early obstetric warning system (MEOWS) criteria, with concordance assessed using Cohen’s kappa. Descriptive statistics were calculated to describe the primary symptoms, impressions, and frequency of each type of obstetric event among these activations. Results A total of 107,771 (0.6%) of EMS emergency activations were identified as involving an obstetric event. The most common presentation was early or threatened labor (15%). Abdominal complaints, including pain and other digestive/abdomen signs and symptoms, was the most common primary symptom (29%) and primary impression (18%). We identified 3,489 (3%) out-of-hospital deliveries, of which 1,504 were preterm. Overall, EMS providers documented 34% of patients as being high acuity, similar to the MEOWS criteria (35%); however, there were high rates of missing data for EMS documented acuity (19%), poor concordance between the two measures (Cohen’s kappa=0.12), and acuity differences for specific conditions (e.g., high acuity of non-cephalic presentations, 77% in EMS documentation versus 53% identified by MEOWS). Conclusion Prehospital obstetric events were infrequently encountered by EMS personnel, and about one-third were high acuity. Additional work to understand the epidemiology and clinical care of these patients by EMS would help to optimize prehospital care and outcomes.

Download Full-text

New Interpretations of Cohen’s Kappa

Journal of Mathematics ◽

10.1155/2014/203907 ◽

2014 ◽

Vol 2014 ◽

pp. 1-9 ◽

Cited By ~ 7

Author(s):

Matthijs J. Warrens

Keyword(s):

Weighted Kappa ◽

Interrater Agreement ◽

Cohen’S Kappa ◽

Association Coefficient ◽

Nominal Scale ◽

Weighted Averages ◽

Cohen's Kappa ◽

Single Number ◽

Research Studies

Cohen’s kappa is a widely used association coefficient for summarizing interrater agreement on a nominal scale. Kappa reduces the ratings of the two observers to a single number. With three or more categories it is more informative to summarize the ratings by category coefficients that describe the information for each category separately. Examples of category coefficients are the sensitivity or specificity of a category or the Bloch-Kraemer weighted kappa. However, in many research studies one is often only interested in a single overall number that roughly summarizes the agreement. It is shown that both the overall observed agreement and Cohen’s kappa are weighted averages of various category coefficients and thus can be used to summarize these category coefficients.

Download Full-text

Fixed-Effects Modeling of Cohen's Weighted Kappa for Bivariate Multinomial Data: A Perspective of Generalized Inverse

Journal of Probability and Statistics ◽

10.1155/2011/603856 ◽

2011 ◽

Vol 2011 ◽

pp. 1-14

Author(s):

Jingyun Yang ◽

Vernon M. Chinchilli

Keyword(s):

Fixed Effects ◽

Generalized Inverse ◽

Weighted Kappa ◽

Small Sample ◽

Kappa Statistics ◽

Cohen’S Kappa ◽

Multinomial Data ◽

Cohen's Kappa ◽

Bootstrap Simulation ◽

Measuring Agreement

Cohen's kappa and weighted kappa statistics are the conventional methods used frequently in measuring agreement for categorical responses. In this paper, through the perspective of a generalized inverse, we propose an alternative general framework of the fixed-effects modeling of Cohen's weighted kappa, proposed by Yang and Chinchilli (2011). Properties of the proposed method are provided. Small sample performance is investigated through bootstrap simulation studies, which demonstrate good performance of the proposed method. When there are only two categories, the proposed method reduces to Cohen's kappa.

Download Full-text