scholarly journals High inter-rater reliability of Japanese bedriddenness ranks and cognitive function scores: a hospital-based prospective observational study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Masaki Tago ◽  
Naoko E. Katsuki ◽  
Shizuka Yaita ◽  
Eiji Nakatani ◽  
Shun Yamashita ◽  
...  
Keyword(s):  
Cognitive Function ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Barthel Index ◽  
Prospective Observational Study ◽  
Intraclass Correlation ◽  
Kappa Coefficient ◽  
Concordance Rate ◽  
Rater Reliability ◽  
Katz Index

Abstract Background The statistical validities of the official Japanese classifications of activities of daily living (ADLs), including bedriddenness ranks (BR) and cognitive function scores (CFS), have yet to be assessed. To this aim, we evaluated the ability of BR and CFS to assess ADLs using inter-rater reliability and criterion-related validity. Methods New inpatients aged ≥75 years were enrolled in this hospital-based prospective observational study. BR and CFS were assessed once by an attending nurse, and then by a social worker/medical clerk. We evaluated inter-rater reliability between different professions by calculating the concordance rate, kappa coefficient, Cronbach’s α, and intraclass correlation coefficient. We also estimated the relationship of the Barthel Index and Katz Index with the BR and CFS using Spearman’s correlation coefficients. Results For the 271 patients enrolled, BR at the first assessment revealed 66 normal, 10 of J1, 15 of J2, 18 of A1, 31 of A2, 37 of B1, 35 of B2, 22 of C1, and 32 of C2. The concordance rate between the two BR assessments was 68.6%, with a kappa coefficient of 0.61, Cronbach’s α of 0.91, and an intraclass correlation coefficient of 0.83, thus showing good inter-rater reliability. BR was negatively correlated with the Barthel Index (r = − 0.848, p < 0.001) and Katz Index (r = − 0.820, p < 0.001), showing justifiable criterion-related validity. Meanwhile, CFS at the first assessment revealed 92 normal, 47 of 1, 19 of 2a, 30 of 2b, 60 of 3a, 8 of 3b, 8 of 4, and 0 of M. The concordance rate between the two CFS assessments was 70.1%, with a kappa coefficient of 0.62, Cronbach’s α of 0.87, and an intraclass correlation coefficient 0.78, thus also showing good inter-rater reliability. CFS was negatively correlated with the Barthel Index (r = − 0.667, p < 0.001) and Katz Index (r = − 0.661, p < 0.001), showing justifiable criterion-related validity. Conclusions BR and CFS could be reliable and easy-to-use grading scales of ADLs in acute clinical practice or large-scale screening, with high inter-rater reliabilities among different professions and significant correlations with well-established, though complicated to use, instruments to assess ADLs. Trial registration UMIN000041051 (2020/7/10).

User testing of the psychometric properties of pictorial-based disability assessment Longshi Scale by healthcare professionals and non-professionals: a Chinese study in Shenzhen

2019 ◽  
Vol 33 (9) ◽  
pp. 1479-1491 ◽  
Author(s):  
Yulong Wang ◽  
Shanshan Guo ◽  
Jiejiao Zheng ◽  
Qing Mei Wang ◽  
Yuling Zhang ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Barthel Index ◽  
Public Hospital ◽  
Healthcare Professionals ◽  
Intraclass Correlation ◽  
Random Effect ◽  
Second Phase ◽  
Spearman Correlation ◽  
Rater Reliability

Objective:The aim of this study was to validate a novel pictorial-based Longshi Scale for evaluating a patient’s disability by healthcare professionals and non-professionals.Design:Prospective study.Setting:Rehabilitation departments from a grade A, class 3 public hospital, a grade B, class 2 public hospital, and a private hospital and seven community rehabilitation centers.Subjects:A total of 618 patients and 251 patients with functional disabilities were recruited in a two-phase study, respectively.Main measures:Outcome measure: pictorial scale of activities of daily living (ADLs, Longshi Scale). Reference measure: Barthel Index. The Spearman correlation coefficient was used to analyze the validity of Longshi Scale against Barthel Index.Results:In phase 1 study, from March 2016 to August 2016, the results demonstrated that the Longshi Scale was both reliable and valid (intraclass correlation coefficient based on two-way random effect (ICC2,1) = 0.877–0.974 for intra-rater reliability; ICC2,1= 0.928–0.979; κ = 0.679–1.000 for inter-rater reliability; intraclass correlation coefficient based on one-way random effect (ICC1,1) = 0.921–0.984 for test–retest reliability and Spearman correlation coefficient = 0.836–0.899). In the second phase, in March 2018, results further demonstrated that the Longshi Scale had good inter-rater and intra-rater reliability among healthcare professionals and non-professionals including therapists, interns, and personal care aids (ICC1,1= 0.822–0.882 on Day 1; ICC1,1= 0.842–0.899 on Day 7 for inter-rater reliability). In addition, the Longshi Scale decreased assessment time significantly, compared with the Barthel Index assessment ( P < 0.01).Conclusion:The Longshi Scale could potentially provide an efficient way for healthcare professionals and non-professionals who may have minimal training to assess the ADLs of functionally disabled patients.

scholarly journals Intra-rater reliability of transversus abdominis measurement by a novice examiner: Comparison of “freehand” to “probe force device” method of real-time ultrasound imaging

Ultrasound ◽  
2019 ◽  
Vol 27 (3) ◽  
pp. 156-166 ◽  
Author(s):  
Vanessa L Kennedy ◽  
Carol A Flavell ◽  
Kenji Doma
Keyword(s):  
Measurement Error ◽  
Real Time ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Coefficient Of Variation ◽  
Repeated Measures ◽  
Intraclass Correlation ◽  
Transversus Abdominis ◽  
Rater Reliability ◽  
Transverse Abdominis

A “free hand” real-time-ultrasound method is commonly applied to measure transversus abdominis. Potentially, this increases transversus abdominis measurement error due to uncontrolled variability in probe to skin force, inclination, and roll, particularly for novice examiners. This single-group repeated-measures reliability study compared the intra-rater reliability of transversus abdominis thickness and activation measurement by a novice examiner between free hand and a standardized probe force device method. The examiner captured ultrasound videos of transversus abdominis in a single session in healthy participants ( n =  33). Free hand ultrasound featured uncontrolled probe force, inclination, and roll, while probe force device method ultrasound standardized these parameters. Images of transversus abdominis at rest and contracted were measured and transversus abdominis activation calculated. Intraclass correlation coefficient, coefficient of variation, standard error of measurement, and worthwhile differences were calculated. The probe force device method resulted in greater reliability (intraclass correlation coefficient = 0.75–0.96) and lower measurement error (coefficient of variation = 8.89–28.7%) compared to free hand (intraclass correlation coefficient = 0.63–0.93; coefficient of variation = 6.52–29.4%). Reliability was good for all measurements except free hand TrA-C, which was moderate. TrA-C had the lowest reliability, followed by contracted thickness of the transverse abdominis, with resting thickness of the transverse abdominis being highest. Worthwhile differences were lower using a probe force device method versus free hand for resting thickness of the transverse abdominis and contracted thickness of the transverse abdominis and similar for TrA-C. Standardization using probe force device method ultrasound to measure transversus abdominis improved intra-rater reliability in a novice examiner. Use of a probe force device method is recommended to improve reliability through reduced sources of measurement error. Probe force device method intra- and inter-rater reliability in examiners of varying experience, in clinical populations, and to visualize other structures merits exploration.

scholarly journals Validity and reliability of smartphone-based application for chronic ankle instability

2021 ◽  
Vol 28 (9) ◽  
pp. 1-10
Author(s):  
Taelim Yoon ◽  
Jihyun Lee
Keyword(s):  
Medical Device ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Ankle Instability ◽  
Intraclass Correlation ◽  
Validity And Reliability ◽  
Rater Reliability ◽  
Eyes Closed ◽  
Eyes Open ◽  
Cumberland Ankle Instability Tool

Background/aims Ankle instability is one of the most common injuries that can occur during everyday life, sports and exercise. Recently, smartphone accelerometers have been used to measure single leg balance associated with ankle instability, because they are easy to use, inexpensive and can be used in small spaces. Thus, the purpose of this study was to introduce and investigate the intra- and inter-rater reliability of the smartphone accelerometer when assessing ankle instability. Methods A total of 26 individuals who had ankle instability were recruited. The single leg stance balance was measured using a smartphone accelerometer (Accelerometer application) and a force platform (I-Balance) for 5 seconds with their eyes open or their eyes closed. Results In the eyes open position, intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.87–0.90); and the inter-rater reliability was moderate (intraclass correlation coefficient: 0.71). In the eyes closed position, the intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.90–0.93); the inter-rater reliability was good (intraclass correlation coefficient: 0.82). Additionally, there were fair positive correlations between the smartphone accelerometer and the Cumberland Ankle Instability Tool, and between the smartphone accelerometer and I-Balance (r=0.33, 0.30 respectively). Conclusions The present study demonstrated excellent intra-rater reliabilities of two raters and moderate to good inter-rater reliabilities. The smartphone accelerometer offers several important advantages as a potential portable medical device to assess ankle instability accurately. Although there was a positive correlation, the relationships between the smartphone accelerometer and Cumberland Ankle Instability Tool and that between the smartphone accelerometer and I-Balance were fair. Future studies should investigate the validity of the smartphone accelerometer as a portable medical device for determining ankle instability.

scholarly journals Intra- and inter-rater reliability in a comparative study of cross-sectional and spiral computed tomography pelvimetry methods

2019 ◽  
Vol 8 (6) ◽  
pp. 205846011985518 ◽  
Author(s):  
Erika Phexell ◽  
Anna Åkesson ◽  
Marcus Söderberg ◽  
Anetta Bolejko
Keyword(s):  
Computed Tomography ◽  
Radiation Dose ◽  
Confidence Interval ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Measurement Reliability ◽  
Cross Sectional ◽  
Rater Reliability ◽  
Short Spiral

Background Different low-dose computed tomography (CT) pelvimetry methods can be used to evaluate the size of birth canal before delivery. CT pelvimetry might generate an acceptable low fetal radiation dose but its measurement accuracy is unknown. Purpose To investigate intra- and inter-rater measurement reliability of cross-sectional and two spiral CT pelvimetry methods: standard spiral and short spiral. Material and Methods Ten individuals (age ≥60 years, body mass index ≥30 kg/m2) having a CT scan of the abdomen also had CT pelvimetry scans. Three radiologists made independent measurements of each pelvimetry method on two occasions and also in consensus for a reference pelvimetry computed from the standard-dose CT scan of the abdomen. Inter- and intra-rater reliability was analyzed by intraclass correlation coefficient. Results Measurements in the short spiral pelvimetry demonstrated excellent intra- and inter-rater reliability, intraclass correlation coefficient ≥0.93, and good to excellent 95% confidence interval 0.87–0.99. Corresponding results of the standard spiral and cross-sectional pelvimetry showed good to excellent intraclass correlation coefficient ≥0.85 and ≥0.76, and 95% confidence interval was least good and moderate 0.73–0.98 and 0.59–0.97, respectively. Intraclass correlation coefficient between reference pelvimetry and other CT methods showed analogous results. Conclusion The short spiral pelvimetry demonstrated high and best reliability in comparison to other methods. Standard spiral method showed also good measurement reliability but the short spiral pelvimetry generates lower fetal radiation dose. This method might be suitable for measurements at narrow pelvis. Patient acceptance and attitude to CT pelvimetry should be investigated.

scholarly journals Development of the Huddle Observation Tool for structured case management discussions to improve situation awareness on inpatient clinical wards

2017 ◽  
Vol 27 (5) ◽  
pp. 365-372 ◽  
Author(s):  
Julian Edbrooke-Childs ◽  
Jacqueline Hayes ◽  
Evelyn Sharples ◽  
Dawid Gondek ◽  
Emily Stapley ◽  
...  
Keyword(s):  
Case Management ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Situation Awareness ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Weighted Kappa ◽  
Observational Assessment ◽  
Rater Reliability ◽  
Team Processes

Background‘Situation Awareness For Everyone’ (SAFE) was a 3-year project which aimed to improve situation awareness in clinical teams in order to detect potential deterioration and other potential risks to children on hospital wards. The key intervention was the ‘huddle’, a structured case management discussion which is central to facilitating situation awareness. This study aimed to develop an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle.MethodsA cross-sectional observational design was used to psychometrically develop the ‘Huddle Observation Tool’ (HOT) over three phases using standardised psychometric methodology. Huddles were observed across four NHS paediatric wards participating in SAFE by five researchers; two wards within specialist children hospitals and two within district general hospitals, with location, number of beds and length of stay considered to make the sample as heterogeneous as possible. Inter-rater reliability was calculated using the weighted kappa and intraclass correlation coefficient.ResultsInter-rater reliability was acceptable for the collaborative culture (weighted kappa=0.32, 95% CI 0.17 to 0.42), environment items (weighted kappa=0.78, 95% CI 0.52 to 1) and total score (intraclass correlation coefficient=0.87, 95% CI 0.68 to 0.95). It was lower for the structure and risk management items, suggesting that these were more variable in how observers rated them. However, agreement on the global score for huddles was acceptable.ConclusionWe developed an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle. Future research should examine whether observational evaluations of huddles are associated with other indicators of safety on clinical wards (eg, safety climate and incidents of patient harm), and whether scores on the HOT are associated with improved situation awareness and reductions in deterioration and adverse events in clinical settings, such as inpatient wards.

scholarly journals Further Data on the Reliability of the Mentalization Imbalances Scale and of the Modes of Mentalization Scale

2020 ◽  
Vol 23 (1) ◽  
Author(s):  
Giulia Gagliardini ◽  
Laura Gatti ◽  
Antonello Colli
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Test Reliability ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

The aim of this study was to provide data on the Inter-Rater Reliability (IRR) and the test-retest reliability of the Mentalization Imbalances Scale (MIS) and the Modes of Mentalization Scale (MMS) in two different studies. Three junior raters and two senior raters assessed blindly 15 session transcripts of psychotherapy of five patients, using both the MIS and the MMS. The same 15 sessions were rated after the junior raters completed a training at the use of the scales and after on month from the end of the training to assess testretest reliability. Four therapists used the MIS and the MMS to provide different ratings of 22 patients undergoing a psychotherapy in different settings. Intraclass Correlation Coefficient (ICC) values ranged from sufficient to good and increased after the training. Test re-test reliability was sufficient for both scales (Study 1). ICC values ranged from sufficient to good, and were globally higher than the ones found in the first study sample (Study 2). Our results provide support to the inter-rater reliability of the MIS and the MMS.

Proposing a new short screening test for upper limb apraxia

2021 ◽  
pp. 030802262199856
Author(s):  
Mai Yamada ◽  
Masahiko Koyanagi ◽  
Miyo Kawaguchi ◽  
Yuki Sato ◽  
Mitsuhiro Tsujihata ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Upper Limb ◽  
Screening Test ◽  
Intraclass Correlation ◽  
Stroke Patients ◽  
Special Equipment ◽  
Rater Reliability ◽  
Verbal Instructions ◽  
Limb Apraxia

Background Apraxia has a major impact on activities of daily living in stroke patients. The proper assessment and treatment of apraxia is important for maintaining a good quality of life. We developed a short evaluation test for upper limb apraxia. Patients and Methods The present Screening Test of Gestures for Stroke consists of 10 items for each verbal instruction and imitation. Each item includes three meaningless gestures, three meaningful gestures and four pantomimes. The Screening Test of Gestures for Stroke is scored based on a 3-point system: 10, 5 or 0 (maximum score: 200). The test took approximately 2–5 min to complete. We recruited 65 patients admitted to our hospital with left hemisphere stroke and 50 healthy subjects. Results The reliability of the Screening Test of Gestures for Stroke was as follows: the intraclass correlation coefficient of intra-rater reliability was 0.93 for both verbal instructions and imitations, and the intraclass correlation coefficient total scores for inter-rater reliability for verbal instructions and for imitations were 0.97 and 0.95, respectively. The alpha coefficient was ≥0.80. Conclusions The Screening Test of Gestures for Stroke is a reliable and valid bedside test that has a short assessment time, does not require special equipment and can evaluate upper limb apraxia in stroke patients from the acute to the chronic phase.

Validity and reliability of a new ankle dorsiflexion measurement device

2012 ◽  
Vol 37 (4) ◽  
pp. 289-297 ◽  
Author(s):  
Alfred Gatt ◽  
Nachiappan Chockalingam
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Ankle Dorsiflexion ◽  
Reliability Testing ◽  
Validity And Reliability ◽  
Measurement Device ◽  
Rater Reliability ◽  
Measuring Device ◽  
New Device

Background: The assessment of the maximum ankle dorsiflexion angle is an important clinical examination procedure. Evidence shows that the traditional goniometer is highly unreliable, and various designs of goniometers to measure the maximum ankle dorsiflexion angle rely on the application of a known force to obtain reliable results. Hence, an innovative ankle dorsiflexion measurement device was designed to make this measurement more reliable by holding the foot in a selected posture without the application of a known moment. Objectives: To report on the comprehensive validity and reliability testing carried out on the new device. Methods: Following validity testing, four different trials to test reliability of the ankle dorsiflexion measurement device were performed. These trials included inter-rater and intra-rater testings with a controlled moment, intra-rater reliability testing with knees flexed and extended without a controlled moment, intra-rater testing with a patient population, and inter-rater reliability testing between four raters of varying experience without controlling moment. All raters were blinded. Study Design: A series of trials to test intra-rater and inter-rater reliabilities. Results: Intra-rater reliability intraclass correlation coefficient was 0.98 and inter-rater reliability intraclass correlation coefficient (2,1) was 0.953 with a controlled moment. With uncontrolled moment, very high reliability for intra-tester was also achieved (intraclass correlation coefficient = 0.94 with knees extended and intraclass correlation coefficient = 0.95 with knees flexed). For the trial investigating test–retest reliability with actual patients, intraclass correlation coefficient of 0.99 was obtained. In the trial investigating four different raters with uncontrolled moment, intraclass correlation coefficient of 0.91 was achieved. Conclusions: The new ankle dorsiflexion measurement device is a valid and reliable device for measuring ankle dorsiflexion in both healthy subjects and patients, with both controlled and uncontrolled moments, even by multiple raters of varying experience when the foot is dorsiflexed to its end of range of motion. Clinical relevance An ankle dorsiflexion measuring device has been designed to increase the reliability of ankle dorsiflexion measurement and replace the traditional goniometer. While the majority of similar devices rely on application of a known moment to perform this measurement, it has been shown that this is not required with the new ankle dorsiflexion measurement device and, rather, foot posture should be taken into consideration as this affects the maximum ankle dorsiflexion angle.

Reliability of the Clinical Outcome Variables Scale for children with cerebral palsy

2021 ◽  
Vol 28 (9) ◽  
pp. 1-8
Author(s):  
Sharon Merin Varghese ◽  
Thangavelu Senthilvelkumar ◽  
Noble Koshy ◽  
Gokilam Devaraj ◽  
Grace Rebekah ◽  
...  
Keyword(s):  
Cerebral Palsy ◽  
Clinical Outcome ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Functional Mobility ◽  
Outcome Variable ◽  
Minimally Clinically Important Difference ◽  
Rater Reliability ◽  
Children With Cerebral Palsy

Background/aims It can be difficult for rehabilitation professionals to use lengthy scales and different outcome measures for diverse clinical conditions in busy outpatient settings. The Clinical Outcome Variables Scale is a functional mobility measure that is applied to various neurological conditions. Determining the inter- and intra-rater reliability of clinical outcome variable scale for children with cerebral palsy will further enhance its utility. Methods A total of 30 children aged between 3 and 16 years with cerebral palsy, who could obey single-step commands, were recruited for the study. Two independent assessors scored the children using the Clinical Outcome Variable Scale to determine inter-rater reliability. A repeat assessment was done by the principal assessor after 24 hours to establish intra-rater reliability. Reliability was estimated using intra-class correlation coefficient values. Results The Clinical Outcome Variables Scale had high Inter- and intra-rater reliability for the composite score (intraclass correlation coefficient=1), the general mobility subscale (intraclass correlation coefficient=0.99), and the ambulation subscale (intraclass correlation coefficient=0.99). The intraclass correlation coefficient for the individual test items were also showed a high correlation, with the variance between the tests and physiotherapists ranging from 0.95 to 1. Conclusions The Clinical Outcome Variables Scale demonstrated high inter- and intra-rater reliability when assessing functional mobility in children with cerebral palsy. Further studies should establish criterion validity and minimally clinically important difference values to maximise the use of the scale.

Sign in / Sign up

Export Citation Format

Share Document