Reliability of Markers for Breast Hypoplasia in the Early Postpartum Period

Background Breast hypoplasia is one reason for insufficient milk supply. Case reports use wide intra-mammary width and certain breast appearances as markers of breast hypoplasia. However, the reliability of these variables has not been determined. Research aims To test the (i) interrater and intrarater reliability of intra-mammary width measurement and interrater reliability of categorizing women’s breasts into breast types, and (ii) feasibility and acceptability of study procedures for the participants. Methods This was a prospective, longitudinal, non-experimental design with survey and observational components of a convenience sample of early postpartum women ( N = 31). Interrater and intrarater reliability were measured using intraclass correlation coefficient for agreement for intra-mammary width measurements. Interrater reliability was measured using weighted kappa for agreement for categorizing breast type. Feasibility and acceptability of study procedures were collected 1 month later. Results Excellent intrarater and interrater reliability for the intra-mammary width measurement (ICC = 0.99, 95% CI [0.99, 0.997] and ICC = 0.88, 95% CI [0.74, 0.94], respectively) and fair interrater reliability for breast type categorization (k = 0.35, 95% CI [−0.05, 0.75]) with high level of agreement between raters (97%) were achieved. Proportions of participants agreeing to breast photography and acceptability of study procedures were 68% (21/31) and 100% (28/28) respectively. Conclusion The excellent reliability for the intra-mammary width measurement means it provides a useful measure for future research. Since the “fair” reliability of categorizing breast type was due to lack of breast shape range in our sample, future research could assess the breasts of women with different medical profiles.

Download Full-text

Reliability Assessment of Scores From Video-Recorded TGMD-3 Performances

Journal of Motor Learning and Development ◽

10.1123/jmld.2016-0007 ◽

2017 ◽

Vol 5 (1) ◽

pp. 59-68 ◽

Cited By ~ 16

Author(s):

Pauli Olavi Rintala ◽

Arja Kaarina Sääkslahti ◽

Susanna Iivonen

Keyword(s):

Motor Development ◽

Interrater Reliability ◽

Intraclass Correlation ◽

Kappa Statistic ◽

Intrarater Reliability ◽

Gross Motor ◽

Gross Motor Development ◽

Percent Agreement ◽

Two Samples ◽

Ball Skills

This study examined the intrarater and interrater reliability of the Test of Gross Motor Development—3rd Edition (TGMD-3). Participants were 60 Finnish children aged between 3 and 9 years, divided into three separate samples of 20. Two samples of 20 were used to examine the intrarater reliability of two different assessors, and the third sample of 20 was used to establish interrater reliability. Children’s TGMD-3 performances were video-recorded and later assessed using an intraclass correlation coefficient, a kappa statistic, and a percent agreement calculation. The intrarater reliability of the locomotor subtest, ball skills subtest, and gross motor total score ranged from 0.69 to 0.77, and percent agreement ranged from 87 to 91%. The interrater reliability of the locomotor subtest, ball skills subtest, and gross motor total score ranged from 0.56 to 0.64. Percent agreement of 83% was observed for locomotor skills, ball skills, and total skills, respectively. Hop, horizontal jump, and two-hand strike assessments showed the most difference between the assessors. These results show acceptable reliability for the TGMD-3 to analyze children’s gross motor skills.

Download Full-text

Influence of Rater Training on Inter- and Intrarater Reliability When Using the Rat Grimace Scale

Journal of the American Association for Laboratory Animal Science ◽

10.30802/aalas-jaalas-18-000044 ◽

2019 ◽

Vol 58 (2) ◽

pp. 178-183 ◽

Cited By ~ 8

Author(s):

Emily Q Zhang ◽

Vivian SY Leung ◽

Daniel SJ Pang

Keyword(s):

Acute Pain ◽

Interrater Reliability ◽

Intraclass Correlation ◽

Training Group ◽

Intrarater Reliability ◽

Rater Training ◽

Trainee Group ◽

Pain Models ◽

Ongoing Pain ◽

And Performance

Rodent grimace scales facilitate assessment of ongoing pain. Reported rater training using these scales varies considerably and may contribute to the observed variability in interrater reliability. This study evaluated the effect of training on interrater reliability with the Rat Grimace Scale (RGS). Two training sets (42 and 150 images) were prepared from acute pain models. Four trainee raters progressed through 2 rounds of training, scoring 42 images (set 1) followed by 150 images (set 2a). After each round, trainees reviewed the RGS and any problematic images with an experienced rater. The 150 images were then rescored (set 2b). Four years later, trainees rescored the 150 images (set 2c). A second group of raters (no-training group) scored the same image sets without review with the experienced rater. Inter- and intrarater reliability were evaluated by using the intraclass correlation coefficient (ICC), and ICC values were compared by using the Feldt test. In the trainee group, interrater reliability increased from moderate to very good between sets 1 and 2b and increased between sets 2a and 2b. Action units with the highest and lowest ICC at set 2b were orbital tightening and whiskers, respectively. In comparison to an experienced rater, the ICC for all trainees improved, ranging from 0.88 to 0.91 at set 2b. Four years later, very good interrater reliability was retained, and intrarater reliability was good or very good). The interrater reliability of the no-training group was moderate and did not improve from set 1 to set 2b. Training improved interrater reliability, with an associated reduction in 95%CI. In addition, training improved interrater reliability with an experienced rater, and performance was retained.

Download Full-text

Could Residents Adequately Assess the Severity of Hidradenitis Suppurativa? Interrater and Intrarater Reliability Assessment of Major Scoring Systems

Dermatology ◽

10.1159/000501771 ◽

2019 ◽

Vol 236 (1) ◽

pp. 8-14 ◽

Cited By ~ 1

Author(s):

Katarzyna Włodarek ◽

Aleksandra Stefaniak ◽

Łukasz Matusiak ◽

Jacek C. Szepietowski

Keyword(s):

Interrater Reliability ◽

Hidradenitis Suppurativa ◽

Intraclass Correlation ◽

Scoring Systems ◽

Staging System ◽

Severity Index ◽

Assessment Tools ◽

Intrarater Reliability ◽

Global Assessment Scale ◽

Interrater Variability

A wide variety of assessment tools have been proposed for hidradenitis suppurativa (HS) until now, but none of them meets the criteria for an ideal score. Because there is no gold standard scoring system, the choice of the measure instrument depends on the purpose of use and even on the physician’s experience in the subject of HS. The aim of this study was to assess the intrarater and interrater reliability of 6 scoring systems commonly used for grading severity of HS: the Hurley Staging System, the Refined Hurley Staging, the Hidradenitis Suppurativa Severity Score System (IHS4), the Hidradenitis Suppurativa Severity Index (HSSI), the Sartorius Hidradenitis Suppurativa Score and the Hidradenitis Suppurativa Physician’s Global Assessment Scale (HS-PGA). On the scoring day, 9 HS patients underwent a physical examination and disease severity assessment by a group of 16 dermatology residents using all evaluated instruments. Then, intrarater reliability was calculated using intraclass correlation coefficient (ICC), and interrater variability was evaluated using the coefficient of variation (CV). In all 6 scorings the ICCs were >0.75, indicating high intrarater reliability of all presented scales. The study has also demonstrated moderate agreement between raters in most of the evaluated measure instruments. The most reproducible methods, according to CVs, seem to be the Hurley staging, IHS4, and HSSI. None of the 6 evaluated scoring systems showed a significant advantage over the other when comparing ICCs, and all the instruments seem to be very reliable methods. The interrater reliability was usually good, but the most repeatable results between researchers were obtained for the easiest scales, including Hurley scoring, IHS4 and HSSI.

Download Full-text

Intrarater and Interrater Reliability of Infrared Image Analysis of Forearm Acupoints before and after Moxibustion

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2020/6328756 ◽

2020 ◽

Vol 2020 ◽

pp. 1-8

Author(s):

Jiali Lou ◽

Yongliang Jiang ◽

Hantong Hu ◽

Xiaoyu Li ◽

Yajun Zhang ◽

...

Keyword(s):

Image Analysis ◽

Correlation Coefficient ◽

Temperature Change ◽

Intraclass Correlation Coefficient ◽

Interrater Reliability ◽

Intraclass Correlation ◽

Infrared Image ◽

Infrared Images ◽

Intrarater Reliability ◽

Before And After

The objective of this study was to determine the intrarater and interrater reliabilities of infrared image analysis of forearm acupoints before and after moxibustion. In this work, infrared images of acupoints in the forearm of 20 volunteers (M/F, 10/10) were collected prior to and after moxibustion by infrared thermography (IRT). Two trained raters performed the analysis of infrared images in two different periods at a one-week interval. The intraclass correlation coefficient (ICC) was calculated to determine the intrarater and interrater reliabilities. With regard to the intrarater reliability, ICC values were between 0.758 and 0.994 (substantial to excellent). For the interrater reliability, ICC values ranged from 0.707 to 0.964 (moderate to excellent). Given that the intrarater and interrater reliability levels show excellent concordance, IRT could be a reliable tool to monitor the temperature change of forearm acupoints induced by moxibustion.

Download Full-text

Assessment of the Intrarater and Interrater Reliability of an Established Clinical Task Analysis Methodology

Anesthesiology ◽

10.1097/00000542-200205000-00016 ◽

2002 ◽

Vol 96 (5) ◽

pp. 1129-1139 ◽

Cited By ~ 46

Author(s):

Jason Slagle ◽

Matthew B. Weinger ◽

My-Than T. Dinh ◽

Vanessa V. Brumer ◽

Kevin Williams

Keyword(s):

Real Time ◽

Task Analysis ◽

Interrater Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Intrarater Reliability ◽

Intraclass Correlation Coefficients ◽

Percent Time ◽

Analysis Methodology ◽

And Task

Background Task analysis may be useful for assessing how anesthesiologists alter their behavior in response to different clinical situations. In this study, the authors examined the intraobserver and interobserver reliability of an established task analysis methodology. Methods During 20 routine anesthetic procedures, a trained observer sat in the operating room and categorized in real-time the anesthetist's activities into 38 task categories. Two weeks later, the same observer performed task analysis from videotapes obtained intraoperatively. A different observer performed task analysis from the videotapes on two separate occasions. Data were analyzed for percent of time spent on each task category, average task duration, and number of task occurrences. Rater reliability and agreement were assessed using intraclass correlation coefficients. Results Intrarater reliability was generally good for categorization of percent time on task and task occurrence (mean intraclass correlation coefficients of 0.84-0.97). There was a comparably high concordance between real-time and video analyses. Interrater reliability was generally good for percent time and task occurrence measurements. However, the interrater reliability of the task duration metric was unsatisfactory, primarily because of the technique used to capture multitasking. Conclusions A task analysis technique used in anesthesia research for several decades showed good intrarater reliability. Off-line analysis of videotapes is a viable alternative to real-time data collection. Acceptable interrater reliability requires the use of strict task definitions, sophisticated software, and rigorous observer training. New techniques must be developed to more accurately capture multitasking. Substantial effort is required to conduct task analyses that will have sufficient reliability for purposes of research or clinical evaluation.

Download Full-text

Interrater Reliability of the Berg Balance Scale When Used by Clinicians of Various Experience Levels to Assess People With Lower Limb Amputations

Physical Therapy ◽

10.2522/ptj.20130182 ◽

2014 ◽

Vol 94 (3) ◽

pp. 371-378 ◽

Cited By ~ 21

Author(s):

Christopher K. Wong

Keyword(s):

Lower Limb ◽

Interrater Reliability ◽

Clinical Training ◽

Intraclass Correlation ◽

Berg Balance Scale ◽

Intrarater Reliability ◽

Rater Reliability ◽

Study Objective ◽

Balance Scale ◽

Scale Scores

Background People with lower limb amputations frequently have impaired balance ability. The Berg Balance Scale (BBS) has excellent psychometric properties for people with neurologic disorders and elderly people dwelling in the community. A Rasch analysis demonstrated the validity of the BBS for people with lower limb amputations of all ability strata, but rater reliability has not been tested. Objective The study objective was to determine the interrater reliability and intrarater reliability of BBS scores and the differences in scores assigned by testers with various levels of experience when assessing people with lower limb amputations. Design This reliability study of video-recorded single-session BBS assessments had a cross-sectional design. Methods From a larger study of people with lower limb amputations, 5 consecutively recruited participants using prostheses were video recorded during an in-person BBS assessment. Sixteen testers independently rated the video-recorded assessments. Testers were 3 physical therapists, 1 occupational therapist, 3 third-year and 4 second-year doctor of physical therapy (DPT) students, and 5 first-year DPT students without clinical training. Rater reliability was calculated using intraclass correlation coefficients (ICC [2,k]). Differences in scores assigned by testers with various levels of experience were determined by use of an analysis of variance with Tukey post hoc tests. Results The average age of the participants was 53.0 years (SD=15.7). Amputations had occurred at the ankle disarticulation, transtibial, and transfemoral levels because of vascular, trauma, and medical etiologies an average of 8.2 years earlier (SD=7.9). Berg Balance Scale scores spanned all ability strata. Interrater reliability (ICC [2,k]=.99) and intrarater reliability of scores determined in person and through video-recorded assessments by the same testers (ICC [2,k]=.99) were excellent. For participants with the lowest levels of ability, licensed professionals assigned lower scores than did DPT students without clinical training. Limitations Intrarater reliability calculations were based on 2 testers. Conclusions Berg Balance Scale scores assigned to people using prostheses by testers with various levels of clinical experience had excellent interrater reliability and intrarater reliability.

Download Full-text

Validity, Reliability, and Ability to Identify Fall Status of the Berg Balance Scale, BESTest, Mini-BESTest, and Brief-BESTest in Patients With COPD

Physical Therapy ◽

10.2522/ptj.20150391 ◽

2016 ◽

Vol 96 (11) ◽

pp. 1807-1815 ◽

Cited By ~ 35

Author(s):

Cristina Jácome ◽

Joana Cruz ◽

Ana Oliveira ◽

Alda Marques

Keyword(s):

Interrater Reliability ◽

Intraclass Correlation ◽

Berg Balance Scale ◽

Performance Validity ◽

Operating Characteristics ◽

Intrarater Reliability ◽

Balance Test ◽

Balance Scale ◽

Balance Tests ◽

Abc Scale

Abstract Background The Berg Balance Scale (BBS), Balance Evaluation Systems Test (BESTest), Mini-BESTest, and Brief-BESTest are useful in the assessment of balance. Their psychometric properties, however, have not been tested in patients with chronic obstructive pulmonary disease (COPD). Objective This study aimed to compare the validity, reliability, and ability to identify fall status of the BBS, BESTest, Mini-BESTest, and the Brief-BESTest in patients with COPD. Design A cross-sectional study was conducted. Methods Forty-six patients (24 men, 22 women; mean age=75.9 years, SD=7.1) were included. Participants were asked to report their falls during the previous 12 months and to fill in the Activity-specific Balance Confidence (ABC) Scale. The BBS and the BESTest were administered. Mini-BESTest and Brief-BESTest scores were computed based on the participants' BESTest performance. Validity was assessed by correlating balance tests with each other and with the ABC Scale. Interrater reliability (2 raters), intrarater reliability (48–72 hours), and minimal detectable changes (MDCs) were established. Receiver operating characteristics assessed the ability of each balance test to differentiate between participants with and without a history of falls. Results Balance test scores were significantly correlated with each other (Spearman correlation rho=.73–.90) and with the ABC Scale (rho=.53–.75). Balance tests presented high interrater reliability (intraclass correlation coefficient [ICC]=.85–.97) and intrarater reliability (ICC=.52–.88) and acceptable MDCs (MDC=3.3–6.3 points). Although all balance tests were able to identify fall status (area under the curve=0.74–0.84), the BBS (sensitivity=73%, specificity=77%) and the Brief-BESTest (sensitivity=81%, specificity=73%) had the higher ability to identify fall status. Limitations Findings are generalizable mainly to older patients with moderate COPD. Conclusions The 4 balance tests are valid, reliable, and valuable in identifying fall status in patients with COPD. The Brief-BESTest presented slightly higher interrater reliability and ability to differentiate participants' fall status.

Download Full-text

The Development and Reliability of a Simple Field-Based Screening Tool to Assess for Scapular Dyskinesis

Journal of Sport Rehabilitation ◽

10.1123/jsr.2015-0054 ◽

2016 ◽

Vol 25 (3) ◽

Author(s):

Siobhan O'Connor ◽

Noel McCaffrey ◽

Enda Whyte ◽

Kieran Moran

Keyword(s):

Screening Tool ◽

High Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Future Research ◽

Scapular Dyskinesis ◽

Convenience Sample ◽

Physically Active ◽

Overhead Athletes ◽

Lack Of Control

Context: Scapular dyskinesis has been identified as a possible risk factor for injury in overhead athletes. There is a need to develop a simple, inexpensive, time-efficient field-based test that can establish the presence and severity of both scapular dyskinesis and its individual components in a musculoskeletal preparticipation screening. To ensure confidence in the test results, high reliability must be displayed. Objective: To establish the intertester and intratester reliability of a simple field-based screening tool for scapular dyskinesis. Design: Reliability study. Setting: Athletic therapy facilities. Participants: 15 physically active men (19.46 ± 0.63 y) free from any orthopedic or neurological disorders, recruited from a convenience sample of college students. Intervention: Testers underwent 3 training sessions where the instructions and scoring system of the test were explained, demonstrations of the tests were given, and opportunities to practice the test were provided. Three testers independently rated 3 trials, and the process was repeated at the same time the following week. Main Outcome Measures: Scapular dyskinesis was assessed using the following components: winging, loss/lack of control when lifting, loss/lack of control when lowering, and scapular asymmetry. A 0-3 rating (0 = no issue, 1 = slight issue, 2 = moderate issue, 3 = severe issue) was provided by the tester independently on each side separately and on scapular asymmetry. The intraclass correlation coefficients (ICCs), 95% confidence intervals, and standard error of measurement (SEM) were computed to establish reliability. Results: Excellent ICC values were found for intertester reliability (ICCs .80-1.00) and good to excellent intratester reliability (ICCs .60-1.00 for tester A, .63-1.00 for tester B, and .75-1.00 for tester C) for all components of the test. The SEM was not found to be clinically significant. Conclusions: The simple field-based screening tool developed to assess scapular dyskinesis demonstrates high reliability and so is a reliable tool to use in preparticipation screenings. Future research should establish its validity.

Download Full-text

FEASIBILITY AND RELIABILITY OF MUSCULOSKELETAL ULTRASOUND MEASUREMENT OF THE MEDIAL PATELLOFEMORAL LIGAMENT

Orthopaedic Journal of Sports Medicine ◽

10.1177/2325967120s00185 ◽

2020 ◽

Vol 8 (4_suppl3) ◽

pp. 2325967120S0018

Author(s):

Andrea Stracciolini ◽

Laura Boucher ◽

Sarah Jackson ◽

Naomi Brown ◽

Danielle Magrini ◽

...

Keyword(s):

Interrater Reliability ◽

Medial Patellofemoral Ligament ◽

Intraclass Correlation ◽

Musculoskeletal Ultrasound ◽

Reliability Testing ◽

Intrarater Reliability ◽

Good Reliability ◽

Medicine Physician ◽

Mean Width ◽

Musculoskeletal Ultrasonography

Background The medial patellofemoral ligament (MPFL) is an important soft tissue constraint to preventing patellar dislocations in young athletes. The anatomy of the MPFL has been investigated in cadaveric studies and magnetic resonance studies. No studies to date have provided anatomical data of the MPFL on ultrasonography. Purpose To investigate the feasibility of musculoskeletal ultrasonography for the evaluation of the MPFL, and to determine interrater and intrarater reliability for MPFL ultrasound measures. Methods Ten control participants (20 knees) 20 to 50 years underwent ultrasonography performed by 3 researchers (musculoskeletal ultrasound radiologist, athletic trainer/biomechanist, primary care sports medicine physician) from 3 different institutions for interrater reliability testing. Intrarater reliability testing was performed at 2 separate institutions by 4 physicians, each performing the same knee ultrasound protocol on 20 knees in 10 study participants 2 to 3 weeks apart. In total, 180 images were created for interrater reliability, and 480 images for intrarater reliability. Examinations were performed with linear high-frequency transducers (10-18 MHz) with the participant in the supine position and the extremity flexed at 45°. Measurements included ligament length (long axis to ligament) from the patellar to the femoral attachment sites, ligament width (short axis to ligament) at the patellar attachment, and ligament thickness (long axis to ligament) midway between the patella and femur. Mean and SD were calculated for all measurements. Intraclass correlation coefficient (ICC) analysis was used to assess intrarater and interrater reliability. ICC values < 0.40 indicated poor reliability, whereas those between 0.40 and 0.75 indicated fair to good reliability, and those > 0.75 indicated excellent reliability. Results The mean US value for MPFL length was 44.83mm (SD 6.68), mean thickness 2.66mm (SD 0.85), and mean width 11.76mm (SD 2.99). The overall ICC values for interrater reliability testing indicated fair to good reliability for length measures (0.7) and poor reliability for thickness (–0.1) and width (0.3; Table 1.1). Overall ICC values for intrarater reliability indicated fair to good reliability for length (0.5), excellent for thickness (0.9), and poor reliability for width (–0.3; Table 1.2). Conclusions Musculoskeletal ultrasonography is a feasible and reliable office-based method of measuring MPFL length and thickness. These quantitative measures set the groundwork for establishing normative anatomical measures of the MPFL in athletes and establish a protocol for testing and measuring the MPFL using musculoskeletal ultrasonography. [Table: see text][Table: see text]

Download Full-text

Method of Quantifying Size of Retinal Hemorrhages in Eyes with Branch Retinal Vein Occlusion Using 14-Square Grid: Interrater and Intrarater Reliability

Journal of Ophthalmology ◽

10.1155/2016/1960190 ◽

2016 ◽

Vol 2016 ◽

pp. 1-8

Author(s):

Yuko Takashima ◽

Masahiko Sugimoto ◽

Kumiko Kato ◽

Maki Kozawa ◽

Kengo Ikesugi ◽

...

Keyword(s):

Retinal Vein Occlusion ◽

Branch Retinal Vein Occlusion ◽

Interrater Reliability ◽

Intraclass Correlation ◽

Relative Size ◽

Retinal Hemorrhage ◽

Intrarater Reliability ◽

Retinal Hemorrhages ◽

Vein Occlusion ◽

Power Point

Purpose. To describe a method of quantifying the size of the retinal hemorrhages in branch retinal vein occlusion (BRVO) and to determine the interrater and intrarater reliabilities of these measurements. Methods. Thirty-five fundus photographs from 35 consecutive eyes with BRVO were studied. The fundus images were analyzed with Power-Point® software, and a grid of 14 squares was laid over the fundus image. Raters were asked to judge the percentage of each of the 14 squares that was covered by the hemorrhages, and the average of the 14 squares was taken to be the relative size of the retinal hemorrhage. Results. Interrater reliability between three raters was higher when a grid with 14 squares was used (intraclass correlation coefficient (ICC), 0.96) than that when a box with no grid was used (ICC, 0.78). Intrarater reliability, which was calculated by the retinal hemorrhage area measured on two different days, was also higher (ICC, 0.97) than that with no grid (ICC, 0.86). Interrater reliability for five fundus pictures with poor image quality was also good when a grid with 14 squares was used (ICC, 0.88). Conclusions. Although our method is subjective, excellent interrater and intrarater reliabilities indicate that this method can be adapted for clinical use.

Download Full-text