scholarly journals EXCELLENT RELIABILITY OF SEMI-QUANTITATIVE NAILFOLD CAPILLAROSCOPY IN PATIENTS WITH SYSTEMIC SCLEROSIS – A PILOT STUDY

2016 ◽  
Vol 25 (4) ◽  
pp. 194-198
Author(s):  
Ana Maria Gheorghiu ◽  
◽  
Raida Oneata ◽  
Mihai Bojinca ◽  
Rucsandra Dobrota ◽  
...  

Background. Semi-quantitative nailfold capillaroscopy (NFC) scoring represents a promising tool for assessing disease activity, severity and change in systemic sclerosis (SSc), however there is no consensus yet over which capillaroscopy abnormalities should be analyzed and how. Objective. Investigation of the reliability of the qualitative and semi-quantitative scoring of NFC assessment between two raters and test-retest for each rater in a SSc cohort. Methods. This is a single-center pilot study where 2 raters assessed the NFC images of 48 consecutive patients with SSc. Data were analyzed in 3 ways: 1. qualitatively by “normal”/“abnormal” category; 2. qualitatively by the following categories: “early”, “active”, “late” SSc patterns, “normal”, and unclassifiable in any pattern, and step; 3. Semi-quantitatively by calculating the mean score for capillary loss, disorganization of the microvascular array, giant capillaries, microhaemorrhages and capillary ramifications and combinations of giant capillaries and microhaemorrhages (as a surrogate for vascular activity). Disorganization and ramifications (surrogate for vascular damage) were also assessed. Variables for all steps were calculated for all fingers and for each finger. Inter-rater/intra-rater agreement was assessed by Cohen’s kappa coefficients for qualitative variables and by intraclass correlation coefficients (ICC) for mean score values of abnormalities. Results. Inter-rater reliability ranged from good to excellent agreement for mean score values of abnormalities in all fingers (ICC coefficients 0.745 to 0.897) and was excellent for activity (ICC coefficient of 0.923) and damage combinations (ICC coefficient of 0.918). Assessment of abnormalities in a qualitative manner (normal/abnormal or with capillaroscopy patterns) showed weaker inter-rater agreement than the semi-quantitative assessment (k coefficient <0.7). Intra-rater variability was good to excellent for mean score values of abnormalities and activity and damage combinations in all fingers and separate fingers for both raters; for qualitative assessment, only one of the raters had good test-retest reliability. Conclusion. Reliability of NFC assessment is essential in SSc trials/clinical practice to ensure quality of data. This pilot study demonstrates very good reliability between raters of the semi-quantitative NFC assessment in a SSc cohort. Combinations of NFC abnormalities had very good reliability and might be preferred because they are less time consuming.

2010 ◽  
Vol 69 (6) ◽  
pp. 1092-1096 ◽  
Author(s):  
Vanessa Smith ◽  
Carmen Pizzorni ◽  
Filip De Keyser ◽  
Saskia Decuman ◽  
Jens T Van Praet ◽  
...  

ObjectiveInvestigation of the reliability of the qualitative and semiquantitative scoring of nailfold videocapillaroscopy (NVC) assessment between two raters in a systemic sclerosis (SSc) cohort.MethodsTwo raters from different centres blindly assessed the NVC images of 71 consecutive patients with SSc qualitatively as belonging to the scleroderma spectrum (SDS) category (‘early’, ‘active’, ‘late’ scleroderma pattern or ‘scleroderma-like’ pattern) or to the ‘normal’ category and semiquantitatively by calculating the mean score for capillary loss, giant capillaries, microhaemorrhages and capillary ramifications. Inter-rater/intrarater agreement was assessed by calculation of the proportion of agreement and by κ coefficients. Rater agreement of mean score values of hallmark parameters was assessed by intraclass correlation coefficients.ResultsThe inter-rater/intrarater proportion of agreement to qualitatively assess an image as belonging to the SDS category or not was 90% and 96%, whereas the agreement to distinguish between only ‘early’, ‘active’ and ‘late’ scleroderma NVC patterns was 62% and 81%. The agreement of the semiquantitative scoring, as assessed by intraclass correlation coefficient, was 0.96 and 0.95 for capillary loss, 0.84 and 0.95 for giant capillaries, 0.90 and 0.95 for microhaemorrhages and 0.64 and 0.95 for capillary ramifications.ConclusionsThis is the first study to demonstrate reliability of the qualitative and semiquantitative NVC assessment in an SSc cohort between raters at different centres. Reliability of NVC assessment is essential for use of this tool in multicentre SSc trials.


2020 ◽  
pp. jrheum.191391 ◽  
Author(s):  
Stephanie Finzel ◽  
Sarah L. Manske ◽  
Cheryl Barnabe ◽  
Andrew J. Burghardt ◽  
Hubert Marotte ◽  
...  

Objective The aim of this multi-reader exercise was to assess the reliability and change over time of erosion measurements in rheumatoid arthritis (RA) patients using high-resolution peripheral quantitative computed tomography (HR-pQCT). Methods HR-pQCT scans of 23 patients with RA were assessed at baseline and 12 months. Four experienced readers examined the dorsal, palmar, radial, and ulnar surfaces of the metacarpal head (MH) and phalangeal base (PB) of the 2nd and 3rd digits, blinded to time order. In total, 368 surfaces (23 patients x16 surfaces) were evaluated per time point to characterize cortical breaks as pathological (erosion) or physiological, and to quantify erosion width and depth. Reliability was evaluated by intraclass correlation coefficients (ICC), percentage agreement, and Light’s kappa; change over time was defined by means ± SD of erosion numbers and dimensions. Results ICCs for the mean measurements of width and depth of the pathological breaks ranged between 0.819 - 0.883, and 0.771 - 0.907 respectively. Most physiological cortical breaks were found at the palmar PB, whereas most pathological cortical breaks were located at the radial MH. There was a significant increase in both the numbers and the dimensions of erosions between baseline and follow-up (p=0.0001 for erosion numbers, width, and depth in axial plane, and p=0.001 for depth in perpendicular plane). Conclusion This exercise confirmed good reliability of HR-pQCT erosion measurements and their ability to detect change over time.


2019 ◽  
Vol 90 (12) ◽  
pp. 1000-1008
Author(s):  
Caleb D. Johnson ◽  
Alice D. LaGoy ◽  
Gert-Jan Pepping ◽  
Shawn R. Eagle ◽  
Anne Z. Beethe ◽  
...  

INTRODUCTION: Designed as a more ecological measure of reaction times, the Perception-Action Coupling Task (PACT) has shown good reliability and within-subject stability. However, a lengthy testing period was required. Perceptual-motor judgments are known to be affected by proximity of the stimulus to the action boundary. The current study sought to determine the effects of action boundary proximity on PACT performance, and whether redundant levels of stimuli, eliciting similar responses, can be eliminated to shorten the PACT.METHODS: There were 9 men and 7 women who completed 4 testing sessions, consisting of 3 familiarization cycles and 6 testing cycles of the PACT. For the PACT, subjects made judgments on whether a series of balls presented on a tablet afford “posting” (can fit) through a series of apertures. There were 8 ratios of ball to aperture size (B-AR) presented, ranging from 0.2 to 1.8, with each ratio appearing 12 times (12 trials) per cycle. Reaction times and judgment accuracy were calculated, and averaged across all B-ARs. Ratios and individual trials within each B-AR were systematically eliminated. Variables were re-averaged, and intraclass correlation coefficients (ICC) and coefficients of variation (CVTE) were calculated in an iterative manner.RESULTS: With elimination of the 0.2 and 1.8 B-ARs, the PACT showed good reliability (ICC = 0.81–0.99) and consistent within-subject stability (CVTE = 2.2–14.7%). Reliability (ICC = 0.81–0.97) and stability (CVTE = 2.6–15.6%) were unaffected with elimination of up to 8 trials from each B-AR.DISCUSSION: The shortened PACT resulted in an almost 50% reduction in total familiarization/testing time required, significantly increasing usability.Johnson CD, LaGoy AD, Pepping G-J, Eagle SR, Beethe AZ, Bower JL, Alfano CA, Simpson RJ, Connaboy C. Action boundary proximity effects on perceptual-motor judgments. Aerosp Med Hum Perform. 2019; 90(12):1000–1008.


2019 ◽  
Vol 91 (1) ◽  
pp. 75-81 ◽  
Author(s):  
Leonhard A Bakker ◽  
Carin D Schröder ◽  
Harold H G Tan ◽  
Simone M A G Vugts ◽  
Ruben P A van Eijk ◽  
...  

ObjectiveThe Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) is widely applied to assess disease severity and progression in patients with motor neuron disease (MND). The objective of the study is to assess the inter-rater and intra-rater reproducibility, i.e., the inter-rater and intra-rater reliability and agreement, of a self-administration version of the ALSFRS-R for use in apps, online platforms, clinical care and trials.MethodsThe self-administration version of the ALSFRS-R was developed based on both patient and expert feedback. To assess the inter-rater reproducibility, 59 patients with MND filled out the ALSFRS-R online and were subsequently assessed on the ALSFRS-R by three raters. To assess the intra-rater reproducibility, patients were invited on two occasions to complete the ALSFRS-R online. Reliability was assessed with intraclass correlation coefficients, agreement was assessed with Bland-Altman plots and paired samples t-tests, and internal consistency was examined with Cronbach’s coefficient alpha.ResultsThe self-administration version of the ALSFRS-R demonstrated excellent inter-rater and intra-rater reliability. The assessment of inter-rater agreement demonstrated small systematic differences between patients and raters and acceptable limits of agreement. The assessment of intra-rater agreement demonstrated no systematic changes between time points; limits of agreement were 4.3 points for the total score and ranged from 1.6 to 2.4 points for the domain scores. Coefficient alpha values were acceptable.DiscussionThe self-administration version of the ALSFRS-R demonstrates high reproducibility and can be used in apps and online portals for both individual comparisons, facilitating the management of clinical care and group comparisons in clinical trials.


Sports ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. 117
Author(s):  
Mike Climstein ◽  
Jessica L. Alder ◽  
Alyce M. Brooker ◽  
Elissa J. Cartwright ◽  
Kevin Kemp-Smith ◽  
...  

Background: Usage of wrist-worn activity monitors has rapidly increased in recent years, and these devices are being used by both fitness enthusiasts and in clinical populations. We, therefore, assessed the test–retest reliability of the Polar Vantage M (PVM) watch when measuring heart rate (HR) during various treadmill exercise intensities. Methods: HR was measured every 30 s (simultaneous electrocardiography (ECG) and PVM). Test–retest reliability was determined using an intraclass correlation coefficient (ICC) with 95% confidence intervals (CIs). Standard error of measurement (SEM) and smallest real difference (SRD) were used to determine measurement variability. Results: A total of 29 participants completed the trials. ICC values for PVM during stages 1, 2 and 5 demonstrated good to excellent test–retest reliability (0.78, 0.78 and 0.92; 95% CI (0.54–0.90, 0.54–0.9, 0.79–0.97)). For PVM during stages 0 (rest), 3 and 4, the ICC values indicated poor to good reliability (0.42, 0.68 and 0.58; 95% CI (−0.27–0.73, 0.32–0.85, 0.14–0.80)). Conclusion: This study identified that the test–retest reliability of the PVM was comparable at low and high exercise intensities; however, it revealed a poor to good test–retest reliability at moderate intensities. The PVM should not be used in a clinical setting where monitoring of an accurate HR is crucial to the patients’ safety.


2006 ◽  
Vol 10 (4) ◽  
pp. 160-165 ◽  
Author(s):  
Jerry K.L. Tan ◽  
Karen Fung ◽  
Lynne Bulger

Background: There is a paucity of data on the reliability of dermatologists in acne lesion counting and global severity assessments. The effects of training and practice on reliability are also uncertain. The objective of this study was to determine the reliability of these outcome measurements when performed by trained dermatologists. Methods: Eleven dermatologists were divided into two groups that evaluated the same six acne subjects twice on the same day. A training session was provided either after (group A) or before (group B) the first patient evaluation sessions. Reliability of raters in lesion counting and global severity assessment was determined by calculation of intraclass correlation coefficients (ICCs). ICC values close to 1.0 indicate excellent reliability, whereas values less than 0.75 are considered unacceptable. Results: Intrarater ICCs ranged from 0.37 to 0.99 for noninflammatory lesions, 0.26 to 0.97 for inflammatory lesions, and 0.56 to 0.83 for global assessments for group A (trained after); corresponding values for group B (trained before) were 0.84 to 0.98, 0.61 to 0.95, and 0.43 to 0.91. ICC values ≥ 0.75 for all three outcome parameters were observed in one of six group A and three of five group B raters. Interrater ICCs for groups A and B after the first evaluation session were 0.17 versus 0.68 for noninflammatory counts, 0.84 versus 0.72 for inflammatory counts, and 0.71 versus 0.65 for global assessments, respectively. Corresponding values after session 2 were 0.79 and 0.77 for noninflammatory, 0.81 and 0.90 for inflammatory, and 0.61 and 0.77 for global assessments. Conclusion: Dermatologists tended to be reliable in acne lesion counting but somewhat less so in global assessments. Training tended to improve group reliability in noninflammatory lesion counts and increased the proportion of raters with good reliability in all three outcome measures. Practice enhanced reliability in all outcome measurements.


2013 ◽  
Vol 48 (3) ◽  
pp. 331-336 ◽  
Author(s):  
Rebecca Shultz ◽  
Scott C. Anderson ◽  
Gordon O. Matheson ◽  
Brandon Marcello ◽  
Thor Besier

Context: The Functional Movement Screen (FMS) is a popular test to evaluate the degree of painful, dysfunctional, and asymmetric movement patterns. Despite great interest in the FMS, test-retest reliability data have not been published. Objective: To assess the test-retest and interrater reliability of the FMS and to compare the scoring by 1 rater during a live session and the same session on video. Design: Cross-sectional study. Setting: Human performance laboratory in the sports medicine center. Patients or Other Participants: A total of 21 female (age = 19.6 ± 1.5 years, height = 1.7 ± 0.1 m, mass = 64.4 ± 5.1 kg) and 18 male (age = 19.7 ± 1.0 years, height = 1.9 ± 0.1 m, mass = 80.1 ± 9.9 kg) National Collegiate Athletic Association Division IA varsity athletes volunteered. Intervention(s): Each athlete was tested and retested 1 week later by the same rater who also scored the athlete's first session from a video recording. Five other raters scored the video from the first session. Main Outcome Measure(s): The Krippendorff α (K α) was used to assess the interrater reliability, whereas intraclass correlation coefficients (ICCs) were used to assess the test-retest reliability and reliability of live-versus-video scoring. Results: Good reliability was found for the test-retest (ICC = 0.6), and excellent reliability was found for the live-versus-video sessions (ICC = 0.92). Poor reliability was found for the interrater reliability (K α = .38). Conclusions: The good test-retest and high live-versus-video session reliability show that the FMS is a usable tool within 1 rater. However, the low interrater K α values suggest that the FMS within the limits of generalization should not be used indiscriminately to detect deficiencies that place the athlete at greater risk for injury. The FMS interrater reliability may be improved with better training for the rater.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
En-Chi Chiu ◽  
Ya-Chen Lee ◽  
Shu-Chun Lee ◽  
I-Ping Hsueh

Abstract Background The Performance-based measure of Executive Functions (PEF) with four domains is designed to assess executive functions in people with schizophrenia. The purpose of this study was to examine the test-retest reliability of the PEF administered by the same rater (intra-rater agreement) and by different raters (inter-rater agreement) in people with schizophrenia and to estimate the values of minimal detectable change (MDC) and MDC%. Methods Two convenience samples (each sample, n = 60) with schizophrenia were conducted two assessments (two weeks apart). The intraclass correlation coefficient (ICC) was analyzed to examine intra-rater and inter-rater agreements of the test-retest reliability of the PEF. The MDC was calculated through standard error of measurement. Results For the intra-rater agreement study, the ICC values of the four domains were 0.88–0.92. The MDC (MDC%) of the four domains (volition, planning, purposive action, and perfromance effective) were 13.0 (13.0%), 12.2 (16.4%), 16.2 (16.2%), and 16.3 (18.8%), respectively. For the inter-rater agreement study, the ICC values of the four domains were 0.82–0.89. The MDC (MDC%) were 15.8 (15.8%), 17.4 (20.0%), 20.9 (20.9%), and 18.6 (18.6%) for the volition, planning, purposive action, and performance effective domains, respectively. Conclusions The PEF has good test-retest reliability, including intra-rater and inter-rater agreements, for people with schizophrenia. Clinicians and researchers can use the MDC values to verify whether an individual with schizophrenia shows any real change (improvement or deterioration) between repeated PEF assessments by the same or different raters.


2021 ◽  
pp. 1-6
Author(s):  
Fei Tian ◽  
Yaqi Zhao ◽  
Jixin Li ◽  
Wenjin Wang ◽  
Danni Wu ◽  
...  

Context: Many methods used to evaluate knee proprioception have shortcomings that limit their use in clinical settings. Based on an inexpensive 3D camera, a new portable device was recently used to evaluate the joint position sense (JPS) of the knee joint. However, the test–retest reliability of the new method remains unclear. This study aimed to evaluate the test–retest reliability of the new device and a long-arm goniometer for assessing knee JPS, and to compare the variability of the 2 methods. Design: Prospective observational study of the test–retest reliability of knee JPS measurements. Methods: Twenty-one healthy adults were tested in 2 sessions with a 1-week interval. Three target knee flexion angles (30°, 45°, and 60°) were reproduced in each session. Target and reproduced angles were measured with both methods. Intraclass correlation coefficients, standard error of the measurement, and Bland–Altman plots were used to quantify test–retest reliability. Paired t tests were used to compare knee JPS (absolute error of the target-reproduced angle) between the methods. Results: The new device (good to excellent intraclass correlation coefficients .74–.80; standard error of the measurement 0.52°–0.61°) demonstrated better test–retest reliability than the goniometer (poor to fair intraclass correlation coefficients .23–.43; standard error of the measurement 0.89°–2.07°) and better test–retest agreement (respective mean differences for the 30°, 45°, and 60° knee angles: 0.11°, 0.13°, and 0.41° for the new system; 0.84°, 1.52°, and 1.18° for the goniometer). The measurements (absolute errors of the target-reproduced angles) with the goniometer were significantly greater than those with the new device (P < .05); the SDs of repeated measurements with the goniometer (1.50°–2.41°) were greater than with the new device (1.08°–1.38°). Conclusions: Given that the new device has good reliability and sufficient precision, it is the better alternative for evaluating knee JPS. Goniometers should be used with caution to assess knee JPS.


2020 ◽  
pp. 1-13
Author(s):  
Louise Capling ◽  
Janelle A. Gifford ◽  
Kathryn L. Beck ◽  
Victoria M. Flood ◽  
Fiona Halar ◽  
...  

Abstract Diet quality indices are a practical, cost-effective method to evaluate dietary patterns, yet few have investigated diet quality in athletes. This study describes the relative validity and reliability of the recently developed Athlete Diet Index (ADI). Participants completed the electronic ADI on two occasions, 2 weeks apart, followed by a 4-d estimated food record (4-dFR). Relative validity was evaluated by directly comparing mean scores of the two administrations (mAdm) against scores derived from 4-dFR using Spearman’s rank correlation coefficient and Bland–Altman (B–A) plots. Construct validity was investigated by comparing mAdm scores and 4-dFR-derived nutrient intakes using Spearman’s coefficient and independent t test. Test–retest reliability was assessed using paired t test, intraclass correlation coefficients (ICC) and B–A plots. Sixty-eight elite athletes (18·8 (sd 4·2) years) from an Australian sporting institute completed the ADI on both occasions. Mean score was 84·1 (sd 15·2; range 42·5–114·0). The ADI had good reliability (ICC = 0·80, 95 % CI 0·69, 0·87; P < 0·001), and B–A plots (mean 1·9; level of agreement −17·8, 21·7) showed no indication of systematic bias (y = 4·57–0·03 × x) (95 % CI −0·2, 0·1; P = 0·70). Relative validity was evaluated in fifty athletes who completed all study phases. Comparison of mAdm scores with 4-dFR-derived scores was moderate (rs 0·69; P < 0·001) with no systematic bias between methods of measurement (y = 6·90–0·04 × x) (95 % CI −0·3, 0·2; P = 0·73). Higher scores were associated with higher absolute nutrient intake consistent with a healthy dietary pattern. The ADI is a reliable tool with moderate validity, demonstrating its potential for application to investigate the diet quality of athletes.


Sign in / Sign up

Export Citation Format

Share Document