Test-retest reliability of the human functional connectome over consecutive days: identifying highly reliable portions and assessing the impact of methodological choices

Countless studies have advanced our understanding of the human brain and its organization by using functional magnetic resonance imaging (fMRI) to derive network representations of human brain function. However, we do not know to what extent these “functional connectomes” are reliable over time. In a large public sample of healthy participants ( N = 833) scanned on two consecutive days, we assessed the test-retest reliability of fMRI functional connectivity and the consequences on reliability of three common sources of variation in analysis workflows: atlas choice, global signal regression, and thresholding. By adopting the intraclass correlation coefficient as a metric, we demonstrate that only a small portion of the functional connectome is characterized by good (6–8%) to excellent (0.08–0.14%) reliability. Connectivity between prefrontal, parietal, and temporal areas is especially reliable, but also average connectivity within known networks has good reliability. In general, while unreliable edges are weak, reliable edges are not necessarily strong. Methodologically, reliability of edges varies between atlases, global signal regression decreases reliability for networks and most edges (but increases it for some), and thresholding based on connection strength reduces reliability. Focusing on the reliable portion of the connectome could help quantify brain trait-like features and investigate individual differences using functional neuroimaging.

Download Full-text

A new visual feedback-based system for the assessment of pinch force, endurance, accuracy and precision. A test-retest reliability study

Hand Therapy ◽

10.1177/17589983211002550 ◽

2021 ◽

pp. 175899832110025

Author(s):

Alberto Dottor ◽

Eleonora Camerone ◽

Mirko Job ◽

Diletta Barbiani ◽

Elisa Frisaldi ◽

...

Keyword(s):

Visual Feedback ◽

Intraclass Correlation ◽

Precision Grip ◽

Voluntary Contraction ◽

Good Reliability ◽

Retest Reliability ◽

Relative Reliability ◽

Time Accuracy ◽

In The Beginning ◽

Test Retest Reliability

Introduction Given that pinch is a precision grip involved in sustained submaximal activities, a Sustained Contraction (SC) task could be associated to Maximal Voluntary Contraction (MVC). To better evaluate the thumb-index system, the test-retest reliability of pinch MVC and SC, measured by a visual feedback-based pinch gauge was assessed. Methods 26 healthy participants performed MVC and SC in two separate sessions. SC required to maintain 40%MVC as long as possible and it was evaluated in terms of time, accuracy (Mean Distance between force trace and target force, MD), precision (Coefficient of Variability of force trace, CV). MD and CV analyses were conducted dividing the SC task into three equivalent time stages (beginning, middle, exhaustion). Relative Reliability (RR) was measured by Intraclass Correlation Coefficient, and Absolute Reliability (AR) was measured by Standard Error of Measurement and by Bland-Altman plot. Results MVC and Time showed high RR and AR in both hands. RR of MD and CV in right hand was excellent in the beginning and middle stages, and fair in the exhaustion one, showing decreasing reliability as fatigue increases. In the left hand RR of MD and CV was generally lower. MD showed excellent reliability in the beginning stage and good reliability in the other stages. CV showed fair relative reliability at both beginning and middle stages, excellent in the last one. Conversely, it was observed high AR of MD and CV in all stages in both hands. Conclusions All indices are reliable to assess motor control of thumb-index pinch in both hands.

Download Full-text

Reliability of the Polar Vantage M Sports Watch when Measuring Heart Rate at Different Treadmill Exercise Intensities

Sports ◽

10.3390/sports8090117 ◽

2020 ◽

Vol 8 (9) ◽

pp. 117

Author(s):

Mike Climstein ◽

Jessica L. Alder ◽

Alyce M. Brooker ◽

Elissa J. Cartwright ◽

Kevin Kemp-Smith ◽

...

Keyword(s):

Heart Rate ◽

Treadmill Exercise ◽

Intraclass Correlation ◽

Good Test ◽

Good Reliability ◽

Measurement Variability ◽

Retest Reliability ◽

Activity Monitors ◽

Test Retest Reliability ◽

Standard Error Of Measurement

Background: Usage of wrist-worn activity monitors has rapidly increased in recent years, and these devices are being used by both fitness enthusiasts and in clinical populations. We, therefore, assessed the test–retest reliability of the Polar Vantage M (PVM) watch when measuring heart rate (HR) during various treadmill exercise intensities. Methods: HR was measured every 30 s (simultaneous electrocardiography (ECG) and PVM). Test–retest reliability was determined using an intraclass correlation coefficient (ICC) with 95% confidence intervals (CIs). Standard error of measurement (SEM) and smallest real difference (SRD) were used to determine measurement variability. Results: A total of 29 participants completed the trials. ICC values for PVM during stages 1, 2 and 5 demonstrated good to excellent test–retest reliability (0.78, 0.78 and 0.92; 95% CI (0.54–0.90, 0.54–0.9, 0.79–0.97)). For PVM during stages 0 (rest), 3 and 4, the ICC values indicated poor to good reliability (0.42, 0.68 and 0.58; 95% CI (−0.27–0.73, 0.32–0.85, 0.14–0.80)). Conclusion: This study identified that the test–retest reliability of the PVM was comparable at low and high exercise intensities; however, it revealed a poor to good test–retest reliability at moderate intensities. The PVM should not be used in a clinical setting where monitoring of an accurate HR is crucial to the patients’ safety.

Download Full-text

Test-Retest and Interrater Reliability of the Functional Movement Screen

Journal of Athletic Training ◽

10.4085/1062-6050-48.2.11 ◽

2013 ◽

Vol 48 (3) ◽

pp. 331-336 ◽

Cited By ~ 51

Author(s):

Rebecca Shultz ◽

Scott C. Anderson ◽

Gordon O. Matheson ◽

Brandon Marcello ◽

Thor Besier

Keyword(s):

Interrater Reliability ◽

Intraclass Correlation ◽

Video Recording ◽

Functional Movement Screen ◽

Good Test ◽

Good Reliability ◽

Functional Movement ◽

Retest Reliability ◽

Test Retest Reliability ◽

First Session

Context: The Functional Movement Screen (FMS) is a popular test to evaluate the degree of painful, dysfunctional, and asymmetric movement patterns. Despite great interest in the FMS, test-retest reliability data have not been published. Objective: To assess the test-retest and interrater reliability of the FMS and to compare the scoring by 1 rater during a live session and the same session on video. Design: Cross-sectional study. Setting: Human performance laboratory in the sports medicine center. Patients or Other Participants: A total of 21 female (age = 19.6 ± 1.5 years, height = 1.7 ± 0.1 m, mass = 64.4 ± 5.1 kg) and 18 male (age = 19.7 ± 1.0 years, height = 1.9 ± 0.1 m, mass = 80.1 ± 9.9 kg) National Collegiate Athletic Association Division IA varsity athletes volunteered. Intervention(s): Each athlete was tested and retested 1 week later by the same rater who also scored the athlete's first session from a video recording. Five other raters scored the video from the first session. Main Outcome Measure(s): The Krippendorff α (K α) was used to assess the interrater reliability, whereas intraclass correlation coefficients (ICCs) were used to assess the test-retest reliability and reliability of live-versus-video scoring. Results: Good reliability was found for the test-retest (ICC = 0.6), and excellent reliability was found for the live-versus-video sessions (ICC = 0.92). Poor reliability was found for the interrater reliability (K α = .38). Conclusions: The good test-retest and high live-versus-video session reliability show that the FMS is a usable tool within 1 rater. However, the low interrater K α values suggest that the FMS within the limits of generalization should not be used indiscriminately to detect deficiencies that place the athlete at greater risk for injury. The FMS interrater reliability may be improved with better training for the rater.

Download Full-text

Measuring the Impact of Advertisement on Children Diet Choice in the Pediatric Population.

10.21203/rs.3.rs-30196/v1 ◽

2020 ◽

Author(s):

José Antonio Ponce-Blandón ◽

Macarena Romero-Martín ◽

Juan Gómez-Salgado ◽

Nerea Jiménez-Picón ◽

Lucía Ramírez-Baena ◽

...

Keyword(s):

Nutritional Value ◽

Pediatric Population ◽

Television Advertising ◽

Selection Method ◽

Food Advertising ◽

Good Reliability ◽

Retest Reliability ◽

Interclass Correlation ◽

Test Retest Reliability ◽

The Impact

Abstract Background: The aim of this study was to evaluate the test–retest reliability of the paired selection method for measuring the impact of food advertising on the consumption preferences of children. Methods: In this study, 52 children aged 4 to 6 years participated in watching an advertisement style video showing four foodstuffs with low nutritional value. Two pollsters were introduced at two instances showing pairs of pictures of the advertised foodstuffs right next to similar products, so that children could express their preferences. The test–retest reliability was measured through the concordance and interclass co-relation between both answers. Results: Kappa concordance indices of 0.71 (0.46–0.97), 0.88 (0.62–1.15), and 0.90 (0.64–1.16) were obtained for the three foodstuff pairs. The interclass correlation coefficient for the sum of manifested preferred scores was 0.72 (p = 0.0000). Conclusions: The paired selection method has been demonstrated to be efficient with good or very good reliability, making it useful for measuring the impact of food advertising on this age group. Keywords: food advertising; food habits; childhood obesity; television advertising; assessment.

Download Full-text

Test–Retest Reliability of a New Device Versus a Long-Arm Goniometer to Evaluate Knee Proprioception

Journal of Sport Rehabilitation ◽

10.1123/jsr.2021-0146 ◽

2021 ◽

pp. 1-6

Author(s):

Fei Tian ◽

Yaqi Zhao ◽

Jixin Li ◽

Wenjin Wang ◽

Danni Wu ◽

...

Keyword(s):

Standard Error ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Joint Position Sense ◽

Repeated Measurements ◽

Good Reliability ◽

Retest Reliability ◽

New Device ◽

Intraclass Correlation Coefficients ◽

Test Retest Reliability

Context: Many methods used to evaluate knee proprioception have shortcomings that limit their use in clinical settings. Based on an inexpensive 3D camera, a new portable device was recently used to evaluate the joint position sense (JPS) of the knee joint. However, the test–retest reliability of the new method remains unclear. This study aimed to evaluate the test–retest reliability of the new device and a long-arm goniometer for assessing knee JPS, and to compare the variability of the 2 methods. Design: Prospective observational study of the test–retest reliability of knee JPS measurements. Methods: Twenty-one healthy adults were tested in 2 sessions with a 1-week interval. Three target knee flexion angles (30°, 45°, and 60°) were reproduced in each session. Target and reproduced angles were measured with both methods. Intraclass correlation coefficients, standard error of the measurement, and Bland–Altman plots were used to quantify test–retest reliability. Paired t tests were used to compare knee JPS (absolute error of the target-reproduced angle) between the methods. Results: The new device (good to excellent intraclass correlation coefficients .74–.80; standard error of the measurement 0.52°–0.61°) demonstrated better test–retest reliability than the goniometer (poor to fair intraclass correlation coefficients .23–.43; standard error of the measurement 0.89°–2.07°) and better test–retest agreement (respective mean differences for the 30°, 45°, and 60° knee angles: 0.11°, 0.13°, and 0.41° for the new system; 0.84°, 1.52°, and 1.18° for the goniometer). The measurements (absolute errors of the target-reproduced angles) with the goniometer were significantly greater than those with the new device (P < .05); the SDs of repeated measurements with the goniometer (1.50°–2.41°) were greater than with the new device (1.08°–1.38°). Conclusions: Given that the new device has good reliability and sufficient precision, it is the better alternative for evaluating knee JPS. Goniometers should be used with caution to assess knee JPS.

Download Full-text

The Mandarin Voice Handicap Index for Laryngectomees With Pneumatic Artificial Laryngeal and Esophageal Speech

American Journal of Speech-Language Pathology ◽

10.1044/2021_ajslp-20-00241 ◽

2021 ◽

pp. 1-12

Author(s):

Chen-Chi Wang ◽

Jia-Shiou Liao ◽

Hsiu-Chin Lai ◽

Yi-Hsuan Lo

Keyword(s):

Internal Consistency ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Voice Handicap Index ◽

Good Test ◽

Retest Reliability ◽

Related Quality ◽

Valid Instrument ◽

Test Retest Reliability ◽

The Impact

Purpose The Voice Handicap Index (VHI) questionnaire assesses the impact of total laryngectomy on voice-related quality of life. This study evaluates the Mandarin VHI, including its internal consistency, test–retest reliability, content validity, and differences in scores for Mandarin alaryngeal patients with pneumatic artificial laryngeal (PA) and esophageal (ES) speech. Method Translation and validation of the VHI questionnaire was performed through the forward–backward translation technique. This study used a sample of 78 PA and 23 ES participants from Taiwan who completed the Mandarin VHI. Forty-two of the alaryngeal participants completed the Mandarin VHI twice over a period of 7–63 days. Results The measurement of the internal consistency of the Mandarin VHI showed a high Cronbach's alpha coefficient for the total score (.975) and the functional (.930), physical (.939), and emotional (.938) subscales. Based on the results of the intraclass correlation coefficients, good test–retest reliability for the total and domain scores was found (intraclass correlation coefficient = .827–.863). Conclusion The Mandarin VHI was validated as an instrument with proper internal consistency and reliability, which supports the Mandarin VHI as a valid instrument for the self-evaluation of handicaps related to voice problems in PA and ES speakers.

Download Full-text

Translation, adaptation and validation of the Bulgarian version of the King’s Parkinson’s Disease Pain Scale

BMC Neurology ◽

10.1186/s12883-021-02392-5 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Galina Stoyanova-Piroth ◽

Ivan Milanov ◽

Katerina Stambolieva

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Rating Scale ◽

Intraclass Correlation ◽

Pain Scale ◽

Radicular Pain ◽

Good Reliability ◽

Retest Reliability ◽

Nocturnal Pain ◽

Test Retest Reliability

Abstract Background The purpose of the present study was to translate and cross-culturally adapt the King’s Parkinson’s Disease Pain Scale (KPPS) into Bulgarian and to investigate its psychometric properties in order to provide a validated Parkinson’s disease-specific pain instrument in Bulgarian language (KPPS-BG). Methods Translation into Bulgarian and a cultural adaptation were performed to obtain KPPS-BG. A total of 162 patients with idiopathic Parkinson’s disease were screened for pain using the complementary to the KPPS questionnaire – King’s Parkinson’s Disease Pain Questionnaire (KPPQ). KPPS-BG domain and total scores were calculated and internal consistency, construct validity and test-retest reliability were examined for 129 patients having one or more positive items in the KPPQ-BG. Results 79.6 % of the patients reported one or more types of pain. The most common type was musculoskeletal pain (83.7 %), followed by nocturnal pain (55.0 %), fluctuation-related pain (50.1 %), radicular pain (43.4 %), chronic pain (31.0 %), discoloration, edema/swelling (27.1 %) and, oro-facial pain (14.3 %). Mean KPPS-BG total score was 21.1 ± 17.3 SD. KPPS-BG showed a good reliability (Cronbach’s alpha 0.75). The test-retest reliability of the KPPS-BG was high and the intraclass correlation coefficient was 0.92, demonstrating а good repeatability. KPPS-BG total score was higher in patients with postural instability gait difficulty motor subtype, compared to tremor-dominant or indeterminate subtype. Significant positive correlations were found between KPPS-BG total score and modified H&Y, Movement Disorders Society Unified Parkinson’s Disease Rating Scale part III. Conclusions The KPPS-BG constitutes a reliable, comprehensive and useful tool for pain assessment in native Bulgarian patients with Parkinson’s disease.

Download Full-text

Test–Retest Reliability of Soccer Dribbling Tests in Children

Journal of Motor Learning and Development ◽

10.1123/jmld.2021-0003 ◽

2021 ◽

pp. 1-7

Author(s):

Michael J. Duncan ◽

Darren Richardson ◽

Rhys Morris ◽

Emma Eyre ◽

Neil D. Clarke

Keyword(s):

Coefficient Of Variation ◽

Pediatric Population ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Systematic Bias ◽

Limits Of Agreement ◽

Good Reliability ◽

Retest Reliability ◽

Intraclass Correlation Coefficients ◽

Test Retest Reliability

The present study examined the test–retest reliability of the Ghent University dribbling test and short dribble test in a pediatric population. Fifty-four boys aged 9–14 years (mean ± SD = 11 ± 2 years) undertook the Ghent University and dribbling tests on two occasions separated by 2 weeks. Intraclass correlation coefficients, coefficient of variation, and 95% limits of agreement were conducted for each test. Intraclass correlation coefficients and coefficient of variation indicated good to excellent reliability, and relatively small variability for both dribbling tests. The 95% limits of agreement indicated relatively little bias Ghent University dribbling test for both running alone and running with the ball components, and the Bangsbo and Mohr short dribble test. The results of the present study suggest both dribbling tests examined in the present study demonstrate good reliability and low levels of systematic bias.

Download Full-text

Short-term test-retest reliability of the human intrinsic functional connectome

10.1101/755900 ◽

2019 ◽

Author(s):

Leonardo Tozzi ◽

Scott L. Fleming ◽

Zachary D. Taylor ◽

Cooper D. Raterink ◽

Leanne M. Williams

Keyword(s):

Functional Connectivity ◽

Intraclass Correlation ◽

Great Promise ◽

Short Term ◽

Retest Reliability ◽

Functional Connectome ◽

Short Term Test ◽

Clear Guideline ◽

Test Retest Reliability ◽

Over Time

ABSTRACTFunctional connectivity is frequently used to quantify the complex synchronous distributed fluctuations in neuronal activity derived from functional Magnetic Resonance Imaging and to generate network representations of human brain function. Such “functional connectomes” have great promise for mechanistic studies and for clinical translation. However, we do not know to what extent a functional connectome is stable over time for an individual. In the present work, we evaluate the short-term test-retest reliability of functional connectomes in a large publicly available sample of healthy participants (N=833) scanned on two consecutive days. We also assess the consequences on reliability of three methodological procedures for which a clear guideline in the community is lacking: atlas choice, global signal regression and thresholding. By adopting the intraclass correlation coefficient as a reliability metric, we demonstrate that a relatively small portion of the intrinsic functional connectome is characterized by good (4-6%) to excellent (0.08-1%) stability over time. In particular, connectivity between prefrontal, parietal and temporal areas appears to be especially stable over short timescales. Also, while unreliable edges of the functional connectome are generally weak in terms of average functional connectivity, reliable edges are not necessarily strong. Methodologically, we demonstrate that multimodal parcellation and averaging of connections within known networks are practices that improve reliability. Harnessing this knowledge, for example by honing in on the reliable portion of the connectome, offers one way forward for studies of trait-like features within the normative connectome and for discovery of biomarkers in clinical cohorts.

Download Full-text

Evaluation of Prolonged Walking in Persons with Multiple Sclerosis: Reliability of the Spatio-Temporal Walking Variables during the 6-Minute Walk Test

Sensors ◽

10.3390/s21093075 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3075

Author(s):

Nawale Hadouiri ◽

Elisabeth Monnet ◽

Arnaud Gouelle ◽

Pierre Decavel ◽

Yoshimasa Sagawa

Keyword(s):

Multiple Sclerosis ◽

Perceived Exertion ◽

Intraclass Correlation ◽

Weighted Kappa ◽

Walk Test ◽

Good Reliability ◽

Retest Reliability ◽

Spatio Temporal ◽

Test Retest Reliability ◽

Healthy Persons

Background: Walking disorders represent the most disabling condition in persons with Multiple Sclerosis (PwMS). Several studies showed good reliability of the 6-min walk test (6MWT) (i.e., especially distance traveled), but little is known about the reliability of the Spatio-temporal (ST) variables in the 6MWT. Objective: To evaluate the test-retest reliability of ST variables and perceived exertion during the 6MWT in PwMS and comparable healthy persons. Methods: We explored three 1-min intervals (initial: 0′–1′, middle: 2′30″–3′30″, end: 5′–6′) of the 6MWT. Six ST variables and perceived exertion were measured (respectively, using the GAITRite system and the Borg Scale). These measurements were performed twice, 1 week apart. The test-retest effects were assessed using the intraclass correlation coefficient (ICC) or the weighted kappa. Results: Forty-five PwMS and 24 healthy persons were included. The test-retest reliability of ST variables values was good-to-excellent for PwMS (ICC range: 0.858–0.919) and moderate-to-excellent for healthy persons (ICC range: 0.569–0.946). The test-retest reliability values of perceived exertion were fair for PwMS (weighted kappa range: 0.279–0.376) and substantial for healthy persons (weighted kappa range: 0.734–0.788). Conclusion: The measurement of ST variables during these 6MWT intervals is reliable and applicable in clinical practice and research to adapt rehabilitation care in PwMS.

Download Full-text