scholarly journals Symbol Digit Modalities Test Variant in a Smartphone App for Persons With Multiple Sclerosis: Validation Study (Preprint)

2020 ◽  
Author(s):  
Pim van Oirschot ◽  
Marco Heerings ◽  
Karine Wendrich ◽  
Bram den Teuling ◽  
Marijn B Martens ◽  
...  

BACKGROUND The decline of cognitive processing speed (CPS) is a common dysfunction in persons with multiple sclerosis (MS). The Symbol Digit Modalities Test (SDMT) is widely used to formally quantify CPS. We implemented a variant of the SDMT in MS sherpa, a smartphone app for persons with MS. OBJECTIVE The aim of this study was to investigate the construct validity and test-retest reliability of the MS sherpa smartphone variant of the SDMT (sSDMT). METHODS We performed a validation study with 25 persons with relapsing-remitting MS and 79 healthy control (HC) subjects. In the HC group, 21 subjects were matched to the persons with MS with regard to age, gender, and education and they followed the same assessment schedule as the persons with MS (the “HC matched” group) and 58 subjects had a less intense assessment schedule to determine reference values (the “HC normative” group). Intraclass correlation coefficients (ICCs) were determined between the paper-and-pencil SDMT and its smartphone variant (sSDMT) on 2 occasions, 4 weeks apart. Other ICCs were determined for test-retest reliability, which were derived from 10 smartphone tests per study participant, with 3 days in between each test. Seven study participants with MS were interviewed regarding their experiences with the sSDMT. RESULTS The SDMT scores were on average 12.06% higher than the sSDMT scores, with a standard deviation of 10.68%. An ICC of 0.838 was found for the construct validity of the sSDMT in the combined analysis of persons with MS and HC subjects. Average ICCs for test-retest reliability of the sSDMT for persons with MS, the HC matched group, and the HC normative group were 0.874, 0.857, and 0.867, respectively. The practice effect was significant between the first and the second test of the persons with MS and the HC matched group and trivial for all other test-retests. The interviewed study participants expressed a positive attitude toward the sSDMT, but they also discussed the importance of adapting a smartphone cognition test in accordance with the needs of the individual persons with MS. CONCLUSIONS The high correlation between sSDMT and the conventional SDMT scores indicates a very good construct validity. Similarly, high correlations underpin a very good test-retest reliability of the sSDMT. We conclude that the sSDMT has the potential to be used as a tool to monitor CPS in persons with MS, both in clinical studies and in clinical practice.

10.2196/18160 ◽  
2020 ◽  
Vol 8 (10) ◽  
pp. e18160
Author(s):  
Pim van Oirschot ◽  
Marco Heerings ◽  
Karine Wendrich ◽  
Bram den Teuling ◽  
Marijn B Martens ◽  
...  

Background The decline of cognitive processing speed (CPS) is a common dysfunction in persons with multiple sclerosis (MS). The Symbol Digit Modalities Test (SDMT) is widely used to formally quantify CPS. We implemented a variant of the SDMT in MS sherpa, a smartphone app for persons with MS. Objective The aim of this study was to investigate the construct validity and test-retest reliability of the MS sherpa smartphone variant of the SDMT (sSDMT). Methods We performed a validation study with 25 persons with relapsing-remitting MS and 79 healthy control (HC) subjects. In the HC group, 21 subjects were matched to the persons with MS with regard to age, gender, and education and they followed the same assessment schedule as the persons with MS (the “HC matched” group) and 58 subjects had a less intense assessment schedule to determine reference values (the “HC normative” group). Intraclass correlation coefficients (ICCs) were determined between the paper-and-pencil SDMT and its smartphone variant (sSDMT) on 2 occasions, 4 weeks apart. Other ICCs were determined for test-retest reliability, which were derived from 10 smartphone tests per study participant, with 3 days in between each test. Seven study participants with MS were interviewed regarding their experiences with the sSDMT. Results The SDMT scores were on average 12.06% higher than the sSDMT scores, with a standard deviation of 10.68%. An ICC of 0.838 was found for the construct validity of the sSDMT in the combined analysis of persons with MS and HC subjects. Average ICCs for test-retest reliability of the sSDMT for persons with MS, the HC matched group, and the HC normative group were 0.874, 0.857, and 0.867, respectively. The practice effect was significant between the first and the second test of the persons with MS and the HC matched group and trivial for all other test-retests. The interviewed study participants expressed a positive attitude toward the sSDMT, but they also discussed the importance of adapting a smartphone cognition test in accordance with the needs of the individual persons with MS. Conclusions The high correlation between sSDMT and the conventional SDMT scores indicates a very good construct validity. Similarly, high correlations underpin a very good test-retest reliability of the sSDMT. We conclude that the sSDMT has the potential to be used as a tool to monitor CPS in persons with MS, both in clinical studies and in clinical practice.


2021 ◽  
Author(s):  
Pim van Oirschot ◽  
Marco Heerings ◽  
Karine Wendrich ◽  
Bram den Teuling ◽  
Frank Dorssers ◽  
...  

BACKGROUND Walking disturbances are a common dysfunction in persons with multiple sclerosis (MS). The 2-Minute Walking Test (2MWT) is widely used to quantify walking speed. We implemented a smartphone-based 2MWT (s2MWT) in MS sherpa, an app for persons with MS. When performing the s2MWT, users of the app are instructed to walk as fast as safely possible for 2 minutes in the open air, while the app records their movement and calculates the distance walked. OBJECTIVE The aim of this study is to investigate the concurrent validity and test-retest reliability of the MS sherpa s2MWT. METHODS We performed a validation study on 25 persons with relapsing-remitting MS and 79 healthy control (HC) participants. In the HC group, 21 participants were matched to the persons with MS based on age, gender, and education and these followed the same assessment schedule as the persons with MS (the <i>HC-matched</i> group), whereas 58 participants had a less intense assessment schedule to determine reference values (the <i>HC-normative</i> group). Intraclass correlation coefficients (ICCs) were determined between the distance measured by the s2MWT and the distance measured using distance markers on the pavement during these s2MWT assessments. ICCs were also determined for test-retest reliability and derived from 10 smartphone tests per study participant, with 3 days in between each test. We interviewed 7 study participants with MS regarding their experiences with the s2MWT. RESULTS In total, 755 s2MWTs were completed. The adherence rate for the persons with MS and the participants in the HC-matched group was 92.4% (425/460). The calculated distance walked on the s2MWT was, on average, 8.43 m or 5% (SD 18.9 m or 11%) higher than the distance measured using distance markers (n=43). An ICC of 0.817 was found for the concurrent validity of the s2MWT in the combined analysis of persons with MS and HC participants. Average ICCs of 9 test-retest reliability analyses of the s2MWT for persons with MS and the participants in the HC-matched group were 0.648 (SD 0.150) and 0.600 (SD 0.090), respectively, whereas the average ICC of 2 test-retest reliability analyses of the s2MWT for the participants in the HC-normative group was 0.700 (SD 0.029). The interviewed study participants found the s2MWT easy to perform, but they also expressed that the test results can be confronting and that a pressure to reach a certain distance can be experienced. CONCLUSIONS The high correlation between s2MWT distance and the conventional 2MWT distance indicates a good concurrent validity. Similarly, high correlations underpin a good test-retest reliability of the s2MWT. We conclude that the s2MWT can be used to measure the distance that the persons with MS walk in 2 minutes outdoors near their home, from which both clinical studies and clinical practice can benefit.


10.2196/29128 ◽  
2021 ◽  
Vol 5 (11) ◽  
pp. e29128
Author(s):  
Pim van Oirschot ◽  
Marco Heerings ◽  
Karine Wendrich ◽  
Bram den Teuling ◽  
Frank Dorssers ◽  
...  

Background Walking disturbances are a common dysfunction in persons with multiple sclerosis (MS). The 2-Minute Walking Test (2MWT) is widely used to quantify walking speed. We implemented a smartphone-based 2MWT (s2MWT) in MS sherpa, an app for persons with MS. When performing the s2MWT, users of the app are instructed to walk as fast as safely possible for 2 minutes in the open air, while the app records their movement and calculates the distance walked. Objective The aim of this study is to investigate the concurrent validity and test-retest reliability of the MS sherpa s2MWT. Methods We performed a validation study on 25 persons with relapsing-remitting MS and 79 healthy control (HC) participants. In the HC group, 21 participants were matched to the persons with MS based on age, gender, and education and these followed the same assessment schedule as the persons with MS (the HC-matched group), whereas 58 participants had a less intense assessment schedule to determine reference values (the HC-normative group). Intraclass correlation coefficients (ICCs) were determined between the distance measured by the s2MWT and the distance measured using distance markers on the pavement during these s2MWT assessments. ICCs were also determined for test-retest reliability and derived from 10 smartphone tests per study participant, with 3 days in between each test. We interviewed 7 study participants with MS regarding their experiences with the s2MWT. Results In total, 755 s2MWTs were completed. The adherence rate for the persons with MS and the participants in the HC-matched group was 92.4% (425/460). The calculated distance walked on the s2MWT was, on average, 8.43 m or 5% (SD 18.9 m or 11%) higher than the distance measured using distance markers (n=43). An ICC of 0.817 was found for the concurrent validity of the s2MWT in the combined analysis of persons with MS and HC participants. Average ICCs of 9 test-retest reliability analyses of the s2MWT for persons with MS and the participants in the HC-matched group were 0.648 (SD 0.150) and 0.600 (SD 0.090), respectively, whereas the average ICC of 2 test-retest reliability analyses of the s2MWT for the participants in the HC-normative group was 0.700 (SD 0.029). The interviewed study participants found the s2MWT easy to perform, but they also expressed that the test results can be confronting and that a pressure to reach a certain distance can be experienced. Conclusions The high correlation between s2MWT distance and the conventional 2MWT distance indicates a good concurrent validity. Similarly, high correlations underpin a good test-retest reliability of the s2MWT. We conclude that the s2MWT can be used to measure the distance that the persons with MS walk in 2 minutes outdoors near their home, from which both clinical studies and clinical practice can benefit.


2021 ◽  
Author(s):  
A Wallin ◽  
M Kierkegaard ◽  
E Franzén ◽  
S Johansson

Abstract Objective The mini-BESTest is a balance measure for assessment of the underlying physiological systems for balance control in adults. Evaluations of test–retest reliability of the mini-BESTest in larger samples of people with multiple sclerosis (MS) are lacking. The purpose of this study was to investigate test–retest reliability of the mini-BESTest total and section sum scores and individual items in people with mild to moderate overall MS disability. Methods This study used a test–retest design in a movement laboratory setting. Fifty-four people with mild to moderate overall MS disability according to the Expanded Disability Status scale (EDSS) were included, with 28 in the mild subgroup (EDSS 2.0–3.5) and 26 in the moderate subgroup (EDSS 4.0–5.5). Test–retest reliability of the mini-BESTest was evaluated by repeated measurements taken 1 week apart. Reliability and measurement error were analyzed. Results Test–retest reliability for the total scores were considered good to excellent, with intraclass correlation coefficients of .88 for the whole sample, .83 for the mild MS subgroup, and .80 for the moderate MS subgroup. Measurement errors were small, with standard error of measurement and minimal detectable change of 1.3 and 3.5, respectively, in mild MS, and 1.7 and 4.7, respectively, in moderate MS. The limits of agreement were − 3.4 and 4.6. Test–retest reliability for the section scores were fair to good or excellent; weighted kappa values ranged from .62 to .83. All items but 1 showed fair to good or excellent test–retest reliability, and percentage agreement ranged from 61% to 100%. Conclusions The mini-BESTest demonstrated good to excellent test–retest reliability and small measurement errors and is recommended for use in people with mild to moderate MS. Impact Knowledge of limits of agreement and minimal detectable change contribute to interpretability of the mini-BESTest total score. The findings of this study enhance the clinical usefulness of the test for evaluation of balance control and for designing individually customized balance training with high precision and accuracy in people with MS.


Author(s):  
Katherine L. Marks ◽  
Alessandra Verdi ◽  
Laura E. Toles ◽  
Kaila L. Stipancic ◽  
Andrew J. Ortiz ◽  
...  

Objective The purpose of this study was to examine the psychometric properties of an ecological vocal effort scale linked to a voicing task. Method Thirty-eight patients with nodules, 18 patients with muscle tension dysphonia, and 45 vocally healthy control individuals participated in a week of ambulatory voice monitoring. A global vocal status question was asked hourly throughout the day. Participants produced a vowel–consonant–vowel syllable string and rated the vocal effort needed to produce the task on a visual analog scale. Test–retest reliability was calculated for a subset using the intraclass correlation coefficient, ICC(A, 1). Construct validity was assessed by (a) comparing the weeklong vocal effort ratings between the patient and control groups and (b) comparing weeklong vocal effort ratings before and after voice rehabilitation in a subset of 25 patients. Cohen's d, the standard error of measurement ( SEM ), and the minimal detectable change (MDC) assessed sensitivity. The minimal clinically important difference (MCID) assessed responsiveness. Results Test–retest reliability was excellent, ICC(A, 1) = .96. Weeklong mean effort was statistically higher in the patients than in controls ( d = 1.62) and lower after voice rehabilitation ( d = 1.75), supporting construct validity and sensitivity. SEM was 4.14, MDC was 11.47, and MCID was 9.74. Since the MCID was within the error of the measure, we must rely upon the MDC to detect real changes in ecological vocal effort. Conclusion The ecological vocal effort scale offers a reliable, valid, and sensitive method of monitoring vocal effort changes during the daily life of individuals with and without vocal hyperfunction.


2020 ◽  
pp. 153944922096107
Author(s):  
Ecem Karanfil ◽  
Yeliz Salcı ◽  
Ayla Fil-Balkan ◽  
Can Ebru Bekircan-Kurt ◽  
Sevim Erdem Özdamar ◽  
...  

Linguistic, reliable, and valid secondary efficacy measures are important in clinical settings and studies. The aim of the study is to report test–retest reliability and construct validity of Turkish version of Myasthenia Gravis-Activities of Daily Living Scale (MG-ADL-T) in Myasthenia Gravis (MG) patients. Fifty-two ocular and generalized individuals with MG, applying to rehabilitation center, were included in the study. MG-ADL-T, MG quality-of-life questionnaire (MG-QoL), MG composite (MGC), quantitative MG score (QMGS), and pulmonary function test were administered. Reliability was assessed with intraclass correlation coefficient (ICC) and Cronbach’s alpha. Spearman correlation test and receiver operating characteristic (ROC) analysis were performed for construct validity. MG-ADL-T had fair internal consistency (Cronbach’s α = .67), excellent test–retest reliability (ICC = 0.96) and moderate construct validity (MG-QoL, r = 0.59; QMGS, r = .58; MGC, r = .68). MG-ADL, a unique scale that evaluates activities of daily living (ADL), has good test–retest reliability and construct validity in Turkish MG patients.


2020 ◽  
Vol 9 (3) ◽  
Author(s):  
Anggi Setyowati ◽  
Min-Huey Chung ◽  
Ah. Yusuf ◽  
Setya Haksama

Background: Curiosity is a personality characteristic, which fits with wellbeing and positive functioning. The objective of this study was to assess the construct validity of the Curiosity and Exploration Inventory II (CEI-II) in Indonesia.Design and Methods: The study included 256 undergraduate students who lived in Indonesia, mean age 19.8 years old. The CEI-II measures stretching and embracing using 11 items. The English version of CEI-II was translated into Bahasa. The Cronbach’s alpha coefficient and intraclass correlation coefficient (ICC) were addressed to examine internal consistency reliability and the test-retest reliability. To evaluate construct validity, exploratory factor analysis (EFA) was used to assess factor structure and confirmatory factor analysis (CFA) was used to evaluate the structural model fit of the CEI-II Indonesia version.Results: The study showed Cronbach’s alpha for the internal consistency of the overall CEI-II Indonesia version was 0.77. The ICC for the test-retest reliability ranged between 0.753-0.829. EFA showed adequate with the Kaiser-Meyer-Olkin value of 0.86 and the Bartlett’s test of sphericity was statistically significant. CFA tested the second-order model with two-order factors and showed a model fit.Conclusions: The CEI-II Indonesia version indicated acceptable construct validity to evaluate curiosity in Indonesia.


2013 ◽  
Vol 10 (8) ◽  
pp. 1079-1090 ◽  
Author(s):  
Adewale L. Oyeyemi ◽  
James F. Sallis ◽  
Adetoyeje Y. Oyeyemi ◽  
Mariam M. Amin ◽  
Ilse De Bourdeaudhuij ◽  
...  

Background:This study adapted the Physical Activity Neighborhood Environment Scale (PANES) to the Nigerian context and assessed the test-retest reliability and construct validity of the Nigerian version (PANES-N).Methods:A multidisciplinary panel of experts adapted the original PANES to reflect the built and social environment of Nigeria. The adapted PANES was subjected to cognitive testing and test retest reliability in a diverse sample of Nigerian adults (N = 132) from different neighborhood types. Intraclass Correlation Coefficients (ICC) was used to assess test-retest reliability, and construct validity was investigated with Analysis of Covariance for differences in environmental attributes between neighborhoods.Results:Four of the 17 items on the original PANES were significantly modified, 3 were removed and 2 new items were incorporated into the final version of adapted PANES-N. Test-retest reliability was substantial to almost perfect (ICC = 0.62–1.00) for all items on the PANES-N, and residents of neighborhoods in the inner city reported higher residential density, land use mix and safety, but lower pedestrian facilities and aesthetics than did residents of government reserved area/new layout neighborhoods.Conclusion:The PANES-N appears promising for assessing environmental perceptions related to physical activity in Nigeria, but further testing is required to assess its applicability across Africa.


2021 ◽  
pp. 036354652110271
Author(s):  
Mark J. Scholes ◽  
Matthew G. King ◽  
Kay M. Crossley ◽  
Denise M. Jones ◽  
Adam I. Semciw ◽  
...  

Background: The International Hip Outcome Tool–33 (iHOT-33) was developed to evaluate patients seeking surgery for hip and/or groin (hip/groin) pain and may not be appropriate for those seeking nonsurgical treatment. Purpose: To evaluate the psychometric properties of the iHOT-33 total (iHOT-Total) score and all subscale scores in adults with hip/groin pain who were not seeking surgery. Study Design: Cohort study (diagnosis); Level of evidence, 3. Methods: Patients with hip/groin pain who were not seeking surgery were recruited from 2 ongoing studies in Australia. Semistructured one-on-one interviews assessed content validity. Construct validity was assessed by testing hypothesized correlations between iHOT-33 and Copenhagen Hip and Groin Outcome Score (HAGOS) subscale scores. Test-retest reliability was assessed in patients not undertaking treatment and who reported “no change” in their Global Rating of Change (GROC) score at 6-month follow-up. Scores were reliable at group and individual levels if intraclass correlation coefficients (ICCs) were ≥0.80 and ≥0.90, respectively. Scores were responsive if Spearman rank correlations (ρ) between the change in the iHOT-33 score and the GROC score were ≥0.40. Results: In total, 278 patients with hip/groin pain (93 women; mean age, 31 years) and 55 pain-free control participants (14 women; mean age, 29 years) were recruited. The iHOT-33 demonstrated acceptable content validity. Construct validity was acceptable, with all hypothesized strong positive correlations between iHOT-33 and HAGOS subscale scores confirmed ( r range, 0.60-0.76; P < .001), except for one correlation between the iHOT-Sport and HAGOS-Sport ( r = .058; P < .001). All scores were reliable at the group level, except for the iHOT-33 job subscale (iHOT-Job) (ICC range, 0.78-0.88 [95% CI, 0.60-0.93]). None of the subscales met the criteria for adequate reliability for use at the individual level (all ICCs <0.90). Minimal detectable change values (group level) ranged from 2.3 to 3.7 (95% CI, 1.7-5.0). All iHOT-33 subscale scores were responsive (ρ range, 0.40-0.58; P≤ .001), except for the iHOT-Job in patients not undertaking treatment (ρ = 0.27; P = .001). Conclusion: All iHOT-33 subscale scores were valid for use in patients with hip/groin pain who were not seeking surgery. Acceptable test-retest reliability was found for all subscale scores at the group level, except the iHOT-Job. All subscale scores, excluding the iHOT-Job, were responsive, regardless of undertaking physical therapist–led treatment or no treatment.


2006 ◽  
Vol 12 (5) ◽  
pp. 573-577 ◽  
Author(s):  
L N Brown ◽  
L M Metz ◽  
M Eliasziw

Background Tactile temporal thresholds are typically significantly higher (ie, prolonged) in multiple sclerosis (MS) patients when compared to controls and increase significantly during relapses, probably reflecting integrity of conduction across a portion of the corpus callosum. As part of an ongoing validation study of tactile temporal thresholds, the test-retest reliability of these thresholds was examined in patients with MS. Methods Tactile temporal thresholds were measured in 61 MS patients during two separate test sessions within three weeks. Test-retest reliability and the standard error of measurement were calculated. The threshold of change in tactile temporal thresholds in MS patients that would correspond to real change beyond measurement error with 95% certainty was also calculated. Results The test-retest reliability of this measure of tactile temporal thresholds was 0.93. The threshold indicating change beyond chance or measurement error with 95% certainty was 19 ms. Conclusions This measure of tactile temporal thresholds has excellent test - retest reliability and a change of greater than 19 ms is highly likely to represent real change. This measure is promising as a precise, reliable outcome measure in MS.


Sign in / Sign up

Export Citation Format

Share Document