Expressive language sampling as a source of outcome measures for treatment studies in fragile X syndrome: feasibility, practice effects, test-retest reliability, and construct validity

Abstract Background The evaluation of treatment efficacy for individuals with fragile X syndrome (FXS) or intellectual disability (ID) more generally has been hampered by the lack of adequate outcome measures. We evaluated expressive language sampling (ELS) as a procedure for generating outcome measures for treatment research in FXS. We addressed: (a) feasibility, (b) practice effects over two administrations, (c) test-retest reliability over the repeated administrations, and (d) construct validity. We addressed these issues for the full sample as well as for subgroups defined by age, IQ, and ASD status. Methods Participants were 106 individuals with FXS between ages 6 and 23 years who had IQs within the range of intellectual disability (IQ < 70). ELS procedures for collecting samples in conversation and narration were followed and analyzed separately. Five measures were derived from transcripts segmented into C-units (i.e., an independent clause and its modifiers): number of C-units per minute (talkativeness), number of different word roots (vocabulary), C-unit length in morphemes (syntax), percentage of C-units containing dysfluency (utterance planning), and percentage of C-units that were fully or partly unintelligible (articulatory quality). ELS procedures were administered twice at 4-week intervals for each participant. Standardized tests and informant reports were administered and provided measures for evaluating construct validity of ELS measures. Results We found low rates of noncompliance, suggesting the task can be completed meaningfully by most individuals with FXS, although noncompliance was higher for younger, lower IQ, and more autistic participants. Minimal practice effects and strong test-retest reliability over the 4-week interval were observed for the full sample and across the range of ages, IQs, and autism symptom severity. Evidence of convergent construct validity was observed for the measures of vocabulary, syntax, and unintelligibility for the full sample and across the range of IQ and autism symptom severity, but not for participants under age 12. Conversation and narration yielded largely similar results in all analyses. Conclusions The findings suggest that the ELS procedures are feasible and yield measures with adequate psychometric properties for a majority of 6 to 23 years with FXS who have ID. The procedures work equally well regardless of level of ID or degree of ASD severity. The procedures, however, are more challenging and have somewhat less adequate psychometric properties for individuals with FXS under age 12.

Download Full-text

Correction to: Expressive language sampling as a source of outcome measures for treatment studies in fragile X syndrome: feasibility, practice effects, test-retest reliability, and construct validity

Journal of Neurodevelopmental Disorders ◽

10.1186/s11689-020-09314-5 ◽

2020 ◽

Vol 12 (1) ◽

Cited By ~ 1

Author(s):

Leonard Abbeduto ◽

Elizabeth Berry-Kravis ◽

Audra Sterling ◽

Stephanie Sherman ◽

Jamie O. Edgin ◽

...

Keyword(s):

Construct Validity ◽

Fragile X Syndrome ◽

Outcome Measures ◽

Fragile X ◽

Expressive Language ◽

Practice Effects ◽

Original Publication ◽

Retest Reliability ◽

Language Sampling ◽

Test Retest Reliability

In the original publication of this article [1], the author name Leonard Abbeduto was misspelled as Leonardkk Abbeduto. The original article has been corrected.

Download Full-text

Spoken language outcome measures for treatment studies in Down syndrome: feasibility, practice effects, test-retest reliability, and construct validity of variables generated from expressive language sampling

Journal of Neurodevelopmental Disorders ◽

10.1186/s11689-021-09361-6 ◽

2021 ◽

Vol 13 (1) ◽

Author(s):

Angela John Thurman ◽

Jamie O. Edgin ◽

Stephanie L. Sherman ◽

Audra Sterling ◽

Andrea McDuffie ◽

...

Keyword(s):

Construct Validity ◽

Outcome Measures ◽

Expressive Language ◽

Spoken Language ◽

Practice Effects ◽

Short Term ◽

Retest Reliability ◽

Language Sampling ◽

Language Outcome ◽

Test Retest Reliability

Abstract Background The purpose of this study was to evaluate expressive language sampling (ELS) as a procedure for generating spoken language outcome measures for treatment research in Down syndrome (DS). We addressed (a) feasibility, (b) practice effects across two short-term administrations, (c) test-retest reliability across two short-term administrations, (d) convergent and discriminant construct validity, and (e) considered comparisons across the conversation and narration contexts. Method Participants were 107 individuals with DS between 6 and 23 years of age who presented with intellectual disability (IQ < 70). The utility of ELS procedures designed to collect samples of spoken language in conversation and narration were evaluated separately. Variables of talkativeness, vocabulary, syntax, utterance planning, and articulation quality, derived from transcripts segmented into C-units (i.e., an independent clause and its modifiers), were considered. A 4-week interval was used to assess practice effects and test-retest reliability. Standardized direct assessments and informant report measures were collected to evaluate construct validity of the ELS variables. Results Low rates of noncompliance were observed; youth who were under 12 years of age, had phrase-level speech or less, and had a 4-year-old developmental level or less were at particular risk for experiencing difficulty completing the ELS procedures. Minimal practice effects and strong test-retest reliability across the 4-week test-retest interval was observed. The vocabulary, syntax, and speech intelligibility variables demonstrated strong convergent and discriminant validity. Although significant correlations were found between the variables derived from both the conversation and narration contexts, some differences were noted. Conclusion The ELS procedures considered were feasible and yielded variables with adequate psychometric properties for most individuals with DS between 6 and 23 years old. That said, studies of outcome measures appropriate for individuals with DS with more limited spoken language skills are needed. Context differences were observed in ELS variables suggest that comprehensive evaluation of expressive language is likely best obtained when utilizing both contexts.

Download Full-text

Using Telehealth-Delivered Procedures to Collect a Parent-Implemented Expressive Language Sampling Narrative Task in Monolingual and Bilingual Families With Autism Spectrum Disorder: A Pilot Study

Frontiers in Rehabilitation Sciences ◽

10.3389/fresc.2021.716550 ◽

2021 ◽

Vol 2 ◽

Author(s):

Laura del Hoyo Soriano ◽

Lauren Bullard ◽

Cesar Hoyos Alvarez ◽

Angela John Thurman ◽

Leonard Abbeduto

Keyword(s):

Autism Spectrum Disorder ◽

Pilot Study ◽

Outcome Measures ◽

Expressive Language ◽

Autism Spectrum ◽

Spectrum Disorder ◽

Practice Effects ◽

Language Sampling ◽

English Speaking ◽

Test Retest Reliability

Language impairments are frequent, severe, and of prognostic value in autism spectrum disorder (ASD). Unfortunately, the evaluation of the efficacy of treatments targeting the language skills of those with ASD continues to be hindered by a lack of psychometrically sound outcome measures. Expressive Language Sampling (ELS) procedures offer a promising alternative to norm-referenced standardized tests for assessing expressive language in treatment studies. Until now, however, research on the validity and utility of ELS as outcome measures has been limited to administrations by a trained professional in a clinic setting and to use with English-speaking families. These limitations are a barrier for many families accessing the benefits of participation in treatment studies. The current study examines the feasibility of teaching native English-speaking parents (NESP) and native Spanish-speaking parents (NSSP) how to administer the ELS narrative task (ELS-N) to their sons and daughters with ASD (between ages 6 and 21) at home through telehealth-delivered procedures. The parent training was provided in the primary language of the participating parent (i.e., 11 NSSP and 11 NESP) and administered by the parent to the youth in the language that the parent reported to use to communicate with the youth at home (i.e., 9 Spanish and 13 English). Families were able to choose between using their own technology or be provided with the technology needed for participation. Of the 19 parents who completed the training, 16 learned to administer the ELS-N procedures. In addition, strong test-retest reliability and no practice effects over the 4-week interval were observed for ELS-N derived youth outcome measures (i.e., talkativeness, vocabulary, syntax, dysfluency, and intelligibility) for both NSSP and NESP. Results from this pilot study suggest that the home-based parent-implemented ELS-N procedures can be learned and administered at acceptable levels of fidelity by parents, with good test-retest reliability and limited practice effects observed in terms of outcome measures for youth with ASD. Implications for treatment studies and future directions are discussed.

Download Full-text

Considerations for Evaluating and Recommending Worker Productivity Outcome Measures: An Update from the OMERACT Worker Productivity Group

The Journal of Rheumatology ◽

10.3899/jrheum.181201 ◽

2019 ◽

Vol 46 (10) ◽

pp. 1401-1405 ◽

Cited By ~ 3

Author(s):

Suzanne M.M. Verstappen ◽

Diane Lacaille ◽

Annelies Boonen ◽

Reuben Escorpizo ◽

Catherine Hofstetter ◽

...

Keyword(s):

Construct Validity ◽

Psychometric Properties ◽

Outcome Measures ◽

Worker Productivity ◽

Single Item ◽

Retest Reliability ◽

Test Retest Reliability

Objective.The Outcome Measures in Rheumatology (OMERACT) Worker Productivity Group continues efforts to assess psychometric properties of measures of presenteeism.Methods.Psychometric properties of single-item and dual answer multiitem scales were assessed, as well as methods to evaluate thresholds of meaning.Results.Test-retest reliability and construct validity of single item global measures was moderate to good. The value of measuring both degree of difficulty and amount of time with difficulty in multiitems questionnaires was confirmed. Thresholds of meaning vary depending on methods and external anchors applied.Conclusion.We have advanced our understanding of the performance of presenteeism measures and have developed approaches to describing thresholds of meaning.

Download Full-text

Test-retest reliability and practice effects of a computerized neuropsychological battery: A solution-oriented approach.

Psychological Assessment ◽

10.1037/pas0000618 ◽

2018 ◽

Vol 30 (12) ◽

pp. 1652-1662 ◽

Cited By ~ 10

Author(s):

Sophie J. M. Rijnen ◽

Sophie D. van der Linden ◽

Wilco H. M. Emons ◽

Margriet M. Sitskoorn ◽

Karin Gehring

Keyword(s):

Practice Effects ◽

Neuropsychological Battery ◽

Retest Reliability ◽

Test Retest Reliability ◽

Oriented Approach

Download Full-text

TEST-RETEST RELIABILITY AND CONSTRUCT VALIDITY OF THE COGNITIVE STYLE INDEX FOR BUSINESS UNDERGRADUATES

Psychological Reports ◽

10.2466/pr0.82.2.595-600 ◽

1998 ◽

Vol 82 (2) ◽

pp. 595 ◽

Cited By ~ 5

Author(s):

H. JOSEPH MURPH

Keyword(s):

Construct Validity ◽

Cognitive Style ◽

Retest Reliability ◽

Business Undergraduates ◽

Test Retest Reliability

Download Full-text

Reliability, Validity and Responsiveness of Physical Activity Monitors in Patients with Inflammatory Myopathy

Rheumatology ◽

10.1093/rheumatology/keab236 ◽

2021 ◽

Author(s):

Bonny Rockette-Wagner ◽

Didem Saygin ◽

Siamak Moghadam-Kia ◽

Chester Oddis ◽

Océane Landon-Cardinal ◽

...

Keyword(s):

Physical Activity ◽

Construct Validity ◽

Disease Activity ◽

Retest Reliability ◽

Activity Monitors ◽

Muscle Testing ◽

Patient Reported ◽

Physical Activity Monitors ◽

Global Disease Activity ◽

Test Retest Reliability

Abstract Objective Idiopathic inflammatory myopathies (IIM) cause proximal muscle weakness, which affect activities of daily living. Wearable physical activity monitors (PAMs) objectively assess continuous activity with potential clinical usefulness in IIM assessment. We examined the psychometric characteristics for PAM outcomes in IIM. Methods Adult IIM patients were prospectively evaluated (baseline, 3 and 6-months) in an observational study. A waist-worn PAM (ActiGraph GT3X-BT) assessed average step counts/min, peak 1-min cadence, and vector magnitude/min. Validated myositis core set measures (CSM) including manual muscle testing (MMT), physician global disease activity (MD global), patient global disease activity (Pt global), extra-muscular disease activity (Ex-muscular global), HAQ-DI, muscle enzymes, and patient-reported physical function were evaluated. Test-retest reliability, construct validity, and responsiveness were determined for PAM measures and CSM using Pearson correlations and other appropriate analyses. Results 50 adult IIM patients enrolled [mean (SD) age, 53.6 (±14.6); 60% female, 94% Caucasian]. PAM measures showed strong test-retest reliability, moderate-to-strong correlations at baseline with MD global (r=-0.37- -0.48), Pt-global (r=-0.43- -0.61), HAQ-DI (r=-0.47- -0.59) and MMT (r = 0.37–0.52), and strong discriminant validity for categorical MMT and HAQ-DI. Longitudinal association with MD global (r=-0.38- -0.44), MMT (r = 0.50–0.57), HAQ-DI (r=-0.45- -0.55), and functional tests (r = 0.30–0.65) were moderate-to-strong. PAM measures were responsive to MMT improvement (≥10%) and moderate-to-major improvement on ACR/EULAR myositis response criteria. Peak 1-min cadence had the largest effect size and Standardized Response Means (SRMs). Conclusion PAM measures showed promising construct validity, reliability, and longitudinal responsiveness; especially peak 1-min cadence. PAMs provide valid outcome measures for future use in IIM clinical trials.

Download Full-text

Cross-cultural adaptation, reliability and validity of the Fremantle Knee Awareness Questionnaire in Italian subjects with painful knee osteoarthritis

Health and Quality of Life Outcomes ◽

10.1186/s12955-021-01754-4 ◽

2021 ◽

Vol 19 (1) ◽

Author(s):

Marco Monticone ◽

Cristiano Sconza ◽

Igor Portoghese ◽

Tomohiko Nishigami ◽

Benedict M. Wand ◽

...

Keyword(s):

Construct Validity ◽

Knee Osteoarthritis ◽

Pain Intensity ◽

Internal Consistency ◽

Structural Validity ◽

Retest Reliability ◽

Knee Oa ◽

Pain Catastrophising ◽

Test Retest Reliability ◽

Painful Knee

Abstract Background and aim Growing attention is being given to utilising physical function measures to better understand and manage knee osteoarthritis (OA). The Fremantle Knee Awareness Questionnaire (FreKAQ), a self-reported measure of body-perception specific to the knee, has never been validated in Italian patients. The aims of this study were to culturally adapt and validate the Italian version of the FreKAQ (FreKAQ-I), to allow for its use with Italian-speaking patients with painful knee OA. Methods The FreKAQ-I was developed by means of forward–backward translation, a final review by an expert committee and a test of the pre-final version to evaluate its comprehensibility. The psychometric testing included: internal structural validity by Rasch analysis; construct validity by assessing hypotheses of FreKAQ correlations with the knee injury and osteoarthritis outcome score (KOOS), a pain intensity numerical rating scale (PI-NRS), the pain catastrophising scale (PCS), and the Hospital anxiety and depression score (HADS) (Pearson’s correlations); known-group validity by evaluating the ability of FreKAQ scores to discriminate between two groups of participants with different clinical profiles (Mann–Whitney U test); reliability by internal consistency (Cronbach’s alpha) and test–retest reliability (intraclass correlation coefficient, ICC2.1); and measurement error by calculating the minimum detectable change (MDC). Results It took one month to develop a consensus-based version of the FreKAQ-I. The questionnaire was administered to 102 subjects with painful knee OA and was well accepted. Internal structural validity confirmed the substantial unidimensionality of the FreKAQ-I: variance explained was 53.3%, the unexplained variance in the first contrast showed an eigenvalue of 1.8, and no local dependence was detected. Construct validity was good as all of the hypotheses were met; correlations: KOOS (rho = 0.38–0.51), PI-NRS (rho = 0.35–0.37), PCS (rho = 0.47) and HADS (Anxiety rho = 0.36; Depression rho = 0.43). Regarding known-groups validity, FreKAQ scores were significantly different between groups of participants demonstrating high and low levels of pain intensity, pain catastrophising, anxiety, depression and the four KOOS subscales (p ≤ 0.004). Internal consistency was acceptable (α = 0.74) and test–retest reliability was excellent (ICC = 0.92, CI 0.87–0.94). The MDC95 was 5.22 scale points. Conclusion The FreKAQ-I is unidimensional, reliable and valid in Italian patients with painful knee OA. Its use is recommended for clinical and research purposes.

Download Full-text