scholarly journals Validation of a novel fully automated story recall task for repeated remote high-frequency administration

2021 ◽  
Author(s):  
Caroline Skirrow ◽  
Marton Meszaros ◽  
Udeepa Meepegama ◽  
Raphael Lenain ◽  
Kathryn V Papp ◽  
...  
Keyword(s):  
High Frequency ◽  
Convergent Validity ◽  
Correlation Coefficients ◽  
Recall Task ◽  
Smart Devices ◽  
Cognitively Impaired ◽  
Similarity Metrics ◽  
Story Recall ◽  
Global Cognitive Function ◽  
Parallel Forms

INTRODUCTION: Longitudinal data is key to identifying cognitive decline and treatment response in Alzheimer's disease (AD). METHODS: The Automatic Story Recall Task (ASRT) is a novel, fully automated test that can be self-administered remotely. In this longitudinal case-control observational study, 151 participants (mean age: 69.99 (range 54-82), 73 mild cognitive impairment/mild AD and 78 cognitively unimpaired) completed parallel ASRT assessments on their smart devices over 7-8 days. Responses were automatically transcribed and scored using text similarity metrics. RESULTS: Participants reported good task usability. Adherence to optional daily assessment was moderate. Parallel forms correlation coefficients between ASRTs were moderate-high. ASRTs correlated moderately with established tests of episodic memory and global cognitive function. Poorer performance was observed in participants with MCI/Mild AD. DISCUSSION: Unsupervised ASRT assessment is feasible in older and cognitively impaired people. This automated task shows good parallel forms reliability and convergent validity with established cognitive tests.

Reliability of the Minimum Masking Level as Outcome Variable in Tinnitus Clinical Research

2020 ◽  
Vol 29 (3) ◽  
pp. 429-435
Author(s):  
Patricia C. Mancini ◽  
Richard S. Tyler ◽  
Hyung Jin Jun ◽  
Tang-Chuan Wang ◽  
Helena Ji ◽  
...  
Keyword(s):  
Standard Deviation ◽  
High Frequency ◽  
Pearson Correlation ◽  
Correlation Coefficients ◽  
Outcome Variable ◽  
Reliable Estimate ◽  
Moderate Correlation ◽  
Trained Subjects ◽  
The Relationship ◽  
Minimum Intensity

Purpose The minimum masking level (MML) is the minimum intensity of a stimulus required to just totally mask the tinnitus. Treatments aimed at reducing the tinnitus itself should attempt to measure the magnitude of the tinnitus. The objective of this study was to evaluate the reliability of the MML. Method Sample consisted of 59 tinnitus patients who reported stable tinnitus. We obtained MML measures on two visits, separated by about 2–3 weeks. We used two noise types: speech-shaped noise and high-frequency emphasis noise. We also investigated the relationship between the MML and tinnitus loudness estimates and the Tinnitus Handicap Questionnaire (THQ). Results There were differences across the different noise types. The within-session standard deviation averaged across subjects varied between 1.3 and 1.8 dB. Across the two sessions, the Pearson correlation coefficients, range was r = .84. There was a weak relationship between the dB SL MML and loudness, and between the MML and the THQ. A moderate correlation ( r = .44) was found between the THQ and loudness estimates. Conclusions We conclude that the dB SL MML can be a reliable estimate of tinnitus magnitude, with expected standard deviations in trained subjects of about 1.5 dB. It appears that the dB SL MML and loudness estimates are not closely related.

Validation of the preference-based multiple sclerosis index

2018 ◽  
Vol 25 (11) ◽  
pp. 1496-1505 ◽  
Author(s):  
Ayse Kuspinar ◽  
Nancy E Mayo
Keyword(s):  
Multiple Sclerosis ◽  
Factor Analysis ◽  
Convergent Validity ◽  
Correlation Coefficients ◽  
Structural Validity ◽  
Related Quality ◽  
Health Related ◽  
Secondary Endpoints ◽  
Study Support ◽  
Generic Measures

Background: Preference-based measures of health-related quality of life (HRQL) are used as primary or secondary endpoints in multiple sclerosis (MS) research. Objective: The purpose of this paper was to evaluate the structural, convergent, and known-groups validity of the preference-based multiple sclerosis index (PBMSI) of HRQL in people with MS. Methods: Participants were recruited from three MS clinics in Montreal. Structural validity was assessed using polychoric correlation coefficients and factor analysis. To assess convergent validity, hypotheses were formulated about the strength of correlations between the PBMSI and other HRQL measures. Known-groups validity was assessed against different measures of disability. Results: The average age of the sample was 46 and 77% were women. Factor analysis supported the structural validity of the PBMSI; the items collectively were measuring one underlying construct. The PBMSI showed convergent validity against generic measures of HRQL, and known-groups validity between persons with different levels of disability. Conclusion: The results of this study support the construct validity of the PBMSI as an outcome measure of HRQL in MS. The PBMSI overcomes limitations observed with currently used HRQL measures in MS and may be used to contrast different interventions for people with MS.

scholarly journals Identifying Psychological Symptoms Based on Facial Movements

2020 ◽  
Vol 11 ◽  
Author(s):  
Xiaoyang Wang ◽  
Yilin Wang ◽  
Mingjie Zhou ◽  
Baobin Li ◽  
Xiaoqian Liu ◽  
...  
Keyword(s):  
Mental Health ◽  
Mental Illness ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Prediction Models ◽  
Psychological Symptoms ◽  
Correlation Coefficients ◽  
Criterion Validity ◽  
Microsoft Kinect ◽  
Facial Movements

Background: Many methods have been proposed to automatically identify the presence of mental illness, but these have mostly focused on one specific mental illness. In some non-professional scenarios, it would be more helpful to understand an individual's mental health status from all perspectives.Methods: We recruited 100 participants. Their multi-dimensional psychological symptoms of mental health were evaluated using the Symptom Checklist 90 (SCL-90) and their facial movements under neutral stimulation were recorded using Microsoft Kinect. We extracted the time-series characteristics of the key points as the input, and the subscale scores of the SCL-90 as the output to build facial prediction models. Finally, the convergent validity, discriminant validity, criterion validity, and the split-half reliability were respectively assessed using a multitrait-multimethod matrix and correlation coefficients.Results: The correlation coefficients between the predicted values and actual scores were 0.26 and 0.42 (P < 0.01), which indicated good criterion validity. All models except depression had high convergent validity but low discriminant validity. Results also indicated good levels of split-half reliability for each model [from 0.516 (hostility) to 0.817 (interpersonal sensitivity)] (P < 0.001).Conclusion: The validity and reliability of facial prediction models were confirmed for the measurement of mental health based on the SCL-90. Our research demonstrated that fine-grained aspects of mental health can be identified from the face, and provided a feasible evaluation method for multi-dimensional prediction models.

scholarly journals Adaptation and Psychometric Evaluation of the Chinese Counseling Competencies Scale-Revised

2021 ◽  
Vol 12 ◽  
Author(s):  
Wei Xia ◽  
William Ho Cheung Li ◽  
Tingna Liang ◽  
Yuanhui Luo ◽  
Laurie Long Kwan Ho ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Psychometric Evaluation ◽  
Counseling Competencies ◽  
Counselors In Training ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

Objectives: This study conducted a linguistic and psychometric evaluation of the Chinese Counseling Competencies Scale-Revised (CCS-R).Methods: The Chinese CCS-R was created from the original English version using a standard forward-backward translation process. The psychometric properties of the Chinese CCS-R were examined in a cohort of 208 counselors-in-training by two independent raters. Fifty-three counselors-in-training were asked to undergo another counseling performance evaluation for the test-retest. The confirmatory factor analysis (CFA) was conducted for the Chinese CCS-R, followed by internal consistency, test-retest reliability, inter-rater reliability, convergent validity, and concurrent validity.Results: The results of the CFA supported the factorial validity of the Chinese CCS-R, with adequate construct replicability. The scale had a McDonald's omega of 0.876, and intraclass correlation coefficients of 0.63 and 0.90 for test-retest reliability and inter-rater reliability, respectively. Significantly positive correlations were observed between the Chinese CCS-R score and scores of performance checklist (Pearson's γ = 0.781), indicating a large convergent validity, and knowledge on drug abuse (Pearson's γ = 0.833), indicating a moderate concurrent validity.Conclusion: The results support that the Chinese CCS-R is a valid and reliable measure of the counseling competencies.Practice implication: The CCS-R provides trainers with a reliable tool to evaluate counseling students' competencies and to facilitate discussions with trainees about their areas for growth.

scholarly journals High frequency ultrasound assessment of systemic sclerosis skin involvement: intra-observer repeatability and relationship with clinician assessment and dermal collagen content

2020 ◽  
pp. jrheum.200234
Author(s):  
Victoria A. Flower ◽  
Shaney L. Barratt ◽  
Darren J. Hart ◽  
Amanda B. Mackenzie ◽  
Jacqueline A. Shipley ◽  
...  
Keyword(s):  
Systemic Sclerosis ◽  
High Frequency ◽  
Convergent Validity ◽  
Objective Assessment ◽  
Collagen Content ◽  
Skin Thickness ◽  
High Frequency Ultrasound ◽  
Dermal Collagen ◽  
The Relationship ◽  
Frequency Ultrasound

Objective The modified Rodnan skin score (mRSS) remains the preferred method for skin assessment in systemic sclerosis (SSc). There are concerns regarding high inter-observer variability of mRSS and negative clinical trials utilising mRSS as the primary endpoint. High frequency ultrasound (HFUS) allows objective assessment of cutaneous fibrosis in SSc. We investigated the relationship between HFUS with both mRSS and dermal collagen. Methods Skin thickness (ST), echogenicity and novel Shear wave elastography (SWE) were assessed in 53 SSc patients and 15 healthy controls (HC) at the finger, hand, forearm and abdomen. The relationship between HFUS parameters with mRSS (n=53) and dermal collagen (10 SSc patients and 10 HC) was investigated. Intra-observer repeatability of HFUS was calculated using intra-class correlation coefficients (ICCs). Results HFUS assessment of ST (hand/forearm) and SWE (finger/hand) correlated with local mRSS at some sites. Subclinical abnormalities in ST, echogenicity and SWE were present in clinically uninvolved SSc skin. Additionally, changes in echogenicity and SWE were sometimes apparent despite objectively normal ST on HFUS. ST, SWE and local mRSS correlated strongly with collagen quantification (rho 0.697, 0.709, 0.649 respectively). Intra-observer repeatability was high for all HFUS parameters (ICCs for ST 0.946-0.978, echogenicity 0.648- 0.865 and SWE 0.953-0.973). Conclusion Our data demonstrates excellent reproducibility and reassuring convergent validity with dermal collagen content. Detection of subclinical abnormalities is an additional benefit of HFUS. The observed correlations with collagen quantification support further investigation of HFUS as an alternative to mRSS in clinical trial settings.

Preliminary appraisal of the reliability and validity of the Colorado State University Feline Acute Pain Scale

2018 ◽  
Vol 21 (4) ◽  
pp. 335-339 ◽  
Author(s):  
Hilary Shipley ◽  
Alonso Guedes ◽  
Lynelle Graham ◽  
Elizabeth Goudie-DeAngelis ◽  
Erin Wendt-Hornickle
Keyword(s):  
Acute Pain ◽  
Convergent Validity ◽  
Correlation Coefficients ◽  
Pain Scale ◽  
State University ◽  
Good Reliability ◽  
Rater Reliability ◽  
Spearman Rank ◽  
Colorado State University ◽  
Cohen's Kappa

Objectives The objective of this study was to determine the inter-rater reliability and convergent validity of the Colorado State University Feline Acute Pain Scale (CSU-FAPS) in a preliminary appraisal of its performance in a clinical teaching setting. Methods Sixty-eight female cats were assessed for pain after ovariohysterectomy. A cohort of 21 cats was examined independently by four raters (two board-certified anesthesiologists and two anesthesia residents) with the CSU-FAPS, and intra-class correlation coefficient (ICC) was used to determine inter-rater reliability. Weighted Cohen’s kappa was used to determine inter-rater reliability centered on the ‘need to reassess analgesic plan’ (dichotomous scale). A separate cohort of 47 cats was evaluated independently by two raters (one board-certified anesthesiologist and one veterinary small animal rotating intern) using the CSU-FAPS and the Glasgow Composite Measure Pain Scale (CMPS-Feline), and Spearman rank-order correlation was determined to assess convergent validity. Reliability was interpreted using Altman’s classification as very good, good, moderate, fair and poor. Validity was considered adequate if correlation coefficients were between 0.4 and 0.8. Results The ICC was 0.61 for anesthesiologists and 0.67 for residents, indicating good reliability. Weighted Cohen’s kappa was 0.79 for anesthesiologists and 0.44 for residents, indicating moderate to good reliability. The Spearman rank correlation indicated a statistically significant ( P = 0.0003) positive correlation (0.31; 95% confidence interval 0.14–0.46) between the CSU-FAPS and the CMPS-Feline. Conclusions and relevance The CSU-FAPS showed moderate-to-good inter-rater reliability when used by veterinarians to assess pain level or need to reassess analgesic plan after ovariohysterectomy in cats. The validity fell short of current guidelines for correlation coefficients and further refinement and testing are warranted to improve its performance.

Associations Among the MCMI-II Clinical Syndrome Scales and the MMPI-2 Clinical Scales

Assessment ◽  
1994 ◽  
Vol 1 (4) ◽  
pp. 407-413 ◽  
Author(s):  
Mark A. Blais ◽  
Kenneth B. Benedict ◽  
Dennis K. Norman
Keyword(s):  
Convergent Validity ◽  
Discriminant Validity ◽  
Minnesota Multiphasic Personality Inventory ◽  
Pearson Correlation ◽  
Correlation Coefficients ◽  
Principal Component ◽  
Clinical Syndrome ◽  
Self Report ◽  
Clinical Scales ◽  
Axis I

The Millon Clinical Multiaxial Inventory—II (MCMI-II), a frequently used self-report measure of psychopathology, contains nine scales designed to assess Axis I psychopathology (the clinical syndrome and severe syndrome scales). This study explored the relationships among these nine MCMI-II clinical syndrome scales and the clinical scales of the Minnesota Multiphasic Personality Inventory–2 (MMPI-2). A sample of 108 psychiatric inpatients was administered both the MCMI-II and the MMPI-2 within 7 days of admission. Pearson correlation coefficients and principal component factors were obtained for the MCMI-II and MMPI-2 scales. The results provided support for the convergent validity of all the MCMI-II Axis I scales. However, the majority of the MCMI-II scales failed to demonstrate adequate discriminant validity in relation to the MMPI-2 scales. The principal component analysis revealed that method variance was the principal influence in determining factor loadings for the majority of test scales. This finding suggests that these two popular self-report tests differ substantially in how they measure psychopathology.

A New Approach to Collecting Survey Data: An Item Response Icon Scale (IRIS)

1995 ◽  
Vol 39 (12) ◽  
pp. 804-808
Author(s):  
Leslie Beth Herbert ◽  
Donald I. Tepas
Keyword(s):  
Item Response ◽  
Survey Data ◽  
Convergent Validity ◽  
Word Processing ◽  
Reading Skills ◽  
Discriminant Validity ◽  
Research Process ◽  
Design Tool ◽  
Cultural Backgrounds ◽  
Parallel Forms

Obtaining worker survey data can be a time consuming process. A survey often employs a large number of items, and considerable effort is required to assure that the items used will be understood by the target worker population. To facilitate the research process in a diverse worker population, a five-point Item Response Icon Scale (IRIS) was developed using the symbols found in Wingdings, a standard font included in many microcomputer word processing programs. The IRIS and a five-point verbal Likert-type Scale were found to be parallel forms, with demonstration of statistically significant parallel forms reliability and convergent validity. These findings, along with that of discriminant validity, indicate that IRIS and Likert-type Scale produce interchangeable data when given to individuals with reading skills. This suggests that the IRIS may be a valuable design tool for collecting survey data from worker groups with diverse reading skills and/or from different cultural backgrounds.

The Music Self-Perception Inventory: Development of parallel forms A and B

2016 ◽  
Vol 45 (4) ◽  
pp. 530-549 ◽  
Author(s):  
Alexandre J. S. Morin ◽  
L. Francesca Scalas ◽  
Walter Vispoel
Keyword(s):  
Psychometric Properties ◽  
Convergent Validity ◽  
Repeated Measurement ◽  
Self Concept ◽  
Self Perception ◽  
Musical Practice ◽  
Parallel Forms ◽  
Inventory Development

Music self-concept integrates perceptions, beliefs, and self-schemas about a person’s musical abilities and potential. Like other self-concept dimensions, it is multifaceted, hierarchically organized and has implications for motivation toward musical practice. The Music Self-Perception Inventory (MUSPI) is a theoretically-based instrument assessing six specific music self-concept dimensions, as well as global music self-concept. Nonetheless, its applicability is limited by its length (84 items) and by the fact that it does not provide a way to control for consistency biases in the context of repeated measurement. In this study, we developed and validated two parallel versions (A and B) of the MUSPI, and showed that both yielded equivalent psychometric properties to the original, and were fully equivalent to one another. We also tested whether the MUSPI-A and MUSPI-B psychometric properties generalized (were invariant) across gender and grade-differentiated subgroups. Finally, we examined the convergent validity of the MUSPI-A and MUSPI-B. Results highlighted the psychometric soundness, and equivalence, of the various MUSPI versions on all criteria, and showed that they presented patterns of associations with other constructs equivalent to that observed with the original MUSPI.

The Reliability and Convergent Validity of Field Tests of Body Composition in Young Adolescents

2006 ◽  
Vol 3 (s2) ◽  
pp. S67-S77 ◽  
Author(s):  
Michelle Ihmels ◽  
Gregory J. Welk ◽  
James J. McClain ◽  
Jodee Schaben
Keyword(s):  
Body Composition ◽  
Body Fat ◽  
Repeated Measures ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Field Tests ◽  
Correlation Coefficients ◽  
Young Adolescents ◽  
Intraclass Correlation Coefficients ◽  
Classification Agreement

Background:Advances in BIA offer practical alternative approaches to assessing body composition in young adolescents and have not been studied for comparability.Methods:This study compared reliability and convergent validity of three field tests (2-site skinfold, Omron and Tanita BIA devices) on young adolescents. Reliability was determined using intraclass correlation coefficients, convergent validity was examined by computing correlations among the three estimates, differences in estimated body fat values were evaluated using repeated-measures ANOVA, and classification agreement was computed for achieving FITNESSGRAM® Healthy Fitness Zone.Results:ICC values of all three measures exceeded .97. Correlations ranged from .74 to .81 for males and .79 to .91 for females. Classification agreement values ranged from 82.8% to 92.6%.Conclusions:Results suggest general agreement among the selected methods of body composition assessments in both boys and girls with the exception that percent body fat in boys by Tanita BIA is significantly lower than skinfold estimation.

Sign in / Sign up

Export Citation Format

Share Document