scholarly journals Reliability and Validity of Composite Scores from the NIH Toolbox Cognition Battery in Adults

2014 ◽  
Vol 20 (6) ◽  
pp. 588-598 ◽  
Author(s):  
Robert K. Heaton ◽  
Natacha Akshoomoff ◽  
David Tulsky ◽  
Dan Mungas ◽  
Sandra Weintraub ◽  
...  
Keyword(s):  
Cognitive Abilities ◽  
Gold Standard ◽  
Reliability And Validity ◽  
Epidemiologic Studies ◽  
Memory Processing ◽  
Significant Relationships ◽  
Gender Education ◽  
Computer Based ◽  
Composite Scores ◽  
Test Retest Reliability

AbstractThis study describes psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) Composite Scores in an adult sample. The NIHTB-CB was designed for use in epidemiologic studies and clinical trials for ages 3 to 85. A total of 268 self-described healthy adults were recruited at four university-based sites, using stratified sampling guidelines to target demographic variability for age (20–85 years), gender, education, and ethnicity. The NIHTB-CB contains seven computer-based instruments assessing five cognitive sub-domains: Language, Executive Function, Episodic Memory, Processing Speed, and Working Memory. Participants completed the NIHTB-CB, corresponding gold standard validation measures selected to tap the same cognitive abilities, and sociodemographic questionnaires. Three Composite Scores were derived for both the NIHTB-CB and gold standard batteries: “Crystallized Cognition Composite,” “Fluid Cognition Composite,” and “Total Cognition Composite” scores. NIHTB Composite Scores showed acceptable internal consistency (Cronbach’s alphas=0.84 Crystallized, 0.83 Fluid, 0.77 Total), excellent test–retest reliability (r: 0.86–0.92), strong convergent (r: 0.78–0.90) and discriminant (r: 0.19–0.39) validities versus gold standard composites, and expected age effects (r=0.18 crystallized, r=−0.68 fluid, r=−0.26 total). Significant relationships with self-reported prior school difficulties and current health status, employment, and presence of a disability provided evidence of external validity. The NIH Toolbox Cognition Battery Composite Scores have excellent reliability and validity, suggesting they can be used effectively in epidemiologic and clinical studies. (JINS, 2014, 20, 1–11)

scholarly journals Reliability, Validity, and Feasibility of a Computer-Based Geriatric Assessment for Older Adults With Cancer

2016 ◽  
Vol 12 (12) ◽  
pp. e1025-e1034 ◽  
Author(s):  
Arti Hurria ◽  
Chie Akiba ◽  
Jerome Kim ◽  
Dale Mitani ◽  
Matthew Loscalzo ◽  
...  
Keyword(s):  
Older Adults ◽  
Geriatric Assessment ◽  
Hematologic Malignancy ◽  
Social Activity ◽  
Reliability And Validity ◽  
Data Capture ◽  
Computer Based ◽  
Paper And Pencil ◽  
Test Retest Reliability ◽  
Completion Times

Purpose: The goal of this study was to evaluate the feasibility, reliability, and validity of a computer-based geriatric assessment via two methods of electronic data capture ( SupportScreen and REDCap) compared with paper-and-pencil data capture among older adults with cancer. Methods: Eligible patients were ≥ 65 years old, had a cancer diagnosis, and were fluent in English. Patients were randomly assigned to one of four arms, in which they completed the geriatric assessment twice: (1) REDCap and paper and pencil in sessions 1 and 2; (2) REDCap in both sessions; (3) SupportScreen and paper and pencil in sessions 1 and 2; and (4) SupportScreen in both sessions. The feasibility, reliability, and validity of the computer-based geriatric assessment compared with paper and pencil were evaluated. Results: The median age of participants (N = 100) was 71 years (range, 65 to 91 years) and the diagnosis was solid tumor (82%) or hematologic malignancy (18%). For session 1, REDCap took significantly longer to complete than paper and pencil (median, 21 minutes [range, 11 to 44 minutes] v median, 15 minutes [range, 9 to 29 minutes], P < .01) or SupportScreen (median, 16 minutes [range, 6 to 38 minutes], P < .01). There were no significant differences in completion times between SupportScreen and paper and pencil ( P = .50). The computer-based geriatric assessment was feasible. Few participants (8%) needed help with completing the geriatric assessment (REDCap, n = 7 and SupportScreen, n = 1), 89% reported that the length was “just right,” and 67% preferred the computer-based geriatric assessment to paper and pencil. Test–retest reliability was high (Spearman correlation coefficient ≥ 0.79) for all scales except for social activity. Validity among similar scales was demonstrated. Conclusion: Delivering a computer-based geriatric assessment is feasible, reliable, and valid. SupportScreen methodology is preferred to REDCap.

scholarly journals NIH Toolbox Cognition Battery (CB): Validation of Executive Function Measures in Adults

2014 ◽  
Vol 20 (6) ◽  
pp. 620-629 ◽  
Author(s):  
Philip David Zelazo ◽  
Jacob E. Anderson ◽  
Jennifer Richler ◽  
Kathleen Wallner-Allen ◽  
Jennifer L. Beaumont ◽  
...  
Keyword(s):  
Executive Function ◽  
Discriminant Validity ◽  
Flanker Task ◽  
Dimensional Change ◽  
Epidemiologic Studies ◽  
Card Sort ◽  
Convergent And Discriminant Validity ◽  
Age Related ◽  
Computer Based ◽  
Test Retest Reliability

AbstractThis study describes psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) executive function measures in an adult sample. The NIHTB-CB was designed for use in epidemiologic studies and clinical trials for ages 3 to 85. A total of 268 self-described healthy adults were recruited at four university-based sites, using stratified sampling guidelines to target demographic variability for age (20–85 years), gender, education and ethnicity. The NIHTB-CB contains two computer-based instruments assessing executive function: the Dimensional Change Card Sort (a measure of cognitive flexibility) and a flanker task (a measure of inhibitory control and selective attention). Participants completed the NIHTB-CB, corresponding gold standard convergent and discriminant measures, and sociodemographic questionnaires. A subset of participants (N=89) was retested 7 to 21 days later. Results reveal excellent sensitivity to age-related changes during adulthood, excellent test–retest reliability, and adequate to good convergent and discriminant validity. The NIH Toolbox EF measures can be used effectively in epidemiologic and clinical studies. (JINS, 2014, 20, 1–10)

scholarly journals Constructing the Children’s Future Thinking Questionnaire: A reliable and valid measure of children’s future-oriented cognition

2019 ◽  
Author(s):  
Tessa Mazachowsky ◽  
Caitlin Mahy
Keyword(s):  
Internal Consistency ◽  
Cognitive Abilities ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Delay Of Gratification ◽  
Parent Report ◽  
Future Thinking ◽  
Critical Set ◽  
High Internal Consistency ◽  
Test Retest Reliability

Future-oriented cognition (planning, prospective memory, episodic foresight, saving, and delay of gratification) involves a critical set of skills that children must develop for successful daily functioning. The current study developed the Children’s Future Thinking Questionnaire (CFTQ), a parent-report on 3- to 7-year-olds’ future-oriented cognition. The CFTQ showed high internal consistency and detected development of future-oriented cognition (Study 1; N = 145). Study 2 (N = 255) showed high internal consistency reliability and preliminary validity of the CFTQ. Study 3 (N = 101) confirmed strong reliability and validity of the CFTQ. Study 4 (N = 105) revealed excellent test-retest reliability of the CFTQ. Thus, the CFTQ is the first reliable and valid parent-report measure of children’s developing future-oriented cognitive abilities.

scholarly journals Computerized neuropsychological assessment devices in multiple sclerosis: A systematic review

2019 ◽  
Vol 25 (14) ◽  
pp. 1848-1869 ◽  
Author(s):  
Curtis M Wojcik ◽  
Meghan Beier ◽  
Kathleen Costello ◽  
John DeLuca ◽  
Anthony Feinstein ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Cognitive Impairment ◽  
Neuropsychological Assessment ◽  
Vital Signs ◽  
Reliability And Validity ◽  
Good Evidence ◽  
Computerized Testing ◽  
Online Databases ◽  
Computer Based ◽  
Test Retest Reliability

Background: The proliferation of computerized neuropsychological assessment devices (CNADs) for screening and monitoring cognitive impairment is increasing exponentially. Previous reviews of computerized tests for multiple sclerosis (MS) were primarily qualitative and did not rigorously compare CNADs on psychometric properties. Objective: We aimed to systematically review the literature on the use of CNADs in MS and identify test batteries and single tests with good evidence for reliability and validity. Method: A search of four major online databases was conducted for publications related to computerized testing and MS. Test–retest reliability and validity coefficients and effect sizes were recorded for each CNAD test, along with administration characteristics. Results: We identified 11 batteries and 33 individual tests from 120 peer-reviewed articles meeting the inclusion criteria. CNADs with the strongest psychometric support include the CogState Brief Battery, Cognitive Drug Research Battery, NeuroTrax, CNS-Vital Signs, and computer-based administrations of the Symbol Digit Modalities Test. Conclusion: We identified several CNADs that are valid to screen for MS-related cognitive impairment, or to supplement full, conventional neuropsychological assessment. The necessity of testing with a technician, and in a controlled clinic/laboratory environment, remains uncertain.

The Brief Social Phobia Scale: a psychometric evaluation

1997 ◽  
Vol 27 (1) ◽  
pp. 161-166 ◽  
Author(s):  
J. R. T. DAVIDSON ◽  
C. M. MINER ◽  
J. DE VEAUGH-GEISS ◽  
L. A. TUPLER ◽  
J. T. COLKET ◽  
...  
Keyword(s):  
Factor Analysis ◽  
Social Phobia ◽  
Rating Scale ◽  
Reliability And Validity ◽  
Psychometric Evaluation ◽  
Fear Avoidance ◽  
Significant Relationships ◽  
Point Rating Scale ◽  
High Level ◽  
Test Retest Reliability

The Brief Social Phobia Scale (BSPS) is an observer-rated scale designed to assess the characteristic symptoms of social phobia, using three subscales – fear, avoidance, and physiological arousal – which may be combined into a total score. Each of 18 BSPS items is anchored to a 5-point rating scale. Psychometric evaluation of the BSPS in a sample of 275 social-phobia patients yielded a high level of reliability and validity. Test–retest reliability was excellent, as was internal consistency. The fear and avoidance subscales demonstrated highly significant correlations with remaining item totals; however, the physiological subscale did not. The BSPS also demonstrated significant relationships with other established scales that assess anxiety and disability, and it proved sensitive to treatment effects in a trial of a 5-HT3 antagonist and placebo. Factor analysis yielded six meaningful factors. We conclude that the BSPS provides a reliable, valid, and sensitive measure for the evaluation of social phobia.

scholarly journals A Study of the Effectiveness Verification of Computer-Based Dementia Assessment Contents (Co-Wis): Non-Randomized Study

2020 ◽  
Vol 10 (5) ◽  
pp. 1579
Author(s):  
Seung Il Song ◽  
Hyun Seok Jeong ◽  
Jung Pil Park ◽  
Ji Yean Kim ◽  
Dai Seg Bai ◽  
...  
Keyword(s):  
Factor Analysis ◽  
Concurrent Validity ◽  
Rating Scale ◽  
High Reliability ◽  
Reliability And Validity ◽  
Cognitive Test ◽  
Retest Reliability ◽  
Computer Based ◽  
Neuropsychological Assessments ◽  
Test Retest Reliability

Computer-based neuropsychological assessments have many advantages over traditional neuropsychological assessments. However, limited data are available on the validity and reliability of computer-based assessments. The purpose of this study was to examine the reliability and validity of computer-based dementia assessment contents (Co-Wis). This study recruited 113 participants from Yeungnam University Medical Center in Daegu from June 2019 to December 2019 and received ethical approval. Participants were evaluated using standard and objective dementia cognitive test tools such as the Korean version of the Mini-Mental State Examination (K-MMSE), the Clinical Dementia Rating Scale (CDR), and the Standardized Seoul Neuropsychological Screening Battery-II (SNSB-II). To verify the effectiveness of Co-Wis, the concurrent validity, test–retest reliability (Pearson’s correlation coefficients), construct validity (Factor analysis), and signal detection analysis (ROC curve) were used. In most of the Co-Wis subtests, the concurrent validity and test–retest reliability showed statistically significant correlations (p < 0.05, p < 0.01). The factor analysis showed that Co-Wis assessed the most major cognitive areas (Tucker–Lewis Index (TLI) = 0.876, Comparative Fit Index (CFI) = 0.897, RMSEA = 0.88). Thus, Co-Wis appears clinically applicable and with high reliability and validity. In the future, we should develop tests to evaluate both standard data and big data-based machine learning.

Some Preliminary Findings on New Maacl Scales

1983 ◽  
Vol 53 (3_suppl) ◽  
pp. 1205-1206
Author(s):  
Roberta Y. Gosselin ◽  
Bernard Lubin ◽  
R. Michael Sokoloff
Keyword(s):  
Positive Affect ◽  
Sensation Seeking ◽  
Reliability And Validity ◽  
The State ◽  
Composite Score ◽  
Retest Reliability ◽  
Composite Scores ◽  
Test Retest Reliability ◽  
Anxiety Depression ◽  
State Form

To determine reliability and validity of the new composite scores on the MAACL, 91 male and 98 female adults completed either the state or trait forms of the MAACL together with either the state or trait forms of the POMS on two separate occasions. Test-retest reliability was as expected: high for Composite score 1 (2 anxiety + depression + hostility) and for Composite score 2 (2 positive affect + sensation seeking) on the trait form and for Composite 2 on the state form. As expected, test-retest reliability for Composite 1 on the state form was low. All alphas were .87 and above. Correlations between the MAACL Composite 1 and the Composite score on the POMS were moderate and significant for both trait (.52) and state (.65) forms. Also as expected, correlations between MAACL Composite 1. and POMS Vigor were non-significant (trait and state), and those between MAACL Composite 2 and the Composite score on the POMS (trait and state) were negative.

Is There a Measurement Crisis in Disgust Research?

2020 ◽  
Author(s):  
Thomas Armstrong ◽  
Danica Wilbanks ◽  
Daniel Leong ◽  
Kean J. Hsu
Keyword(s):  
Mental Disorders ◽  
Clinical Psychology ◽  
Reliability And Validity ◽  
Self Report ◽  
Behavioral Therapies ◽  
Cognitive Behavioral Therapies ◽  
Societal Problems ◽  
Test Retest Reliability ◽  
Pathogen Avoidance ◽  
Corrective Information

Once a forgotten emotion, disgust is now studied in fields from evolutionary to clinical psychology. Although highly adaptive as a pathogen avoidance mechanism, disgust is prone to false positives. Indeed, several anxiety-related disorders involve excessive and irrational disgust. Furthermore, disgust resists corrective information, making it difficult to treat through cognitive-behavioral therapies. A deeper understanding of disgust could improve the treatment of mental disorders and other societal problems involving this peculiar emotion. However, researchers may need to improve the measurement of disgust to gain such insights. In this paper, we review psychology’s “measurement crisis” in the context of disgust. We suggest that self-report measures, though optimal in reliability, have compromised validity because the vernacular usage of disgust captures neighboring states of discomfort and disapproval. In addition to potential validity issues, we find that most non-self-report measures of disgust have questionable reliability. Internal consistency and test-retest reliability were rarely reported for psychophysiological and neural measures, but the information available suggests that these measures of disgust have poor reliability and may not support individual difference research crucial to clinical psychology. In light of this assessment, we provide several recommendations for improving the reliability and validity of disgust measurement, including renewed attention to theory.

Improving Skills in Senior Secondary Schools for Effective Performance in Computer Based Test and Security.

2019 ◽  
pp. 167-174
Author(s):  
Agu B.G. ◽  
Eya G.M.
Keyword(s):  
Cognitive Abilities ◽  
Computer Literacy ◽  
Literacy Skills ◽  
School Students ◽  
Effective Performance ◽  
Information And Communication ◽  
Computer Based ◽  
Senior Secondary Schools ◽  
Matriculation Examination ◽  
Senior Secondary

Students are accessed using paper and pen on cognitive abilities in Nigeria. This method of assessment encourages different forms of examination malpractices. The threat of examination malpractices on the validity of examination outcomes has resulted in some examination bodies adopting different methods of examination. One of such methods is the computer based test (CBT). Using survey research method, this study investigated the levels of competency in computer literacy skills possessed by senior secondary school students. Three research questions and one hypothesis guided the study. Problems encountered by the students and prospective methods of enhancing computer based test (CBT) acceptance in Nigeria were also documented. A total of 310 copies of the questionnaire were administered to students who participated in the 2015 Unified Tertiary Matriculation Examination (UTME) at Afrihub Information and Communication Technology (CBT) centre, Institution of Management and Technology (IMT), Enugu and Godfrey Okoye University, Enugu and 237 representing 76% were adequately completed and found usable. The findings revealed among others that majority of the respondents confirmed that CBT can curb examination malpractice. Majority of candidates were also found to prefer CBT to the paper and pencil test (PPT). The mean, standard deviation and Pearson‟s Correlation Analysis showed that the respondent is preference for CBT were sensitive across gender. While improving electricity was identified as critical in enhancing CBT examinations, poor ICT skills on the part of students and the invigilators were also identified as the major problems facing the implementation of JAMB CBT examination in Nigeria.

Test-Retest-Reliability of Computer-Based Metamorphopsia Measurement in Macular Diseases

2020 ◽  
Author(s):  
Daniela Claessens ◽  
Alexander K. Schuster ◽  
Ronald V. Krüger ◽  
Marian Liegl ◽  
Laila Singh ◽  
...  
Keyword(s):  
Macular Edema ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Measuring Method ◽  
Repeated Measurements ◽  
Retest Reliability ◽  
Macular Diseases ◽  
Computer Based ◽  
Test Retest Reliability

AbstractIn this study, the test-retest-reliability as one aspect of reliability of metamorphopsia measurements using a computer-based measuring method was determined in patients with macular diseases. Metamorphopsia amplitude, position, and area were quantified using AMD – A Metamorphopsia Detector software (app4eyes GmbH & Co. KG, Germany) in patients with diabetic, myopic, or uveitic macular edema, intermediate or neovascular age-associated macular degeneration, epiretinal membrane, vitelliform maculopathy, Irvine-Gass syndrome, or macular edema due to venous retinal occlusion. The intraclass correlation coefficient (ICC) was calculated in order to determine the repeatability of two repeated measurements and was used as an indicator of the reliability of the measurements. In this study, metamorphopsia measurements were conducted on 36 eyes with macular diseases. Metamorphopsia measurements made using AMD – A Metamorphopsia Detector software were highly reliable and repeatable in patients with maculopathies. The intraclass correlation coefficient of all indices was excellent (0.95 – 0.97). For diseases of the vitreoretinal interface or macular diseases with intra- or subretinal edema, this metamorphopsia measurement represents a supplement for visual function testing in the clinic, as well as in clinical studies.

Sign in / Sign up

Export Citation Format

Share Document