scholarly journals Reliability, and Convergent and Discriminant Validity of Gaming Disorder Scales: A Meta-Analysis

2021 ◽  
Vol 12 ◽  
Author(s):  
Seowon Yoon ◽  
Yeji Yang ◽  
Eunbin Ro ◽  
Woo-Young Ahn ◽  
Jueun Kim ◽  
...  
Keyword(s):  
Systematic Review ◽  
Internet Addiction ◽  
Discriminant Validity ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Gaming Disorder ◽  
Reliability Generalization ◽  
Retest Reliability ◽  
Convergent And Discriminant Validity ◽  
Test Retest Reliability

Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs.Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized.Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85–0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81–0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), –0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented.Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies.Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study.Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

A Test-Retest Reliability Generalization Meta-Analysis of Judgments Via the Policy-Capturing Technique

2021 ◽  
pp. 109442812110115
Author(s):  
Ze Zhu ◽  
Alan J. Tomassetti ◽  
Reeshad S. Dalal ◽  
Shannon W. Schrader ◽  
Kevin Loo ◽  
...  
Keyword(s):  
Best Practice ◽  
Meta Analysis ◽  
Temporal Stability ◽  
Future Research ◽  
Policy Capturing ◽  
Reliability Estimate ◽  
Reliability Generalization ◽  
Retest Reliability ◽  
Reliability Estimates ◽  
Test Retest Reliability

Policy capturing is a widely used technique, but the temporal stability of policy-capturing judgments has long been a cause for concern. This article emphasizes the importance of reporting reliability, and in particular test-retest reliability, estimates in policy-capturing studies. We found that only 164 of 955 policy-capturing studies (i.e., 17.17%) reported a test-retest reliability estimate. We then conducted a reliability generalization meta-analysis on policy-capturing studies that did report test-retest reliability estimates—and we obtained an average reliability estimate of .78. We additionally examined 16 potential methodological and substantive antecedents to test-retest reliability (equivalent to moderators in validity generalization studies). We found that test-retest reliability was robust to variation in 14 of the 16 factors examined but that reliability was higher in paper-and-pencil studies than in web-based studies and was higher for behavioral intention judgments than for other (e.g., attitudinal and perceptual) judgments. We provide an agenda for future research. Finally, we provide several best-practice recommendations for researchers (and journal reviewers) with regard to (a) reporting test-retest reliability, (b) designing policy-capturing studies for appropriate reportage, and (c) properly interpreting test-retest reliability in policy-capturing studies.

scholarly journals The 2014 Ontario Child Health Study Emotional Behavioural Scales (OCHS-EBS) Part I: A Checklist for Dimensional Measurement of Selected DSM-5 Disorders

2018 ◽  
Vol 64 (6) ◽  
pp. 423-433 ◽  
Author(s):  
Laura Duncan ◽  
Katholiki Georgiades ◽  
Li Wang ◽  
Jinette Comeau ◽  
Mark A. Ferro ◽  
...  
Keyword(s):  
General Population ◽  
Child Health ◽  
Conduct Disorder ◽  
Discriminant Validity ◽  
Health Study ◽  
Retest Reliability ◽  
Dimensional Measurement ◽  
Dsm 5 ◽  
Convergent And Discriminant Validity ◽  
Test Retest Reliability

Objectives: To describe the development and psychometric properties of the 2014 Ontario Child Health Study Emotional Behavioural Scales (OCHS-EBS) for dimensional measurement of 7 disorders based on criteria from the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders ( DSM-5). Methods: Scale items were selected by agreement among 19 child psychologists and psychiatrists rating the correspondence between item descriptions and DSM-5 symptoms. Psychometric evaluation of the item properties and parent/caregiver and youth scales came from a general population study of 10,802 children and youth aged 4 to 17 years in 6537 families. Test-retest reliability data were collected from a subsample of 280 children and their caregivers who independently completed the OCHS-EBS checklist on 2 occasions 7 to 14 days apart. Structural equation modelling was used to assess internal and external convergent and discriminant validity—the latter tested against the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). Results: Confirmatory factor analyses exhibited adequate item fit to all scales. Except for conduct disorder and youth-assessed separation anxiety disorder, internal (Cronbach’s α) and test-retest reliability (Pearson’s r) for scale scores were 0.70 or above. Except for youth-assessed conduct disorder, the OCHS-EBS met criteria for internal and convergent and discriminant validity. Compared with the MINI-KID, the OCHS-EBS met criteria for external convergent and discriminant validity. Conclusions: The OCHS-EBS provide reliable and valid dimensional measurement of 7 DSM-5 disorders assessed by caregivers and youth in the general population. Part II describes use of the OCHS-EBS as a categorical (present/absent) measure of disorder.

scholarly journals Recommended motor assessments based on psychometric properties in individuals with dementia: a systematic review

2019 ◽  
Vol 16 (1) ◽  
Author(s):  
Sandra Trautwein ◽  
Philipp Maurus ◽  
Bettina Barisch-Fritz ◽  
Anela Hadzic ◽  
Alexander Woll
Keyword(s):  
Physical Activity ◽  
Systematic Review ◽  
Psychometric Properties ◽  
Randomised Controlled Trials ◽  
Effect Sizes ◽  
Controlled Trials ◽  
Retest Reliability ◽  
Frequency Of Use ◽  
Randomised Controlled ◽  
Test Retest Reliability

Abstract Background Motor assessments are important to determine effectiveness of physical activity in individuals with dementia (IWD). However, inappropriate and non-standardised assessments without sound psychometric properties have been used. This systematic review aims to examine psychometric properties of motor assessments in IWD combined with frequency of use and effect sizes and to provide recommendations based on observed findings. We performed a two-stage systematic literature search using Pubmed, Web of Science, Cochrane Library, ALOIS, and Scopus (inception - July/September 2018, English and German). The first search purposed to identify motor assessments used in randomised controlled trials assessing effectiveness of physical activity in IWD and to display their frequency of use and effect sizes. The second search focused on psychometric properties considering influence of severity and aetiology of dementia and cueing on test-retest reliability. Two reviewers independently extracted and analysed findings of eligible studies in a narrative synthesis. Results Literature searches identified 46 randomised controlled trials and 21 psychometric property studies. While insufficient information was available for validity, we observed sufficient inter-rater and relative test-retest reliability but unacceptable absolute test-retest reliability for most assessments. Combining these findings with frequency of use and effect sizes, we recommend Functional Reach Test, Groningen Meander Walking Test (time), Berg Balance Scale, Performance Oriented Mobility Assessment, Timed Up & Go Test, instrumented gait analysis (spatiotemporal parameters), Sit-to-Stand assessments (repetitions> 1), and 6-min walk test. It is important to consider that severity and aetiology of dementia and cueing influenced test-retest reliability of some assessments. Conclusion This review establishes an important foundation for future investigations. Sufficient relative reliability supports the conclusiveness of recommended assessments at group level, while unacceptable absolute reliability advices caution in assessing intra-individual changes. Moreover, influences on test-retest reliability suggest tailoring assessments and instructions to IWD and applying cueing only where it is inevitable. Considering heterogeneity of included studies and insufficient examination in various areas, these recommendations are not comprehensive. Further research, especially on validity and influences on test-retest reliability, as well as standardisation and development of tailored assessments for IWD is crucial. This systematic review was registered in PROSPERO (CRD42018105399).

scholarly journals Test-Retest Reliability of Five Times Sit to Stand Test (FTSST) in Adults: A Systematic Review and Meta-Analysis

Biology ◽  
2021 ◽  
Vol 10 (6) ◽  
pp. 510
Author(s):  
Laura Muñoz-Bermejo ◽  
José Carmelo Adsuar ◽  
María Mendoza-Muñoz ◽  
Sabina Barrios-Fernández ◽  
Miguel A. Garcia-Gordillo ◽  
...  
Keyword(s):  
Systematic Review ◽  
Lower Limb ◽  
Balance Control ◽  
Meta Analysis ◽  
Functional Independence ◽  
Lower Limbs ◽  
Retest Reliability ◽  
Stand Test ◽  
Sit To Stand ◽  
Test Retest Reliability

Functional independence in adults is conditioned by lower limb muscle strength. Thus, it seems important to assess lower limb strength using reliable and easy to reproduce measurements. The purpose of this study was to conduct a systematic review and meta-analysis to collect studies that examined the test-retest reliability of the Five Times Sit to Stand Test (FTSST) in adults. The search was conducted in PubMed, Web of Science, and Scopus databases, including all studies published up to 28 December 2020. To be included, studies had to include relative reliability scores (ICC) and maximum torque or standard error of measurements (SEM) of FTSST. A total of 693 studies were initially identified, but only 8 met the eligibility criteria and were included in the meta-analysis, covering a total of 14 groups with 400 participants. Relative inter-rater reliability results (ICC = 0.937, p < 0.001, n = 400) revealed excellent reliability of FTSST to assess sitting and standing performance, lower limbs strength and balance control. Conclusion: The Five Times Sit to Stand Test is a highly reliable tool for assessing lower limbs strength, balance control, and mobility in both healthy adults and those with pathologies.

scholarly journals E-Cigarette Demand: Impact of Commodity Definitions and Test–Retest Reliability

2020 ◽  
Author(s):  
Justin C Strickland ◽  
Olga A Vsevolozhskaya ◽  
William W Stoops
Keyword(s):  
Effect Size ◽  
Discriminant Validity ◽  
Good Test ◽  
Retest Reliability ◽  
Cigarette Demand ◽  
Convergent And Discriminant Validity ◽  
Human Participants ◽  
Puff Intensity ◽  
Test Retest Reliability

Abstract Introduction Behavioral economic demand provides a multidimensional understanding of reinforcement. Commodity purchase tasks are an efficient method for measuring demand in human participants. One challenge in translating these procedures to electronic nicotine delivery systems (ENDS or e-cigarettes) is defining commodity units given the lack of standardization in the e-cigarette marketplace. Aims and Methods The purpose of this study was to directly compare methods of operationalizinge-cigarette purchases, puffs, cartridges, and mLs liquid, using a within-subject design. Participants (N = 132) reporting past week e-cigarette use were recruited using crowdsourcing. Purchase tasks were completed operationalizing e-cigarette units as puffs or cartridges at baseline and puffs or mLs liquid at a 3-month follow-up. Results Bivariate associations supported convergent and discriminant validity with the largest effect size correlations for intensity and elasticity observed for the puff version. Interaction models suggested that product preferences moderated the relationship between time-to-first use and cartridge demand with larger effect size correlations among persons reporting a preference for JUULs, but weaker relationships among persons reporting other device preferences. Puff intensity (rxx = .61) and elasticity (rxx = .62) showed good test–retest reliability for participants reporting stable consumption, but poor test–retest reliability for individuals with changed consumption levels (intensity rxx = −.08; elasticity rxx = −.10). Conclusions This study highlights the relevance of commodity definitions in the e-cigarette purchase task. Puffs as an experimental commodity may provide flexibility for studying e-cigarette demand in heterogenous or unknown populations, whereas more tailored or personalized approaches like cartridge or mL-based tasks will likely be helpful when studying known subgroups. Implications The commodity purchase task procedure is widely used for understanding cigarette and e-cigarette demand in nicotine dependence research. This study evaluates the importance of operational definitions of e-cigarette commodities in the purchase task (ie, puffs, cartridges, or mLs liquid). Puffs may provide a more flexible commodity unit when evaluating e-cigarette demand in general or heterogenous populations, whereas device-specific units may prove more valuable when studying populations with consistent and known product use.

scholarly journals Assessing Obsessive-compulsive Symptoms: The Portuguese Version of the Padua Inventory

2017 ◽  
Vol 41 (S1) ◽  
pp. S70-S70
Author(s):  
A. Galhardo ◽  
J. Pinto-Gouveia ◽  
M. Cunha ◽  
I. Massano-Cardoso
Keyword(s):  
Psychometric Properties ◽  
Factor Structure ◽  
Internal Consistency ◽  
Discriminant Validity ◽  
Obsessive Compulsive ◽  
Retest Reliability ◽  
Obsessive Compulsive Symptoms ◽  
Portuguese Version ◽  
Convergent And Discriminant Validity ◽  
Test Retest Reliability

IntroductionSelf-report instruments are useful tools for the assessment of psychopathological symptoms such as obsessive-compulsive symptoms. The Padua Inventory (PI) is a measure that has been widely used in clinical and research settings and studied in several countries.ObjectivesThis study explores the psychometric properties and factor structure of the Portuguese version of the PI.MethodsTranslation and translation-back of the original version were executed. A total of 847 participants (468 women and 379 men) from the general population was recruited through a snowball procedure and completed the PI and other mental health measures. The data set was randomly split in order to conduct principal component analysis (PCA) and confirmatory factor (CFA) analysis in two different samples. Moreover, internal consistency, convergent and discriminant validity and test-retest reliability analyses were conducted.ResultsFrom the PCA analysis four factors emerged, comprising clusters of items related to doubting/impaired control over mental activities, contamination/washing/cleaning, checking and worries about losing control over motor behaviors. CFA results revealed that the model presented a poor fit to the data and indicated that the model would benefit from the establishment of correlations between pairs of error terms of items with identical contents. The PI exhibited excellent internal consistency, good test-retest reliability and good convergent and discriminant validity.ConclusionsThe PI Portuguese version showed a similar factor structure to the one presented in other studies and revealed good psychometric properties. Nevertheless, results from the CFA suggest that shortening the PI may be advantageous.Disclosure of interestThe authors have not supplied their declaration of competing interest.

scholarly journals The development and psychometric properties of a self-report Catastrophising Questionnaire

2020 ◽  
Author(s):  
Alexandra C Pike ◽  
Jade Serfaty ◽  
Oliver Joe Robinson
Keyword(s):  
Discriminant Validity ◽  
Self Report ◽  
Underlying Structure ◽  
Short Version ◽  
Good Test ◽  
Total N ◽  
Retest Reliability ◽  
Convergent And Discriminant Validity ◽  
Exploratory Factor Analyses ◽  
Test Retest Reliability

Catastrophising is a cognitive process that can be defined as predicting the worst possible outcome. It has been shown to be related to psychiatric diagnoses such as depression and anxiety, yet there are no self-report questionnaires specifically measuring it outside the context of pain research. Here, we therefore develop a novel, comprehensive self-report measure of general catastrophising. We performed five online studies (total n=734), in which we created and refined a Catastrophising Questionnaire, and used a factor analytic approach to understand its underlying structure. We also assessed convergent and discriminant validity, and analysed test-retest reliability. Furthermore, we tested the ability of Catastrophising Questionnaire scores to predict relevant clinical variables over and above other questionnaires. Finally, we also developed a four-item short version of this questionnaire. We found that our questionnaire is best fit by a single underlying factor, and shows convergent and discriminant validity. Exploratory factor analyses indicated that catastrophising is independent from other related constructs, including anxiety and worry. Moreover, we demonstrate incremental validity for this questionnaire in predicting diagnostic and medication status. Finally, we demonstrate that our Catastrophising Questionnaire has good test-retest reliability (Intra-Class-Correlation Coefficient=0.77, p&lt;.001). Critically, we can now, for the first time, obtain detailed self-report data on catastrophising.

Sign in / Sign up

Export Citation Format

Share Document