Leveraging the Prolific Platform for Substance Use Research: Recommendations for Repeated Assessment and Psychometric Studies

We present data from multiple samples demonstrating the Prolific platform’s potential for efficiently collecting data for substance use research involving repeated assessments, including for psychometric research requiring large samples. First, we present data from a 5-day daily diary protocol. We recruited a large sample (N = 321 at Day 1) screened for a history of self-identified mental health issues and weekly alcohol use. Participant adherence was good (82%) even without in-person contact. Alcohol use patterns conformed to theoretical expectations: Participants were more likely to drink on Fridays and Saturdays than other days, men drank more than women, and higher Alcohol Use Disorders Identification Test (AUDIT; Saunders et al., 1993) scores were associated with an increased likelihood of use and more overall drinking. Second, we present data from 429 Prolific participants screened for a history of mental health issues who completed assessments 2 weeks apart with strong retention (N = 377; 88%). We compare these data with data from undergraduates (N = 529) to demonstrate Prolific’s utility for conducting psychometrically-oriented substance use research. Internal consistency estimates for measures from the Prolific data matched or exceeded those from the undergraduate data. Furthermore, measure scores showed strong temporal stability, and factor structures (e.g., AUDIT item level structures) conformed to theoretical expectations. Collectively, findings indicate that Prolific can be used successfully for repeated measures data collection. We also provide recommendations for using Prolific to target participant recruitment for repeated measures studies, especially in the areas of psychometric and substance use research.

Psychometric Evaluation of a 10-Item Health Insurance Knowledge Scale

BackgroundCollege students are a priority population for health insurance literacy interventions. Yet, there are few psychometric studies on measuring health insurance knowledge – a core construct of health insurance literacy.MethodsWe administered a health insurance survey to 2,250 college students. We applied Classical Test Theory and Item Response Theory methods to estimate psychometric properties of the Kaiser Family Foundation's 10-item health insurance knowledge quiz.ResultsThe scale is unidimensional, and a two-parameter logistic model best fit the data. IRT estimates indicated varying item discriminations (a range: 0.717–2.578) and difficulties (b range: −0.913–1.790). Precision of measurement was maximized for students half a standard deviation below the mean (θ = −0.686) health insurance knowledge ability.ConclusionsThis scale can be used to identify gaps in health insurance knowledge among college students and be applied in clinical and community health education practice.

Sensitivity and Specificity of a Short Scale for Assessing Psychological Violence in Peruvian Health Professionals

Objective: Psychological violence at work is becoming more acute in the health sector due to the precariousness of psychosocial work conditions at a global scale. To date, there have been no psychometric studies to classify this situation. The objective of the study was to determine the sensitivity and specificity of the psychological violence scale in health professionals (PVS-Health) among the Peruvian population.Results: The study included 188 professionals from ten care centres in Peru. Two groups of 94 subjects were formed: subjects who had experienced psychological violence at work (PVW) and subjects who had not experienced. The average age was 36.8 ± 10.5 years; 59% of the sample were women. The subjects’ work experience ranged between 1 to 35 years. The analysis based on the receiver operating characteristic ( ROC ) curve concludes significantly: a) area under the curve, AUC x 0.974; standard error, SE x 0.10; p < 0.0003 (95% CI – 0.954 – 0.994), showing adequate randomness; b) cut-off point for maximum sensitivity (S x 0.94) and specificity (E x 0.89) was 35 out of 73 as the maximum score; and c) PVS-Health effectively distinguish subjects with PVW from those without PVW (89% with PVW, 94% of those without).

eHealth Literacy Instruments: a Systematic Review of Measurement Properties (Preprint)

BACKGROUND The internet is now a major source of health information. With the growth of internet users, electronic health literacy (eHealth literacy) has emerged as a new concept for digital health care. Health professionals need to consider the eHealth literacy of consumers when providing care utilizing digital health technologies. OBJECTIVE The objectives of this study were to identify currently available eHealth literacy instruments and evaluate their measurement properties with the intention of providing robust evidence to researchers and clinicians who are selecting an eHealth literacy instrument. METHODS A systematic review and meta-analysis was conducted of self-reported eHealth literacy instruments by applying the updated COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. RESULTS This study included 7 instruments from 41 articles describing 57 psychometric studies, as identified in 4 databases (PubMed, CINAHL, Embase, and PsycInfo). No eHealth literacy instrument provided evidence for all measurement properties. The eHealth Literacy Scale (eHEALS) was originally developed with a single-factor structure under the definition of eHealth literacy before the rise of social media and the mobile Web. That instrument was evaluated in 18 different languages in 26 countries involving diverse populations. However, a variety of other factor structures were exhibited: seven types of two-factor structure, three types of three-factor structure, and one-bifactor structure. The Transactional eHealth Literacy Instrument (TeHLI) was developed to reflect broader concept of eHealth literacy, and was demonstrated to have a sufficient low- and very-low-quality evidence for content validity (relevance, comprehensiveness, and comprehensibility), and sufficient high-quality evidence for structural validity and internal consistency; however, that instrument has rarely been evaluated. CONCLUSIONS The eHEALS was the most frequently investigated instrument. However, it is strongly recommended that the content of the instrument be updated to reflect recent advancements in digital health technologies. The TeHLI needs improvements in content validity and further psychometric studies to increase the credibility of its synthesized evidence. CLINICALTRIAL Not applicable.

Bifactor analysis of the rosenberg self-esteem scale in morbid obesity

Self-esteem is a set of feelings and thoughts a person has about his/her own worth and competence, which is reflected in a positive or negative attitude toward himself/herself. This study examined the factor structure of the Rosenberg Self-Esteem Scale with scores of 288 obese brazilian women waiting for bariatric surgery. Psychometric studies have found controversial results regarding the factor structure of this measure with samples from different profiles. In this investigation, the unifactor, two-oblique-factor and bifactor models were tested by Confirmatory Factor Analysis. The results clearly support the existence of a single dimension for the scale, but also for the method effects relating to the positively and negatively worded items. Further investigations with the inclusion of obese males are suggested.

Toward a General Factor of Disruptiveness: The Most Novel Creative Objects Tend to Be the Least Valuable and Feasible Ones

Talking about creative productions seems to be a common activity in both everyday language and the language used by researchers. The use of the term creative implies the existence of a creativity variable that allows for comparisons between different productions. According to the standard definition of creativity (Runco &amp; Jaeger, 2012), to be creative, a production must have both value and novelty. So far, empirical psychometric studies looking for a creativity variable with these two dimensions have shown that value and novelty are not only independent, but are also only weakly correlated. This empirical evidence, which has been widely replicated in the literature, indicates that, according to psychometric rules, it is impossible and indeed paradoxical to talk about the creativity of a production. In the present study, we sought to replicate these results by including a new dimension that has mostly been omitted in psychometric studies of creativity dimensions, namely feasibility. Results (N = 662 ideas) tended to show that this new dimension, negatively correlated with novelty and positively correlated with value, led to a second-order general factor of creativity. We named the axis formed by these three dimensions disruptiveness in order to underline the subtle difference from what would be an axis of creativity. The theoretical and applied implications of these results are discussed.

Measuring Forms and Functions of Aggression in Portuguese Young Adults: Validation of the Peer Conflict Scale

This work reports a preliminary validation of the Peer Conflict Scale (PCS) for Portuguese young adults (ages 18–30 years). This instrument assesses aggression considering two of its forms (overt and relational aggression) and its two functions (reactive and proactive aggression). The initially proposed 4-factor model provided the best fit for our data and was partially invariant by sex. All subscales revealed good reliability based on internal consistency and test–retest indicators. Construct validity was obtained through the investigation of sex differences that align with previous findings on aggressive behavior and in relation to emotion regulation strategies. These initial results suggest that the PCS, originally designed for adolescents, is a promising tool to assess aggression in young adults, notwithstanding the need of additional psychometric studies to further establish the quality of this instrument.

Psychometric properties of the Beck Anxiety Inventory and Snaith-Hamilton Pleasure Scale in Spanish-speaking countries during the COVID-19 pandemic

Psychometric studies are needed to decide the most suitable instrument in psychological research. Whereas Beck Depression Inventory has more psychometrics studies in Spanish-speaking countries, Beck Anxiety Inventory has less evidence. Additionally, anhedonia is a crucial construct to understand the relationship between depression and anxiety and their outcomes. However, there a scarce psychometric studies in anhedonia measures in non-English languages. Therefore, the first objective was to describe the Beck Anxiety Inventory and Snaith-Hamilton Pleasure Scale (SHAPS) psychometric properties. The second objective was to evaluate the measurement invariance of two psychometric scales, the Beck Anxiety Inventory (BAI) and the Snaith-Hamilton Pleasure Scale (SHAPS), in 12 Spanish-speaking countries. The first included 1287 participants over 18 years of age from Chile. The second with a sample of 9954 participants from 12 Spanish-speaking countries. We analyze the construct, convergent, concurrent, and discriminant validities using confirmatory factor analysis and structural equation modeling. This study showed good internal consistency for both scales. First, we provided evidence for construct validity, convergent validity, concurrent validity, and discriminant validity in a Chilean sample. Secondly, we established measurement invariance between Chile and those other 11 countries. This study represents the most thorough psychometric analysis of the BAI and SHAPS in the context of which we are aware.

Psychometric properties of measures of upper limb activity performance in adults with and without spasticity undergoing neurorehabilitation–A systematic review

Introduction This systematic review appraises the measurement quality of tools which assess activity and/or participation in adults with upper limb spasticity arising from neurological impairment, including methodological quality of the psychometric studies. Differences in the measurement quality of the tools for adults with a neurological impairment, but without upper limb spasticity, is also presented. Methods 29 measurement tools identified in a published review were appraised in this systematic review. For each identified tool, we searched 3 databases (Medline, Embase, CINAHL) to identify psychometric studies completed with neurorehabilitation samples. Methodological quality of instrument evaluations was assessed with use of the Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN) checklist. Synthesis of ratings allowed an overall rating of the psychometric evidence for each measurement tool to be calculated. Results 149 articles describing the development or evaluation of psychometric properties of 22 activity and/or participation measurement tools were included. Evidence specific to tool use for adults with spasticity was identified within only 15 of the 149 articles and provided evidence for 9 measurement tools only. Overall, COSMIN appraisal highlighted a lack of evidence of measurement quality. Synthesis of ratings demonstrated all measures had psychometric weaknesses or gaps in evidence (particularly for use of tools with adults with spasticity). Conclusions The systematic search, appraisal and synthesis revealed that currently there is insufficient measurement quality evidence to recommend one tool over another. Notwithstanding this conclusion, newer tools specifically designed for use with people with neurological conditions who have upper limb spasticity, have emergent measurement properties that warrant further research. Systematic review registration PROSPERO CRD42014013190.