scholarly journals Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study (Preprint)

2020 ◽  
Author(s):  
Saskia Marion Kelders ◽  
Hanneke Kip ◽  
Japie Greeff
Keyword(s):  
Behavior Change ◽  
Psychometric Properties ◽  
Predictive Validity ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Digital Health ◽  
Internal Consistency Reliability ◽  
Divergent Validity ◽  
Time Points ◽  
Perceived Behavior

BACKGROUND Engagement emerges as a predictor for the effectiveness of digital health interventions. However, a shared understanding of engagement is missing. Therefore, a new scale has been developed that proposes a clear definition and creates a tool to measure it. The TWente Engagement with Ehealth Technologies Scale (TWEETS) is based on a systematic review and interviews with engaged health app users. It defines engagement as a combination of behavior, cognition, and affect. OBJECTIVE This paper aims to evaluate the psychometric properties of the TWEETS. In addition, a comparison is made with the experiential part of the Digital Behavior Change Intervention Engagement Scale (DBCI-ES-Ex), a scale that showed some issues in previous psychometric analyses. METHODS In this study, 288 participants were asked to use any step counter app on their smartphones for 2 weeks. They completed online questionnaires at 4 time points: T0=baseline, T1=after 1 day, T2=1 week, and T3=2 weeks. At T0, demographics and personality (conscientiousness and intellect/imagination) were assessed; at T1-T3, engagement, involvement, enjoyment, subjective usage, and perceived behavior change were included as measures that are theoretically related to our definition of engagement. Analyses focused on internal consistency, reliability, and the convergent, divergent, and predictive validity of both engagement scales. Convergent validity was assessed by correlating the engagement scales with involvement, enjoyment, and subjective usage; divergent validity was assessed by correlating the engagement scales with personality; and predictive validity was assessed by regression analyses using engagement to predict perceived behavior change at later time points. RESULTS The Cronbach alpha values of the TWEETS were .86, .86, and .87 on T1, T2, and T3, respectively. Exploratory factor analyses indicated that a 1-factor structure best fits the data. The TWEETS is moderately to strongly correlated with involvement and enjoyment (theoretically related to cognitive and affective engagement, respectively; <i>P</i>&lt;.001). Correlations between the TWEETS and frequency of use were nonsignificant or small, and differences between adherers and nonadherers on the TWEETS were significant (<i>P</i>&lt;.001). Correlations between personality and the TWEETS were nonsignificant. The TWEETS at T1 was predictive of perceived behavior change at T3, with an explained variance of 16%. The psychometric properties of the TWEETS and the DBCI-ES-Ex seemed comparable in some aspects (eg, internal consistency), and in other aspects, the TWEETS seemed somewhat superior (divergent and predictive validity). CONCLUSIONS The TWEETS performs quite well as an engagement measure with high internal consistency, reasonable test-retest reliability and convergent validity, good divergent validity, and reasonable predictive validity. As the psychometric quality of a scale is a reflection of how closely a scale matches the conceptualization of a concept, this paper is also an attempt to conceptualize and define engagement as a unique concept, providing a first step toward an acceptable standard of defining and measuring engagement.

scholarly journals Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study

10.2196/17757 ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. e17757
Author(s):  
Saskia Marion Kelders ◽  
Hanneke Kip ◽  
Japie Greeff
Keyword(s):  
Behavior Change ◽  
Psychometric Properties ◽  
Predictive Validity ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Digital Health ◽  
Internal Consistency Reliability ◽  
Divergent Validity ◽  
Time Points ◽  
Perceived Behavior

Background Engagement emerges as a predictor for the effectiveness of digital health interventions. However, a shared understanding of engagement is missing. Therefore, a new scale has been developed that proposes a clear definition and creates a tool to measure it. The TWente Engagement with Ehealth Technologies Scale (TWEETS) is based on a systematic review and interviews with engaged health app users. It defines engagement as a combination of behavior, cognition, and affect. Objective This paper aims to evaluate the psychometric properties of the TWEETS. In addition, a comparison is made with the experiential part of the Digital Behavior Change Intervention Engagement Scale (DBCI-ES-Ex), a scale that showed some issues in previous psychometric analyses. Methods In this study, 288 participants were asked to use any step counter app on their smartphones for 2 weeks. They completed online questionnaires at 4 time points: T0=baseline, T1=after 1 day, T2=1 week, and T3=2 weeks. At T0, demographics and personality (conscientiousness and intellect/imagination) were assessed; at T1-T3, engagement, involvement, enjoyment, subjective usage, and perceived behavior change were included as measures that are theoretically related to our definition of engagement. Analyses focused on internal consistency, reliability, and the convergent, divergent, and predictive validity of both engagement scales. Convergent validity was assessed by correlating the engagement scales with involvement, enjoyment, and subjective usage; divergent validity was assessed by correlating the engagement scales with personality; and predictive validity was assessed by regression analyses using engagement to predict perceived behavior change at later time points. Results The Cronbach alpha values of the TWEETS were .86, .86, and .87 on T1, T2, and T3, respectively. Exploratory factor analyses indicated that a 1-factor structure best fits the data. The TWEETS is moderately to strongly correlated with involvement and enjoyment (theoretically related to cognitive and affective engagement, respectively; P<.001). Correlations between the TWEETS and frequency of use were nonsignificant or small, and differences between adherers and nonadherers on the TWEETS were significant (P<.001). Correlations between personality and the TWEETS were nonsignificant. The TWEETS at T1 was predictive of perceived behavior change at T3, with an explained variance of 16%. The psychometric properties of the TWEETS and the DBCI-ES-Ex seemed comparable in some aspects (eg, internal consistency), and in other aspects, the TWEETS seemed somewhat superior (divergent and predictive validity). Conclusions The TWEETS performs quite well as an engagement measure with high internal consistency, reasonable test-retest reliability and convergent validity, good divergent validity, and reasonable predictive validity. As the psychometric quality of a scale is a reflection of how closely a scale matches the conceptualization of a concept, this paper is also an attempt to conceptualize and define engagement as a unique concept, providing a first step toward an acceptable standard of defining and measuring engagement.

scholarly journals Psychometric properties of BREALD-30 for assessing adolescents’ oral health literacy

2019 ◽  
Vol 53 ◽  
pp. 53 ◽  
Author(s):  
Larissa Chaves Morais de Lima ◽  
Érick Tássio Barbosa Neves ◽  
Laio Da Costa Dutra ◽  
Ramon Targino Firmino ◽  
Luiza Jordânia Serafim de Araújo ◽  
...  
Keyword(s):  
Oral Health ◽  
Health Literacy ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Divergent Validity ◽  
Oral Health Literacy ◽  
Functional Literacy ◽  
Carious Lesions ◽  
Test Retest Reliability

OBJECTIVE: To evaluate the psychometric properties of the Brazilian Portuguese version of the Rapid Estimate of Adult Literacy in Dentistry (BREALD-30) administered to adolescents. METHODS: The study included 750 adolescents: 375 aged 12 years and 375 aged 15–19 years, attending public and private schools in Campina Grande, state of Paraíba, Brazil, in 2017. Reliability was measured based on internal consistency and test-retest reliability. Convergent validity was measured based on correlations between BREALD-30 and Functional Literacy Indicator scores. Divergent validity was measured by comparing BREALD-30 scores with sociodemographic variables. For predictive validity, the association between BREALD-30 scores and the presence of cavitated carious lesions was tested using a multiple logistic regression model. All statistical tests were performed with a significance level of 5%. RESULTS: BREALD-30 showed good internal consistency for the 12 year olds and 15 to19 year olds (Cronbach’s alpha = 0.871 and 0.834, respectively) and good test-retest reliability [intraclass correlation coefficient (ICC) = 0.898 and 0.974; kappa = 0.804 and 0.808, respectively]. Moreover, item-total correlation was satisfactory for all items. BREALD-30 had convergent validity with the Functional Literacy Indicator for 12 year olds (rs = 0.558, p < 0.001) and for 15 to 19 year olds (rs = 0.652, p < 0.001). Participants with higher oral health literacy levels who attended private schools (p < 0.001), belonged to economic classes A and B2 (p < 0.001), and who had parents with higher education levels (p < 0.001) were included, indicating the divergent validity of the BREALD-30. Participants with lower BREALD-30 scores were more likely to have cavitated carious lesions [12 year olds: odds ratio (OR) = 2.37; 95% confidence interval (95%CI): 1.48–3.80; 15 to 19 year olds: OR = 1.96; 95%CI 1.24–3.11]. CONCLUSIONS: BREALD-30 shows satisfactory psychometric properties for use on Brazilian adolescents and can be applied as a fast, simple, and reliable measure of oral health literacy.

scholarly journals Development and Evaluation of a Swedish Short Version of the Questionnaire About the Process of Recovery (QPR)

2019 ◽  
Vol 56 (2) ◽  
pp. 376-382
Author(s):  
Eklund Mona ◽  
Neil Sandra ◽  
Argentzell Elisabeth
Keyword(s):  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Psychiatric Symptoms ◽  
Internal Consistency Reliability ◽  
Short Version ◽  
Level Of Functioning ◽  
Ceiling Effects ◽  
Good Internal Consistency

Abstract The aim was to develop a short version of the Swedish Process of Recovery Questionnaire (QPR-Swe) for use with people with severe mental illness and to investigate its internal consistency, construct validity, known-groups validity and any floor or ceiling effects. Two independent samples were used, the first (N = 226) to develop the short version and the second (N = 266) to test its psychometric properties. A seven-item version was developed by selecting items based on item-total correlations. The QPR-Swe-7 showed good internal consistency reliability (α = 0.82). It showed moderate correlations with indicators of convergent validity (self-rated health, self-mastery and quality of life) and weak with those selected to test discriminant validity (psychiatric symptoms and level of functioning). QPR-Swe-7 differentiated between people receiving two different levels of housing support. No floor or ceiling effects were found. The QPR-Swe-7 had appropriate psychometric properties for use with people with a variety of mental disorders when a brief scale is warranted.

scholarly journals Psychometric properties of the Geriatric Anxiety Inventory (GAI) and its short-form (GAI-SF) in a clinical and non-clinical sample of older adults

2014 ◽  
Vol 27 (7) ◽  
pp. 1089-1097 ◽  
Author(s):  
Carly Johnco ◽  
Ashleigh Knight ◽  
Dusanka Tadic ◽  
Viviana M. Wuthrich
Keyword(s):  
Older Adults ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Short Form ◽  
Clinical Sample ◽  
Clinical Status ◽  
Control Sample ◽  
Clinical Samples ◽  
Divergent Validity

ABSTRACTBackground:The Geriatric Anxiety Inventory is a 20-item geriatric-specific measure of anxiety severity. While studies suggest good internal consistency and convergent validity, divergent validity from measures of depression are weak. Clinical cutoffs have been developed that vary across studies due to the small clinical samples used. A six-item short form (GAI-SF) has been developed, and while this scale is promising, the research assessing the psychometrics of this scale is limited.Methods:This study examined the psychometric properties of GAI and GAI-SF in a large sample of 197 clinical geriatric participants with a comorbid anxiety and unipolar mood disorder, and a non-clinical control sample (N = 59).Results:The internal consistency and convergent validity with other measures of anxiety was adequate for GAI and GAI-SF. Divergent validity from depressive symptoms was good in the clinical sample but weak in the total and non-clinical samples. Divergent validity from cognitive functioning was good in all samples. The one-factor structure was replicated for both measures. Receiver Operating Characteristic analyses indicated that the GAI is more accurate at identifying clinical status than the GAI-SF, although the sensitivity and specificity for the recommended cutoffs was adequate for both measures.Conclusions:Both GAI and GAI-SF show good psychometric properties for identifying geriatric anxiety. The GAI-SF may be a useful alternative screening measure for identifying anxiety in older adults.

The HEXACO-Middle School Inventory (MSI)

2020 ◽  
Vol 36 (4) ◽  
pp. 681-693 ◽  
Author(s):  
Ida Sergi ◽  
Augusto Gnisci ◽  
Vincenzo P. Senese ◽  
Marco Perugini
Keyword(s):  
Middle School ◽  
Factor Analysis ◽  
Predictive Validity ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Principal Component ◽  
Divergent Validity ◽  
Clear Cut ◽  
Convergent And Divergent Validity ◽  
Italian Children

Abstract. We developed and validated a novel measure, the 6-factor personality HEXACO-Middle School Inventory (MSI). We started with a pool of 16 items for each of the six dimensions of the HEXACO. In Study 1, we administered the HEXACO-MSI to 1,089 Italian children and the Observer version to their parents. Using principal component analyses (PCA) and extension factor analysis (EFA), we selected the best eight items for each dimension. Confirmatory factor analysis (CFA) confirmed the 6-factor dimensionality and its invariance. Internal consistency of each dimension was adequate. Convergent and divergent validity were successfully established with a version of the scale filled by parents. Convergent validity was also established with the Big Five Questionnaire – Children (BFQ-Children) whereas divergent validity was less clear-cut. Conscientiousness, Honesty-Humility, and eXtraversion demonstrated predictive validity of school marks (criterion validity). In Study 2 ( N = 317), we replicated dimensionality, internal consistency, and established test-retest reliability of each dimension in two measurements at a 1 month distance. The HEXACO-MSI showed a clear personality structure organized in six traits, and evidence of predictive validity of relevant school criteria particularly via Conscientiousness, Honesty-Humility, and eXtraversion.

scholarly journals Herth Hope Index: A Psychometric Evaluation Study within a Sample of Greek Patients with Cancer

2021 ◽  
Vol 0 ◽  
pp. 1-8
Author(s):  
Maria Nikoloudi ◽  
Eleni Tsilika ◽  
Efi Parpa ◽  
Sotiria Kostopoulou ◽  
Anastasios Tentolouris ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Performance Status ◽  
Intraclass Correlation ◽  
Eastern Cooperative Oncology Group ◽  
Depression Scale ◽  
Internal Consistency Reliability ◽  
Cross Sectional ◽  
Herth Hope Index

Objectives: This study aims to develop the Greek version of the Herth Hope Index (HHI) and assess its psychometric properties to a palliative care patient sample, using a cross-sectional design. Materials and Methods: The HHI was translated into Greek (HHI-Gr) using the ‘forward-backward’ procedure. It was administered to 130 eligible cancer patients, while for the stability of patients’ responses, 40 of these patients completed the HHI-Gr 3 days later. Along with the HHI-Gr, patients also completed the Hospital Anxiety and Depression Scale (HADS) and the Beck Hopelessness Scale (BHS). The HHI-Gr internal consistency reliability (Cronbach’s a), stability (intraclass correlation coefficient [ICC]), factor structure (factor analysis) and convergent validity (correlation with the HADS and the BHS questionnaires) were examined using the Eastern Cooperative Oncology Group (ECOG) performance status. Results: The HHI-Gr yielded a one-factor model and a Cronbach alpha (0.860) with excellent internal consistency reliability and stability ICC (>0.90). Satisfactory convergent validity was supported by the correlation analysis between the HHI-Gr and BHS (r = 0.718, P < 0.001). Overall test-retest reliability was satisfactory with a range between 0.77 and 0.96 (P < 0.001). Conclusion: These results demonstrate that the HHI-Gr is an instrument with satisfactory psychometric properties and is a valid research tool for the measurement of the levels of hope among Greek oncology patients.

scholarly journals Examining the dimensions of employee safety behavior: Validation from the microfinance sector of Pakistan

2021 ◽  
Vol 8 (2) ◽  
pp. 17-21
Author(s):  
Hira Rani Shaikh ◽  
◽  
Syed Mir Mohammad Shah ◽  
Keyword(s):  
Developing Countries ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Internal Consistency Reliability ◽  
Dynamic Phenomenon ◽  
Safety Behavior ◽  
Theoretical Rationale ◽  
Employee Safety

Employee safety behavior is a dynamic phenomenon that takes place in every organization where employee’s concerns are taken for granted. Organizations and their respective authority are in search of ways to reduce the magnitude of such behavior by counseling employees. Research scholars play an important role in understanding and developing employee safety behavior. In this regard, specifically for assessing the way and magnitude of employee safety behavior (ESB), researchers have developed a tool to measure it. For achieving such an objective, researchers theoretically proposed the indicators to measure employee safety behavior effectively. These behaviors were: Avoidance and aggression coded as SBAV (safety behavior for avoidance) and SBAG (safety behavior for aggression). The present study strengthens the theoretical rationale of previous studies and validated the psychometric properties of employee safety behavior in the Pakistani context. 400 employees from 11 branches situated in different regions were surveyed, and data was analyzed using SMART PLS 3.0–software prominent due to its methodological usefulness. Findings illustrated that instrument satisfaction met the criteria of internal consistency reliability, convergent validity, and discriminant validity for both ESB dimensions. Findings clearly demonstrated that the ESB scale is effective enough in measuring employee safety behavior in the microfinance sector of Sindh, Pakistan. Hence, Future researchers are recommended to use this tool in measuring employee safety behavior in developing countries, specifically in Pakistan.

scholarly journals Reliability and validity of the RS14 in orphaned and separated adolescents and youths in western Kenya

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0241699
Author(s):  
Sarah C. Sutherland ◽  
Harry S. Shannon ◽  
David Ayuku ◽  
David L. Streiner ◽  
Olli Saarela ◽  
...  
Keyword(s):  
Social Support ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Psychological Resilience ◽  
Western Kenya ◽  
Significant Difference ◽  
Confirmatory Factor

Purpose The 14-item Resilience Scale (RS14) is a tool designed to measure psychological resilience. It has been used effectively in diverse populations. However, its applicability is largely unknown for Sub-Saharan adolescent populations and completely unknown for orphaned and separated adolescents and youths (OSAY), a highly vulnerable population for whom resilience may be critical. This study assesses the RS14’s psychometric properties for OSAY in Uasin Gishu County, Kenya. Methods Survey responses from a representative sample of 1016 OSAY (51.3% female) aged 10–25 (mean = 16; SD = 3.5) living in institutional and home-based environments in Uasin Gishu County were analyzed. The RS14’s psychometric properties were assessed by examining internal consistency reliability, confirmatory factor analyses, and convergent validity using correlations between resilience and each of social support and depression. Sub-analyses were conducted by age and sex. Results Resilience scores ranged from 14–98 (mean = 66; SD = 19) with no sex-based significant difference. Resilience was higher for those aged ≥18 (mean = 69; range = 14–98) versus age <18 (mean = 65; range = 14–98). Internal consistency was good (Cronbach’s α = .90). Confirmatory factor analysis indicated a 1-factor solution, though the model fit was only moderate. Resilience was positively correlated with social support in all ages (.22; p < .001) and negatively correlated with depression in individuals age <18 (-.22; p < .001). The relationship between resilience and depression in individuals age ≥18 was statistically significant only in females (-.17; p = .026). Conclusion This study demonstrates reasonable evidence that the RS14 is both valid and reliable for measuring psychological resilience in the population of OSAY in western Kenya.

Psychometric Properties of the German Version of the Scale to Assess the Therapeutic Relationship in Community Mental Health Care (D-STAR)

2012 ◽  
Vol 28 (4) ◽  
pp. 255-261 ◽  
Author(s):  
Sabine Loos ◽  
Reinhold Kilian ◽  
Thomas Becker ◽  
Birgit Janssen ◽  
Harald Freyberger ◽  
...  
Keyword(s):  
Mental Health ◽  
Health Care ◽  
Community Mental Health ◽  
Mental Health Care ◽  
Psychometric Properties ◽  
Therapeutic Relationship ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Community Mental Health Care ◽  
German Version

Objective: There are presently no instruments available in German language to assess the therapeutic relationship in psychiatric care. This study validates the German version of the Scale to Assess the Therapeutic Relationship in Community Mental Health Care (D-STAR). Method: 460 persons with severe mental illness and 154 clinicians who had participated in a multicenter RCT testing a discharge planning intervention completed the D-STAR. Psychometric properties were established via item analysis, analyses of missing values, internal consistency, and confirmatory factor analysis. Furthermore, convergent validity was scrutinized via calculating correlations of the D-STAR scales with two measures of treatment satisfaction. Results: As in the original English version, fit indices of a 3-factor model of the therapeutic relationship were only moderate. However, the feasibility and internal consistency of the D-STAR was good, and correlations with other measures suggested reasonable convergent validity. Conclusions: The psychometric properties of the D-STAR are acceptable. Its use can be recommended in German-speaking countries to assess the therapeutic relationship in both routine care and research.

scholarly journals The Family Psychoeducation Fidelity Scale: Psychometric Properties

2020 ◽  
Vol 47 (6) ◽  
pp. 894-900 ◽  
Author(s):  
I. Joa ◽  
J. O. Johannessen ◽  
K. S. Heiervang ◽  
A. A. Sviland ◽  
H. A. Nordin ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Internal Consistency ◽  
Interrater Reliability ◽  
Review Process ◽  
Trial Registration ◽  
Family Psychoeducation ◽  
Time Points ◽  
The Family ◽  
Fidelity Scale

Abstract This study examined psychometric properties and feasibility of the Family Psychoeducation (FPE) Fidelity Scale. Fidelity assessors conducted reviews using the FPE fidelity scale four times over 18 months at five sites in Norway. After completing fidelity reviews, assessors rated feasibility of the fidelity review process. The FPE fidelity scale showed excellent interrater reliability (.99), interrater item agreement (88%), and internal consistency (mean = .84 across four time points). By the 18-month follow-up, all five sites increased fidelity and three reached adequate fidelity. Fidelity assessors rated feasibility as excellent. The FPE fidelity scale has good psychometric properties and is feasible for evaluating the implementation of FPE programs. Trial registration ClinicalTrials.gov Identifier: NCT03271242.

Sign in / Sign up

Export Citation Format

Share Document