Validation of the research capacity and culture (RCC) tool: measuring RCC at individual, team and organisation levels

2012 ◽  
Vol 18 (1) ◽  
pp. 62 ◽  
Author(s):  
Libby Holden ◽  
Susan Pager ◽  
Xanthe Golenko ◽  
Robert S. Ware

Research capacity building (RCB) in Australia has recently focussed on strategies that take a whole of system approach to developing research culture at individual, team and organisation levels. Although a theoretical framework exists, no tool has been published that quantitatively measures the effectiveness of RCB interventions aimed at these three levels. A sample of 134 allied health workers was used to validate the research capacity and culture (RCC) tool. Item level analysis was undertaken using Cronbach’s α and exploratory factor analysis, and test–retest reliability was examined using intra-class correlations (ICC). The tool had one factor emerge for each domain, with excellent internal consistency for organisation, team and individual domains (α = 0.95, 0.96 and 0.96 respectively; and factor loadings ranges of 0.58–0.89, 0.65–0.89 and 0.59–0.93 respectively). The overall mean score (total) for each domain was: 5.4 (inter-quartile range 3.9–7.7), 4.4 (IQR 2.6–6.1) and 3.9 (IQR 2.9–6) for the organisation, team and individual domains respectively. Test–retest reliability was strong for each domain: organisation ICC = 0.77, team ICC = 0.83 and individual ICC = 0.82. The RCC tool has three domains measuring research capacity and culture at organisation, team and individual levels. It demonstrates excellent internal consistency and strong test–retest reliability.

2021 ◽  
Vol 12 ◽  
Author(s):  
Justin Mason ◽  
Sherrilene Classen ◽  
James Wersal ◽  
Virginia Sisiopiku

Fully automated vehicles (AVs) hold promise toward providing numerous societal benefits including reducing road fatalities. However, we are uncertain about how individuals’ perceptions will influence their ability to accept and adopt AVs. The 28-item Automated Vehicle User Perception Survey (AVUPS) is a visual analog scale that was previously constructed, with established face and content validity, to assess individuals’ perceptions of AVs. In this study, we examined construct validity, via exploratory factor analysis and subsequent Mokken scale analyses. Next, internal consistency was assessed via Cronbach’s alpha (α) and 2-week test–retest reliability was assessed via Spearman’s rho (ρ) and intraclass correlation coefficient (ICC). The Mokken scale analyses resulted in a refined 20-item AVUPS and three Mokken subscales assessing specific domains of adults’ perceptions of AVs: (a) Intention to use; (b) perceived barriers; and (c) well-being. The Mokken scale analysis showed that all item-coefficients of homogeneity (H) exceeded 0.3, indicating that the items reflect a single latent variable. The AVUPS indicated a strong Mokken scale (Hscale = 0.51) with excellent internal consistency (α = 0.95) and test–retest reliability (ρ = 0.76, ICC = 0.95). Similarly, the three Mokken subscales ranged from moderate to strong (range Hscale = 0.47–0.66) and had excellent internal consistency (range α = 0.84–0.94) and test–retest reliability (range ICC = 0.84–0.93). The AVUPS and three Mokken subscales of AV acceptance were validated in a moderate sample size (N = 312) of adults living in the United States. Two-week test–retest reliability was established using a subset of Amazon Mechanical Turk participants (N = 84). The AVUPS, or any combination of the three subscales, can be used to validly and reliably assess adults’ perceptions before and after being exposed to AVs. The AVUPS can be used to quantify adults’ acceptance of fully AVs.


2021 ◽  
Vol 36 ◽  
pp. 153331752199532
Author(s):  
Jennifer R. Roberts ◽  
Molly Maxfield

A modified version of the Dementia Worry Scale (DWS) used the terminology “Alzheimer’s disease and related dementias” (versus the DWS’ use of “dementia”). Two studies investigated psychometric properties of the modified DWS (MDWS). Study 1 compared the psychometric properties of the DWS and MDWS; both versions had single factor structures and exhibited excellent internal consistency (αs ≥ .95). The MDWS exhibited greater test-retest reliability after a 4-week interval (DWS r = .68; MDWS r = .90). In Study 2, the MDWS again displayed a single factor structure, excellent internal consistency (α = .95), and good test-retest reliability after an 8-week interval (r = .78). Additionally, results support convergent validity between the MDWS and fear of dementia, subjective memory, general anxiety, health anxiety, and neuroticism. The MDWS is psychometrically consistent with the DWS, maintains strong test-retest reliability, and is appropriate for use in cross-sectional and longitudinal research.


Author(s):  
Ali H. Alnahdi ◽  
Ali Albarrati

The aim of the current study was to examine the psychometric properties of the upper extremity functional index (UEFI) in patients with chronic obstructive pulmonary disease (COPD). Seventy patients with stable COPD completed the UEFI and St. George Respiratory Questionnaire (SGRQ) and performed lung function tests in the first testing session. They completed the UEFI and the Global Rating of Change Scale in the second session, which was within ten days of the first session. The UEFI floor and ceiling effects, internal consistency, test–retest reliability, measurement error, and construct validity were examined. The UEFI was found to have no floor and ceiling effects. The UEFI was also found to have an excellent internal consistency (Cronbach’s alpha = 0.955) and an excellent test–retest reliability (ICC2,1 = 0.91). Totals of 4.85 points and 11.32 points represent the scale’s standard error of measurement, and a minimal detectable change at 90% confidence was used. The UEFI scores showed a significant correlation with the SGRQ activity domain (r = −0.66, p < 0.001) and differed significantly between participants with severe disease and those with mild disease (p = 0.03). The UEFI had no floor or ceiling issues, an excellent internal consistency, a good test–retest reliability, and an acceptable measurement error. The UEFI also demonstrated evidence supporting its construct validity as a measure of upper extremity-related activity limitations in patients with COPD.


2019 ◽  
Vol 11 (6) ◽  
pp. 19 ◽  
Author(s):  
Thammanard Charernboon

OBJECTIVES: The study aimed to evaluate content validity, convergent validity, internal consistency and test-retest reliability of the Thai version of the Scale for the Assessment of Negative Symptoms (SANS-Thai). METHODS: The content validity of the SANS-Thai was assessed using four experts. The average-content validity index and item level content validity index were analyzed. The SANS-Thai and the Thai versions of the Addenbrooke&rsquo;s Cognitive Examination (ACE) were administered to 40 people with schizophrenia to examine convergent validity and internal consistency. Twenty participants took the second SANS-Thai assessment within four weeks to evaluate test-retest reliability. RESULTS: The results demonstrated that the SANS-Thai has excellent content validity with the average-content validity index of 0.94. The majority of the item level content validity index range from 0.75 to 1. The global and total SANS-Thai score moderately correlated with the ACE with the correlation coefficient of -0.48 (p = 0.002) and -0.49 (p=0.001), respectively. Internal consistency by the Cronbach alpha coefficient was 0.95. Test-retest reliabilities by intraclass correlation were 0.91 (p&lt;0.001) for global SANS-Thai and 0.9 (p&lt;0.001) for total SANS-Thai. The Bland-Altman plot demonstrated that only 5% of the participants fell outside the limits of agreement for both global SANS and total SANS scores. CONCLUSION: The SANS-Thai appears to be a valid and reliable measure of negative symptoms in schizophrenia and could be useful for patient care and research studies.


2008 ◽  
Vol 24 (1) ◽  
pp. 219-224 ◽  
Author(s):  
Dóra Chor ◽  
Guilherme Loureiro Werneck ◽  
Eduardo Faerstein ◽  
Márcia Guimarães de Mello Alves ◽  
Lúcia Rotenberg

The effort-reward imbalance (ERI) model has been used to assess the health impact of job stress. We aimed at describing the cross-cultural adaptation of the ERI questionnaire into Portuguese and some psychometric properties, in particular internal consistency, test-retest reliability, and factorial structure. We developed a Brazilian version of the ERI using a back-translation method and tested its reliability. The test-retest reliability study was conducted with 111 health workers and University staff. The current analyses are based on 89 participants, after exclusion of those with missing data. Reproducibility (interclass correlation coefficients) for the "effort", "'reward", and "'overcommitment"' dimensions of the scale was estimated at 0.76, 0.86, and 0.78, respectively. Internal consistency (Cronbach's alpha) estimates for these same dimensions were 0.68, 0.78, and 0.78, respectively. The exploratory factorial structure was fairly consistent with the model's theoretical components. We conclude that the results of this study represent the first evidence in favor of the application of the Brazilian Portuguese version of the ERI scale in health research in populations with similar socioeconomic characteristics.


2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Amin Kordi Yoosefinejad ◽  
Fatemeh Karjalian ◽  
Marzieh Momennasab ◽  
Shahrokh Ezzatzadegan Jahromi

Abstract Background Hemodialysis is considered a major therapeutic method for patients with chronic kidney disease. Pruritus is a common complaint of hemodialysis patients. The 5-D pruritus scale is amongst the most common tools to evaluate several dimensions of itch. Psychometric properties of the 5-D scale have not been evaluated in Persian speaking population with hemodialysis; hence, the objective of this study was to assess reliability and validity of the Persian version of the scale. Methods Ninety hemodialysis patients (men: 50, women: 40, mean age: 54.4 years) participated in this cross-sectional study. The final Persian version of 5-D scale was given to the participants. Tests Compared: One-third of the participants completed the scale twice within 3–7 days apart to evaluate test- retest reliability. Other psychometric properties including internal consistency, absolute reliability, convergent, discriminative and construct validity, floor/ceiling effects were also evaluated. Results The Persian 5-D scale has strong test-retest reliability (ICC= 0.98) and internal consistency (Cronbach’s alpha= 0.99). Standard error of measurement and minimal detectable change were 0.33 and 0.91, respectively. Regarding convergent validity, the scale had moderate correlation with numeric rating scale (r =0.67) and quality of life questionnaire related to itch (r = 0.59). Exploratory factor analysis revealed two factors within the scale. No floor or ceiling effect was found for the scale. Conclusion The Persian version of 5-D the itching scale is a brief instrument with acceptable reliability and validity. Therefore, the scale could be used by experts, nurses, and other health service providers to evaluate pruritus among Persian speaking hemodialysis patients.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Liyuan Cui ◽  
Yaxin Zhu ◽  
Jinglou Qu ◽  
Liming Tie ◽  
Ziqi Wang ◽  
...  

Abstract Background Critical thinking disposition helps medical students and professionals overcome the effects of personal values and beliefs when exercising clinical judgment. The lack of effective instruments to measure critical thinking disposition in medical students has become an obstacle for training and evaluating students in undergraduate programs in China. The aim of this study was to evaluate the psychometric properties of the CTDA test. Methods A total of 278 students participated in this study and responded to the CTDA test. Cronbach’s α coefficient, internal consistency, test-retest reliability, floor effects and ceiling effects were measured to assess the reliability of the questionnaire. Construct validity of the pre-specified three-domain structure of the CTDA was evaluated by explanatory factor analysis (EFA) and confirmatory factor analysis (CFA). The convergent validity and discriminant validity were also analyzed. Results Cronbach’s alpha coefficient for the entire questionnaire was calculated to be 0.92, all of the domains showed acceptable internal consistency (0.81–0.86), and the test-retest reliability indicated acceptable intra-class correlation coefficients (ICCs) (0.93, p < 0.01). The EFA and the CFA demonstrated that the three-domain model fitted the data adequately. The test showed satisfactory convergent and discriminant validity. Conclusions The CTDA is a reliable and valid questionnaire to evaluate the disposition of medical students towards critical thinking in China and can reasonably be applied in critical thinking programs and medical education research.


2021 ◽  
Author(s):  
Qi Zhang ◽  
Ke Zhang ◽  
Miao Li ◽  
Jiaxin Gu ◽  
Xintong Li ◽  
...  

Abstract Objectives To examine the validity and reliability of the Mandarin version of the Treatment Burden Questionnaire (TBQ) among stroke patients. Background Stroke patients need long-term management of symptoms and life situation, and treatment burden has recently emerged as a new concept that can influence the health outcomes during the rehabilitation process. Methods The convenience sampling method was used to recruit 187 cases of stroke patients in a tertiary grade hospital in Tianjin for a formal investigation. Item analysis, reliability and validity tests were carried out. The reliability test included internal consistency and test–retest reliability. And as well as content, structure and convergent validity were performed for the validity test. Results Of the 187 completed questionnaires, only 180 (96.3%) were suitable for analysis. According to the experts’ evaluation, the I-CVI of each item was from 0.833 to 1.000, and the S-CVI was 0.967. The exploratory factor analysis yielded three-factor components with a cumulative variation of 53.054%. Convergent validity was demonstrated using measures of Morisky’s Medication Adherence Scale 8 (r = –0.450, P &lt; 0.01). All correlations between items and global scores ranged from 0.403 to 0.638. Internal consistency reliability and test–retest reliability were found to be acceptable, as indicated by a Cronbach’s α of 0.824 and an intraclass correlation coefficient of 0.846, respectively. Conclusions The Mandarin TBQ had acceptable validity and reliability. The use of TBQ in the assessment of treatment burden of stroke survivor may benefit health resources allocation and provide tailor therapeutic interventions to construct minimally disruptive care.


2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Marco Monticone ◽  
Cristiano Sconza ◽  
Igor Portoghese ◽  
Tomohiko Nishigami ◽  
Benedict M. Wand ◽  
...  

Abstract Background and aim Growing attention is being given to utilising physical function measures to better understand and manage knee osteoarthritis (OA). The Fremantle Knee Awareness Questionnaire (FreKAQ), a self-reported measure of body-perception specific to the knee, has never been validated in Italian patients. The aims of this study were to culturally adapt and validate the Italian version of the FreKAQ (FreKAQ-I), to allow for its use with Italian-speaking patients with painful knee OA. Methods The FreKAQ-I was developed by means of forward–backward translation, a final review by an expert committee and a test of the pre-final version to evaluate its comprehensibility. The psychometric testing included: internal structural validity by Rasch analysis; construct validity by assessing hypotheses of FreKAQ correlations with the knee injury and osteoarthritis outcome score (KOOS), a pain intensity numerical rating scale (PI-NRS), the pain catastrophising scale (PCS), and the Hospital anxiety and depression score (HADS) (Pearson’s correlations); known-group validity by evaluating the ability of FreKAQ scores to discriminate between two groups of participants with different clinical profiles (Mann–Whitney U test); reliability by internal consistency (Cronbach’s alpha) and test–retest reliability (intraclass correlation coefficient, ICC2.1); and measurement error by calculating the minimum detectable change (MDC). Results It took one month to develop a consensus-based version of the FreKAQ-I. The questionnaire was administered to 102 subjects with painful knee OA and was well accepted. Internal structural validity confirmed the substantial unidimensionality of the FreKAQ-I: variance explained was 53.3%, the unexplained variance in the first contrast showed an eigenvalue of 1.8, and no local dependence was detected. Construct validity was good as all of the hypotheses were met; correlations: KOOS (rho = 0.38–0.51), PI-NRS (rho = 0.35–0.37), PCS (rho = 0.47) and HADS (Anxiety rho = 0.36; Depression rho = 0.43). Regarding known-groups validity, FreKAQ scores were significantly different between groups of participants demonstrating high and low levels of pain intensity, pain catastrophising, anxiety, depression and the four KOOS subscales (p ≤ 0.004). Internal consistency was acceptable (α = 0.74) and test–retest reliability was excellent (ICC = 0.92, CI 0.87–0.94). The MDC95 was 5.22 scale points. Conclusion The FreKAQ-I is unidimensional, reliable and valid in Italian patients with painful knee OA. Its use is recommended for clinical and research purposes.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Widjane Sheila Ferreira Goncalves ◽  
Rebecca Byrne ◽  
Pedro Israel Cabral de Lira ◽  
Marcelo Tavares Viana ◽  
Stewart G. Trost

Abstract Background Childhood obesity has increased remarkably in low and middle-income (LMIC) countries. Movement behaviors (physical activity, screen time, and sleep) are crucial in the development of overweight and obesity in young children. Yet, few studies have investigated the relationship between children’s movement behaviors and parenting practices because validated measures for use among families from LMIC are lacking. This study evaluated the psychometric properties of previously validated measures of young children’s physical activity, screen time, and sleep and parenting practices, translated and culturally adapted to Brazilian families. Methods A total of 78 parent-child dyads completed an interviewer-administered survey twice within 7 days. Child physical activity, sedentary time and sleep were concurrently measured using a wrist-worn accelerometer. Internal consistency and test-retest reliability was assessed using McDonald’s Omega and Intraclass Correlation Coefficients (ICC’s). Concurrent validity was evaluated by calculating Spearman correlations between parent reported child behaviors and accelerometer measured behaviors. Results Seventeen of the 19 parenting practices scales exhibited acceptable internal consistency reliability (Ω ≥ 0.70). Test-retest reliability ICC’s were acceptable and ranged from 0.82 - 0.99. Parent reported child physical activity was positively correlated with objectively measured total movement (rho= 0.29 - 0.46, p < .05) and energetic play (rho= 0.29 – 0.40, p < .05). Parent reported child screen time was positively correlated with objectively measured sedentary time; (rho = 0.26, p < .05), and inversely correlated with total movement (rho = - 0.39 – - 0.41, p < .05) and energetic play (rho = - 0.37 – - 0.41, p < .05). Parent reported night-time sleep duration was significantly correlated with accelerometer measured sleep duration on weekdays (rho = 0.29, p < .05), but not weekends. Conclusions Measurement tools to assess children’s movement behaviors and parenting practices, translated and culturally adapted for use in Brazilian families, exhibited acceptable evidence of concurrent validity, internal consistency, and test-retest reliability.


Sign in / Sign up

Export Citation Format

Share Document