Evaluating the effectiveness of a single-day simulation-based program in psychiatry for medical students: a controlled study

Abstract Background Training in psychiatry requires specific knowledge, attitudes, and skills that are obtainable by simulation, of which the use is only recent and still needs further development. Evidence is accumulating on its effectiveness but requires further validation for medical students. We aimed to evaluate the effectiveness of a single-day optional teaching program in psychiatry by simulation for medical students and validate a scale measuring Confidence in Psychiatric Clinical Skills (CPCQ), as part of the assessment. Methods This was a controlled study in a French University that compared (using paired-sample Student t-tests) knowledge and attitudes (university grades and CPCQ scores) before, just after teaching with simulated patients, and 2 months later. Satisfaction with the program (including the quality of the debriefing) was also investigated. The CPCQ scale was validated by assessing the factor structure, internal consistency, and test-retest reliability. Finally, a comparison was run with a control group who received the usual psychiatric instruction using covariance analyses. Results Twenty-four medical students were included in the simulation group and 76 in the control group. Just after the simulation, knowledge and attitudes increased significantly in the simulation group. Satisfaction with the training and debriefing was very high. The CPCQ scale showed good psychometric properties: a single-factor structure, acceptable internal consistency (α = 0.73 [0.65–0.85]), and good test-retest reliability (ICC = 0.71 [0.35–0.88]). Two months after the simulation, knowledge and attitudes were significantly higher in the simulation group than the control group, despite a lack of difference in knowledge before the simulation. Conclusions Adding a simulation program in psychiatry to the usual teaching improved the knowledge and confidence of medical students. The CPCQ scale could be used for the evaluation of educational programs.

Download Full-text

Effectiveness of a One-day Simulation-based Program in Psychiatry for Medical Students: A Controlled Study

10.21203/rs.3.rs-129706/v1 ◽

2020 ◽

Author(s):

Nadia YOUNES ◽

Anne Laure Delaunay ◽

Matthieu Roger ◽

Pierre Serra ◽

France Hirot ◽

...

Keyword(s):

Medical Students ◽

Factor Structure ◽

Internal Consistency ◽

Simulation Program ◽

Controlled Study ◽

Control Group ◽

Retest Reliability ◽

Group Satisfaction ◽

Simulation Group ◽

Test Retest Reliability

Abstract Background: Training in psychiatry requires specific knowledge, attitudes, and skills that are obtainable by simulation, which needs further development for medical students. After an analysis of previous research with medical students, we evaluated the effectiveness of a one-day teaching program in psychiatry by simulation and validated a scale measuring Confidence in Psychiatric Clinical Skills (CPCQ). Methods: The population was recruited during the 2019-2020 academic year among the 131 fifth-year undergraduate students at the French University of Versailles Saint-Quentin-en-Yvelines-Paris Saclay (the year of compulsory psychiatric training). A controlled study compared knowledge (university grades on the entire national program of psychiatry) and confidence (CPCQ scores) between a control group who received the usual psychiatric instruction and the simulation group who also participated in the simulation program. In the simulation group, satisfaction (including the quality of the debriefing) was investigated. The CPCQ scale was validated by assessing the factor structure, internal consistency, and test-retest reliability.Results: All fifth-year undergraduate medical students were included: 24 in the simulation group (voluntarily recruited) and 76 in the control group. Their knowledge did not differ before the simulation. After the simulation, knowledge and confidence increased significantly in the simulation group. Two months after the simulation, knowledge and confidence were significantly higher in the simulation group. Satisfaction with the training and debriefing was very high. The CPCQ scale showed good psychometric properties: a single-factor structure, acceptable internal consistency (α=0.73 [0.65 - 0.85]), and good test-retest reliability (ICC=0.71 [0.35 - 0.88]).Several limits were discussed.Conclusions: Adding a one-day simulation program in psychiatry to the usual teaching improved the knowledge and confidence of medical students even 2 months after. The CPCQ scale could be used for the evaluation of educational programs.

Download Full-text

Psychometric properties of the critical thinking disposition assessment test amongst medical students in China: a cross-sectional study

BMC Medical Education ◽

10.1186/s12909-020-02437-2 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Liyuan Cui ◽

Yaxin Zhu ◽

Jinglou Qu ◽

Liming Tie ◽

Ziqi Wang ◽

...

Keyword(s):

Factor Analysis ◽

Critical Thinking ◽

Medical Students ◽

Psychometric Properties ◽

Internal Consistency ◽

Discriminant Validity ◽

Retest Reliability ◽

Critical Thinking Disposition ◽

Thinking Disposition ◽

Test Retest Reliability

Abstract Background Critical thinking disposition helps medical students and professionals overcome the effects of personal values and beliefs when exercising clinical judgment. The lack of effective instruments to measure critical thinking disposition in medical students has become an obstacle for training and evaluating students in undergraduate programs in China. The aim of this study was to evaluate the psychometric properties of the CTDA test. Methods A total of 278 students participated in this study and responded to the CTDA test. Cronbach’s α coefficient, internal consistency, test-retest reliability, floor effects and ceiling effects were measured to assess the reliability of the questionnaire. Construct validity of the pre-specified three-domain structure of the CTDA was evaluated by explanatory factor analysis (EFA) and confirmatory factor analysis (CFA). The convergent validity and discriminant validity were also analyzed. Results Cronbach’s alpha coefficient for the entire questionnaire was calculated to be 0.92, all of the domains showed acceptable internal consistency (0.81–0.86), and the test-retest reliability indicated acceptable intra-class correlation coefficients (ICCs) (0.93, p < 0.01). The EFA and the CFA demonstrated that the three-domain model fitted the data adequately. The test showed satisfactory convergent and discriminant validity. Conclusions The CTDA is a reliable and valid questionnaire to evaluate the disposition of medical students towards critical thinking in China and can reasonably be applied in critical thinking programs and medical education research.

Download Full-text

Developing a measure to assess clinicians’ ability to reflect on key staff–patient dynamics in forensic settings

Journal of Forensic Practice ◽

10.1108/jfp-07-2021-0041 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Adam Polnay ◽

Helen Walker ◽

Christopher Gallacher

Keyword(s):

Reflective Practice ◽

Factor Structure ◽

Internal Consistency ◽

Self Report ◽

Face Validity ◽

Good Test ◽

Data Set ◽

Retest Reliability ◽

Content Type ◽

Test Retest Reliability

Purpose Relational dynamics between patients and staff in forensic settings can be complicated and demanding for both sides. Reflective practice groups (RPGs) bring clinicians together to reflect on these dynamics. To date, evaluation of RPGs has lacked quantitative focus and a suitable quantitative tool. Therefore, a self-report tool was designed. This paper aims to pilot The Relational Aspects of CarE (TRACE) scale with clinicians in a high-secure hospital and investigate its psychometric properties. Design/methodology/approach A multi-professional sample of 80 clinicians were recruited, completing TRACE and attitudes to personality disorder questionnaire (APDQ). Exploratory factor analysis (EFA) determined factor structure and internal consistency of TRACE. A subset was selected to measure test–retest reliability. TRACE was cross-validated against the APDQ. Findings EFA found five factors underlying the 20 TRACE items: “awareness of common responses,” “discussing and normalising feelings;” “utilising feelings,” “wish to care” and “awareness of complicated affects.” This factor structure is complex, but items clustered logically to key areas originally used to generate items. Internal consistency (α = 0.66, 95% confidence interval (CI) = 0.55–0.76) demonstrated borderline acceptability. TRACE demonstrated good test–retest reliability (intra-class correlation = 0.94, 95% CI = 0.78–0.98) and face validity. TRACE indicated a slight negative correlation with APDQ. A larger data set is needed to substantiate these preliminary findings. Practical implications Early indications suggested TRACE was valid and reliable, suitable to measure the effectiveness of reflective practice. Originality/value The TRACE was a distinctive measure that filled a methodological gap in the literature.

Download Full-text

Translation and Cross-Cultural Adaptation of the Aphasia Check List (ACL) for Farsi-Speaking Persons with Aphasia

Basic and Clinical Neuroscience Journal ◽

10.32598/bcn.2021.1906.1 ◽

2021 ◽

Author(s):

Ahmad Reza Khatoonabadi ◽

◽

Amin Modarres Zadeh ◽

Azar Mehri ◽

Elke Kalbe ◽

...

Keyword(s):

Internal Consistency ◽

Discriminant Validity ◽

Control Group ◽

Check List ◽

Time Saving ◽

Cross Sectional ◽

Retest Reliability ◽

Ceiling Effects ◽

Language And Cognition ◽

Test Retest Reliability

The Aphasia Check List (ACL) test is a comprehensive, time-saving tool for language evaluation in aphasia, including a cognitive assessment part. This cross-sectional study aimed to translate this test into Farsi and analyze the psychometric features of the translated version. The original version of the ACL was translated and adapted from German; its psychometric features were then determined. Twenty participants with aphasia (PWA) and 50 age- and education-matched, cognitively healthy controls participated. Possible floor and ceiling effects, discriminant validity, test-retest reliability, and internal consistency were analyzed in addition to the evaluation of internal correlations between the test parts (Language and Cognition). Regarding the performance of PWAs in the language section and the cognitive subtests assessing attention, memory, and reasoning, there were no floor and ceiling effects. Adequate discriminant validities for the language section of the test (i.e., total score: [Mann-Whitney U= 6.000, p<0.001]; diagnostic subtests scores: [U=3.000, p<0.001]; and each subtest individually) and for the attention subtest of the cognition section [U=16.500, p<0.001] were observed. There was no difference between the control group and the patient group in the subtests of memory [U=497.500, p=0.973] and reasoning [U=3.000, p=308]. The test-retest reliability was acceptable in all subtests (ICCagreement =0.573-0.984). The ACL-P test showed appropriate internal consistency (Cronbach’s alpha=0.761 for test and retest scores). There were also significant correlations between language and cognition in the control and patient groups. The ACL-P test showed sufficient reliability and validity for the evaluation of Farsi-speaking PWAs and used in studies on this population

Download Full-text

Patient Continuity of Care Questionnaire in a cardiac sample: A Confirmatory Factor Analysis

BMJ Open ◽

10.1136/bmjopen-2020-037129 ◽

2020 ◽

Vol 10 (7) ◽

pp. e037129

Author(s):

Emma Säfström ◽

Lena Nasstrom ◽

Maria Liljeroos ◽

Lena Nordgren ◽

Kristofer Årestedt ◽

...

Keyword(s):

Psychometric Properties ◽

Factor Structure ◽

Continuity Of Care ◽

Internal Consistency ◽

Intraclass Correlation ◽

Evaluation Study ◽

Sampling Procedure ◽

Retest Reliability ◽

Confirmatory Factor ◽

Test Retest Reliability

ObjectiveEven though continuity is essential after discharge, there is a lack of reliable questionnaires to measure and assess patients’ perceptions of continuity of care. The Patient Continuity of Care Questionnaire (PCCQ) addresses the period before and after discharge from hospital. However, previous studies show that the factor structure needs to be confirmed and validated in larger samples, and the aim of this study was to evaluate the psychometric properties of the PCCQ with focus on factor structure, internal consistency and stability.DesignA psychometric evaluation study. The questionnaire was translated into Swedish using a forward–backward technique and culturally adapted through cognitive interviews (n=12) and reviewed by researchers (n=8).SettingData were collected in four healthcare settings in two Swedish counties.ParticipantsA consecutive sampling procedure included 725 patients discharged after hospitalisation due to angina, acute myocardial infarction, heart failure or atrial fibrillation.MeasurementTo evaluate the factor structure, confirmatory factor analyses based on polychoric correlations were performed (n=721). Internal consistency was evaluated by ordinal alpha. Test–retest reliability (n=289) was assessed with intraclass correlation coefficient (ICC).ResultsThe original six-factor structure was overall confirmed, but minor refinements were required to reach satisfactory model fit. The standardised factor loadings ranged between 0.68 and 0.94, and ordinal alpha ranged between 0.82 and 0.95. All subscales demonstrated satisfactory test–retest reliability (ICC=0.76–0.94).ConclusionThe revised version of the PCCQ showed sound psychometric properties and is ready to be used to measure perceptions of continuity of care. High ordinal alpha in some subscales indicates that a shorter version of the questionnaire can be developed.

Download Full-text

Psychometric Properties of the Critical Thinking Disposition Assessment Test Amongst Medical Students in China: A Cross-Sectional Study

10.21203/rs.3.rs-16612/v2 ◽

2020 ◽

Author(s):

Liyuan Cui ◽

Yaxin Zhu ◽

Jinglou Qu ◽

Liming Tie ◽

Ziqi Wang ◽

...

Keyword(s):

Factor Analysis ◽

Critical Thinking ◽

Medical Students ◽

Psychometric Properties ◽

Internal Consistency ◽

Discriminant Validity ◽

Retest Reliability ◽

Critical Thinking Disposition ◽

Thinking Disposition ◽

Test Retest Reliability

Abstract Background: Critical thinking disposition helps medical students and professionals overcome the effects of personal values and beliefs when exercising clinical judgment. The lack of effective instruments to measure critical thinking disposition in medical students has become an obstacle for training and evaluating students in undergraduate programs in China. The aim of this study was to evaluate the psychometric properties of the CTDA test.Methods: A total of 278 students participated in this study and responded to the CTDA test. Cronbach’s a coefficient, internal consistency, test-retest reliability, floor effects and ceiling effects were measured to assess the reliability of the questionnaire. Construct validity of the pre-specified three-domain structure of the CTDA was evaluated by explanatory factor analysis (EFA) and confirmatory factor analysis (CFA). The convergent validity and discriminant validity were also analyzed.Results: Cronbach’s alpha coefficient for the entire questionnaire was calculated to be 0.92, all of the domains showed acceptable internal consistency (0.81-0.86), and the test-retest reliability indicated acceptable intra-class correlation coefficients (ICCs) (0.93, p<0.01). The EFA and the CFA demonstrated that the three-domain model fitted the data adequately. The test showed satisfactory convergent and discriminant validity.Conclusions: The CTDA is a reliable and valid questionnaire to evaluate the disposition of medical students towards critical thinking in China and can reasonably be applied in critical thinking programs and medical education research.

Download Full-text

Validation of the Opioid Overdose Knowledge (OOKS) and Attitudes (OOAS) Scales in French, Spanish, and German Languages, among a Sample of Opioid Users

European Addiction Research ◽

10.1159/000511770 ◽

2021 ◽

pp. 1-6

Author(s):

Stéphanie Laconi ◽

Raul Felipe Palma-Alvarez ◽

Heino Stöver ◽

Christina Padberg ◽

Daniela Jamin ◽

...

Keyword(s):

Internal Consistency ◽

Intraclass Correlation ◽

Opioid Overdose ◽

Care Providers ◽

Prevention Measures ◽

Retest Reliability ◽

Knowledge And Attitudes ◽

Back Translation ◽

Test Retest Reliability ◽

Opioid Users

Introduction: Improving the knowledge and attitudes of people facing an opioid overdose is one of the key prevention measures for reducing overdose occurrence and severity. In this respect, the Opioid Overdose Knowledge Scale (OOKS) and Opioid Overdose Attitudes Scale (OOAS) have been developed and validated in English to assess and improve knowledge and attitudes of opioid users and their families and care providers, in case of an overdose. Here, the OOKS and OOAS scales have been translated into French, Spanish, and German, and the different versions of the 2 scales have been assessed regarding their psychometric properties. Methods: The translation procedure of the scales was based on the international recommendations, including a back translation by a native English speaker. Subsequently, 80 (Spain: 29, France: 27, Germany: 24) former or current heroin users, aged from 20 to 61 years (M = 39.4 ± 9.23), completed the OOKS and OOAS versions of their native language, in test-retest, without specific between-assessment training. Internal consistency was assessed using Cronbach’s α, while test-retest reliability was assessed using intraclass correlation coefficient (ICC). The correlation between the OOKS and OOAS scores of a same language was assessed using Spearman’s (ρ) coefficient. Results: Internal consistency of the OOKS was found to be good to very good, with Cronbach’s α ranging from 0.62 to 0.87. Test-retest reliability was also very good, with ICCs ranging from 0.71 to 0.82. However, results were less reliable for the OOAS, as internal consistency was questionable to acceptable, with Cronbach’s α ranging from 0.12 to 0.63, while test-retest ICCs were very good for the French (0.91) and Spanish (0.99) versions and barely acceptable for the German version (0.41). No significant correlation was found between the OOKS and OOAS scores, irrespective of the version concerned. Conclusion: While satisfactory results were found for the 3 versions of the OOKS, results on the OOAS were relatively inconsistent, suggesting a possible gap between knowledge and attitudes on overdose among opioid users.

Download Full-text

Arabic Translation of the Weight Self-Stigma Questionnaire: Instrument Validation Study of Factor Structure and Reliability (Preprint)

10.2196/preprints.24169 ◽

2020 ◽

Author(s):

Nasser F BinDhim ◽

Nora A Althumiri ◽

Mada H Basyouni ◽

Omar T Sims ◽

Noara Alhusseini ◽

...

Keyword(s):

Factor Structure ◽

Internal Consistency ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Arabic Version ◽

Cross Sectional ◽

Retest Reliability ◽

Two Factors ◽

Enacted Stigma ◽

Test Retest Reliability

BACKGROUND While it is most often associated with its effects on physical health, obesity is also associated with serious self-stigmatization. The lack of a suitable, validated tool to measure weight-related self-stigma in Arabic countries may be partly responsible for the scarcity of literature about this problem. OBJECTIVE This study investigated the reliability and validity of an Arabic version of the Weight Self-Stigma Questionnaire (WSSQ). METHODS Data on the Arabic-translated version of the 12-item WSSQ were collected using two cross-sectional electronic questionnaires distributed among Saudi nationals through the Sharik Association for Health Research’s database in June 2020. Internal consistency, test-retest reliability, and exploratory factor analysis of the Arabic WSSQ were assessed and compared with the original English version and other translations. RESULTS For reliability analysis, 43 participants completed the Arabic WSSQ during two time periods. Internal consistency was α=.898 for the overall survey, α=.819 for the fear of enacted stigma subscale (factor 1), and α=.847 for the self-devaluation subscale (factor 2). The test-retest reliability of the intraclass correlation coefﬁcient was α=.982. In the factor structure analysis, 295 participants completed the questionnaire. The Arabic WSSQ loading of the items was consistent with the original WSSQ, except for the loading of item 9, which was stronger in factor 2 than in factor 1. The two factors accounted for the observed variances of 47.7% and 10.6%. CONCLUSIONS The Arabic version of the WSSQ has good internal consistency and reliability, and the factorial structure is similar to that of the original WSSQ. The Arabic WSSQ is adaptable for clinicians seeking to assess weight-related self-stigma in Arabic-speaking people.

Download Full-text

The Duke University Religion Index (DUREL): Validation and Reliability of the Farsi Version

Psychological Reports ◽

10.2466/08.07.17.pr0.112.1.151-159 ◽

2013 ◽

Vol 112 (1) ◽

pp. 151-159 ◽

Cited By ~ 20

Author(s):

Sina Hafizi ◽

Amir Hossein Memari ◽

Mohammad Pakrah ◽

Farnam Mohebi ◽

Amene Saghazadeh ◽

...

Keyword(s):

Medical Students ◽

Internal Consistency ◽

Second Phase ◽

Intrinsic Religiosity ◽

Retest Reliability ◽

Duke University ◽

Good Internal Consistency ◽

Two Phases ◽

Test Retest Reliability ◽

Farsi Version

This study examined the validation and reliability of the Farsi version of the Duke University Religion Index (FDUREL), a brief measure designed to evaluate the primary dimensions of religiosity. The study was conducted in two phases. In the first phase, after translation of the original version of DUREL by using standard forward-backward translation, the FDUREL was administered to 427 medical students at different training levels. Reliability of the FDUREL was assessed by internal consistency and test-retest reliability. Principal components factor analysis was employed to assess the construct validity of the measure. In the second phase, 557 medical students were asked to fill out the FDUREL and Hoge Intrinsic Religiosity Scale to examine concurrent validity. The FDUREL was unidimensional and had good internal consistency and test-retest reliability. Results suggest that the FDUREL is a reliable and valid measure of religiosity in Farsispeaking populations.

Download Full-text

Factor Structure and Psychometric Properties of the Child Oral and Motor Proficiency Scale

Journal of Early Intervention ◽

10.1177/1053815119841092 ◽

2019 ◽

Vol 41 (4) ◽

pp. 283-299 ◽

Cited By ~ 1

Author(s):

Jinhee Park ◽

Britt Frisk Pados ◽

Suzanne M. Thoyre ◽

Hayley H. Estrem ◽

Cara McComish

Keyword(s):

Psychometric Properties ◽

Factor Structure ◽

Motor Skills ◽

Internal Consistency ◽

Movement Patterns ◽

Internal Consistency Reliability ◽

Motor Proficiency ◽

Retest Reliability ◽

Oral Motor ◽

Test Retest Reliability

The purpose of this study was to identify the factor structure of the Child Oral and Motor Proficiency Scale (ChOMPS) and to evaluate the psychometric properties, including internal consistency reliability, test–retest reliability, and construct validity as measured by convergent and known-groups validity. Principal component analysis with varimax rotation conducted on responses from 364 parents of children between 6 months and 7 years of age identified four subscales: complex movement patterns, basic movement patterns, oral-motor coordination, and fundamental oral-motor skills. Acceptable internal consistency reliability (Cronbach’s α = .97) and test–retest reliability ( r = .98) were found. Convergent and known-groups validity were supported by significant associations between the ChOMPS and validation measures, as well as significant differences in the ChOMPS scores between children with and without feeding problems. The ChOMPS is a 63-item parent-report measure of eating, drinking, and related motor skills in children aged 6 months to 7 years with evidence of reliability and validity.

Download Full-text