scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Tools

2019 ◽  
Author(s):  
Suha Al Muhaissen ◽  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour
Keyword(s):  
Assessment Tool ◽  
Academic Programs ◽  
Assessment Tools ◽  
Item Format ◽  
Traditional Assessment ◽  
Cognitive Levels ◽  
Examination Quality ◽  
Standardized Examinations ◽  
Case Based

Abstract Background Examination is the traditional assessment tool, and are used to guide the improvement of academic programs. Accreditation committees’ emphasis on the implementation of standardized examinations. The aims of the current study are to evaluate the quality of assessment tools of sixth year PharmD students as a function of assessment item format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods. A total of 173 assessment items used in the examinations of sixth year rotations of PharmD program were included. Items were classified as case based or non-case based and as MCQs or open-ended. The psychometric characteristics of examinations were studied as a function of the level of the Bloom’s levels addressed by an item, item format, and number of choices in MCQs. Results. Items addressing intellectual and analysis skills were more difficult, while items associated with multiple cognitive levels were more discriminative. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the letter. Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as non-case based items. Conclusion. Item formats, structure, and number of options in MCQs, affected students’ performance and overall examination quality. The discrimination of items associated with multiple Bloom’s levels was significantly higher than those associated with a single level. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively.

scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items

2020 ◽  
Author(s):  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour ◽  
Suha Almuhaissen
Keyword(s):  
Psychometric Properties ◽  
Academic Programs ◽  
Assessment Tools ◽  
Pharmacy Students ◽  
Psychometric Characteristics ◽  
Traditional Assessment ◽  
Clinical Clerkships ◽  
Response Choices ◽  
Case Based

Abstract Background Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs. Results Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier. Open-ended questions were significantly more discriminative in comparison to MCQs as case based items while they were more discriminative as noncase based items. Conclusion Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.

scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items

2020 ◽  
Author(s):  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour ◽  
Suha Almuhaissen
Keyword(s):  
Psychometric Properties ◽  
Academic Programs ◽  
Assessment Tools ◽  
Pharmacy Students ◽  
Psychometric Characteristics ◽  
Traditional Assessment ◽  
Clinical Clerkships ◽  
Response Choices ◽  
Case Based

Abstract Background Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items . Methods. A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs. Results . Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as noncase based items. Conclusion. Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.

scholarly journals Assessing Urban Quality of Life: Developing the Criteria for Saudi Cities

2021 ◽  
Vol 7 ◽  
Author(s):  
Jamal Al-Qawasmi ◽  
Muhammad Saeed ◽  
Omar S. Asfour ◽  
Adel S. Aldosary
Keyword(s):  
Quality Of Life ◽  
Assessment Tool ◽  
Group Analysis ◽  
Urban Life ◽  
Assessment Tools ◽  
Urban Quality Of Life ◽  
Wide Range ◽  
Depth Analysis ◽  
Urban Quality

Urban quality of life (QOL) is a complex and multidimensional concept. A wide range of urban QOL assessment tools has been developed worldwide to measure and monitor the quality of urban life taking into account the particular conditions of cities/regions and the needs of their residents. This study aims to develop an urban QOL assessment tool appropriate for the context of Saudi Arabia (SA). For this purpose, this study developed and used a structured approach that consists of an in-depth analysis of 21 urban QOL assessment tools in use worldwide, combined with focus group analysis and feedback from a panel of experts. The results revealed that there is a lack of consensus among the existing tools regarding the usage of QOL indicators and domains, and that the majority of the tools demonstrate a lack of proper coverage of QOL subdomains. The results also show wide variations in the number of indicators used and that most of the examined tools are using objective measurable indicators. This study has identified 67 indicators distributed across 13 domains that constitute the core criteria of the proposed QOL assessment tool. The selected indicators and domains cover all the attributes of urban QOL and are evaluated by experts as important criteria to assess/measure QOL. Moreover, the results demonstrate the advantage of the developed framework and comprehensive list of criteria (CLC) as a structured and efficient approach to design better QOL assessment tools.

scholarly journals The Patchwork Text as Assessment Tool for Postgraduate Law Teaching in South Africa

Obiter ◽  
2020 ◽  
Vol 41 (2) ◽  
pp. 292-308
Author(s):  
JG Horn ◽  
L Van Niekerk
Keyword(s):  
Deep Learning ◽  
Alternative Assessment ◽  
Assessment Tool ◽  
Assessment Tools ◽  
Postgraduate Students ◽  
Traditional Assessment ◽  
Academic Knowledge ◽  
Assessment Approach ◽  
Problem Solvers ◽  
The Right

In the increasingly competitive higher education sphere, delivering graduates with a sound academic grounding in their discipline is no longer enough. Institutions of higher learning must yield lifelong learners who are employable and equipped with the practical skills required by the profession. To ensure this, the right assessment approach is key. While assessment has always been a crucial component of instruction, traditional assessment tools run the risk of being mere tools for certification, facilitating surface learning instead of deep learning. Assessment approaches need to be re-evaluated to strike a balance between encouraging deep learning and instilling proper academic knowledge in graduates. To contribute to such a re-evaluation of traditional assessment methods, this article reports on the introduction of the patchwork text (PWT) as an alternative assessment tool in postgraduate law teaching at the University of the Free State (UFS). After making the case for the move towards more authentic, alternative assessment techniques, the authors embark on a discussion of the main features of the PWT, as well as guidelines for drafting a PWT assessment. The focus then shifts to an overview of PWT implementation in other postgraduate modules, ending with a discussion of the authors’ experience introducing the PWT in their own teaching. Useful information about the authors’ approach is shared, including examples of formative assessment exercises used as part of the PWT, specifics regarding the portfolio of evidence of learning to be handed in, and an outline of the four “patches” making up the assessment. It is concluded that the PWT has proven to be a viable tool for assessing postgraduate students in certain law modules at the UFS. It has managed to promote deep learning, develop students into critical thinkers and problem-solvers, and compel them to continuously engage with the study material – all while achieving the intended learning outcomes. The PWT is therefore recommended to lecturers who seek to equip students with a macro-vision of their field of study, the ability to integrate and contextualise different areas of the discipline, and the skill to reflect critically on new, emerging developments in the field.

Development of a Simulation Scenario Evaluation Tool (SSET): modified Delphi study

2019 ◽  
Vol 6 (6) ◽  
pp. 344-350
Author(s):  
Jessica Hernandez ◽  
Alise Frallicciardi ◽  
Nur-Ain Nadir ◽  
M David Gothard ◽  
Rami A Ahmed
Keyword(s):  
Content Validity ◽  
Assessment Tool ◽  
Assessment Tools ◽  
Evaluation Instrument ◽  
Evaluation Tool ◽  
Delphi Approach ◽  
Scenario Evaluation ◽  
Simulation Scenario ◽  
Written Simulation

IntroductionOne critical aspect of successful simulation facilitation is development of written scenarios. However, there are no validated assessment tools dedicated to the evaluation of written simulation scenarios available. Our aim was to develop a tool to evaluate the quality of written simulation demonstrating content validity.MethodsA comprehensive literature search did not yield a validated assessment tool dedicated for the evaluation of written simulation scenarios. A subsequent search yielded six templates published for written simulation scenario design. From these templates, critical scenario elements were identified to create an evaluation instrument with six components of scenario quality with corresponding anchors and rating scale. Subsequently, a national group of simulation experts were engaged via survey methodology to rate the content of the proposed instrument. Ultimately, a modified two-round Delphi approach was implemented to demonstrate consensus of the final assessment tool.Results38 responses were obtained in round 1, while 22 complete responses were obtained in round 2. Round 1 kappa values ranged from 0.44 to 1.0, indicating moderate to almost perfect rater agreement for inclusion of the six proposed components. Kappa values specifically regarding scale and anchors ranged from 0 to 0.49. After revisions, there was a significant level of agreement (p<0.05) of all items of the proposed assessment tool in the second-round survey except for item 10. Of note, all initial respondents indicated that they had never evaluated written scenarios with an assessment tool.ConclusionsThe Simulation Scenario Evaluation Tool, developed using a national consensus of content experts, is an instrument demonstrating content validity that assesses the quality of written simulation scenarios. This tool provides a basis to guide structured feedback regarding the quality of written simulation scenarios.

scholarly journals Development of a Multi-Domain Assessment Tool for Quality Improvement Projects

2017 ◽  
Vol 9 (4) ◽  
pp. 473-478 ◽  
Author(s):  
Glenn Rosenbluth ◽  
Natalie J. Burman ◽  
Sumant R. Ranji ◽  
Christy K. Boscardin
Keyword(s):  
Medical Education ◽  
Quality Improvement ◽  
Assessment Tool ◽  
Medical Profession ◽  
Assessment Tools ◽  
Validity Evidence ◽  
Learner Engagement ◽  
Summative Feedback ◽  
User Friendly

ABSTRACT Background  Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. Objective  We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. Methods  After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. Results  The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. Conclusions  The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

scholarly journals Quality of life and symptom assessment in paraesophageal hernias: a systematic literature review of reporting standards

2021 ◽  
Vol 34 (7) ◽  
Author(s):  
Nikhil M Patel ◽  
Aiysha Puri ◽  
Viknesh Sounderajah ◽  
Lorenzo Ferri ◽  
Ewen Griffiths ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Literature Review ◽  
Systematic Literature Review ◽  
Symptom Severity ◽  
Assessment Tool ◽  
Assessment Tools ◽  
Life Questionnaire ◽  
Reporting Standards ◽  
Gi Symptoms

Summary Background Paraesophageal hernias (PEH) present with a range of symptoms affecting physical and mental health. This systematic review aims to assess the quality of reporting standards for patients with PEH, identify the most frequently used quality of life (QOL) and symptom severity assessment tools in PEH and to ascertain additional symptoms reported by these patients not captured by these tools. Methods A systematic literature review according to PRISMA protocols was carried out following a literature search of MEDLINE, Embase and Cochrane databases for studies published between January 1960 and May 2020. Published abstracts from conference proceedings were included. Data on QOL tools used and reported symptoms were extracted. Results This review included 220 studies reporting on 28 353 patients. A total of 46 different QOL and symptom severity tools were used across all studies, and 89 different symptoms were reported. The most frequently utilized QOL tool was the Gastro-Esophageal Reflux Disease-Health related quality of life questionnaire symptom severity instrument (47.7%), 57.2% of studies utilized more than 2 QOL tools and ‘dysphagia’ was the most frequently reported symptom, in 55.0% of studies. Notably, respiratory and cardiovascular symptoms, although less common than GI symptoms, were reported and included ‘dyspnea’ reported in 35 studies (15.9%). Conclusions There lacks a QOL assessment tool that captures the range of symptoms associated with PEH. Reporting standards for this cohort must be improved to compare patient outcomes before and after surgery. Further investigations must seek to develop a PEH specific tool, that encompasses the relative importance of symptoms when considering surgical intervention and assessing symptomatic improvement following surgery.

scholarly journals 28 A systematic review in quality of life of patients with meningiomas: Effort towards developing a disease-specific questionnaire

2018 ◽  
Vol 45 (S3) ◽  
pp. S5-S6
Author(s):  
A. Mansouri ◽  
V. Lam Shin Cheung ◽  
B. Karmur ◽  
J. Lam Shin Cheung ◽  
L. Hachem ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Systematic Review ◽  
Physical Functioning ◽  
Assessment Tool ◽  
Negative Impact ◽  
Well Being ◽  
Assessment Tools ◽  
Younger Patients ◽  
Variable Impact

BACKGROUND: Meningiomas are the most common primary benign brain tumors in adults. Given the extended life expectancy of most meningiomas, consideration of quality of life (QOL) is important when selecting the optimal management strategy. There is currently a dearth of meningioma-specific QOL tools in the literature. OBJECTIVE: In this systematic review, we analyze the prevailing themes and propose toward building a meningioma-specific QOL assessment tool. METHODS: A systematic search was conducted, and only original studies based on adult patients were considered. QOL tools used in the various studies were analyzed for identification of prevailing themes in the qualitative analysis. The quality of the studies was also assessed. RESULTS: Sixteen articles met all inclusion criteria. Fifteen different QOL assessment tools assessed social and physical functioning, psychological, and emotional well-being. Patient perceptions and support networks had a major impact on QOL scores. Surgery negatively affected social functioning in younger patients, while radiation therapy had a variable impact. Any intervention appeared to have a greater negative impact on physical functioning compared to observation. CONCLUSION: Younger patients with meningiomas appear to be more vulnerable within social and physical functioning domains. All of these findings must be interpreted with great caution due to great clinical heterogeneity, limited generalizability, and risk of bias. For meningioma patients, the ideal QOL questionnaire would present outcomes that can be easily measured, presented, and compared across studies. Existing scales can be the foundation upon which a comprehensive, standard, and simple meningioma-specific survey can be prospectively developed and validated.

A comparison of chemotherapy-induced peripheral neuropathy assessment tools in patients receiving neurotoxic chemotherapy.

2016 ◽  
Vol 34 (3_suppl) ◽  
pp. 132-132
Author(s):  
Sabrina Ramnarine ◽  
Linda J Williams ◽  
Patrick M. Dougherty ◽  
Marie T Fallon
Keyword(s):  
Peripheral Neuropathy ◽  
Assessment Tool ◽  
Correlation Coefficients ◽  
Assessment Tools ◽  
European Organization ◽  
Lung Cancer Patients ◽  
Highly Correlated ◽  
Md Anderson ◽  
Eortc Qlq

132 Background: Chemotherapy-induced peripheral neuropathy (CIPN) is a challenging toxicity with significant sequelae impacting prognosis and quality of life. Clinical presentation although predominantly sensory can include motor and autonomic dysfunction. As there is no standardised approach to the diagnosis and assessment of CIPN, characterisation of symptoms can be problematic. This study aims to investigate the relationship between various CIPN assessment tools in gynaecological, colorectal and lung cancer patients receiving neurotoxic chemotherapy. Methods: Patients with the above cancers receiving chemotherapy (platinum and/or taxanes) were assessed prospectively every 3 weeks for a cumulative total of 159 cycles. Prior to each cycle the following tools were utilised: European Organization for Research and Treatment of Cancer (EORTC) QLQ-CIPN20 questionnaire, MD Anderson Symptom Inventory (MDASI) and associated descriptors of sensory disturbance/pain. Spearman’s correlation coefficients were used to compare these tools with the most commonly measured CIPN diagnostic tool, the National Cancer Institute-Common Toxicity Criteria for Adverse Events (NCI-CTCAE). Results: The EORTC sensory, motor subscales and total scores were highly correlated as expected. MDASI, question 13 (severity of numbness or tingling at its worst) was most correlated with the EORTC sensory subscale (ρ = 0.76). The correlation between the descriptors and EORTC sensory improved (ρ = 0.64 to ρ = 0.71) when the analysis focused on only numbness or tingling. CTCAE grade was more highly correlated with EORTC sensory (ρ = 0.61) and descriptors (ρ = 0.76) compared with EORTC motor (ρ = 0.46). All correlations were statistically significant (p < 0.001). Conclusions: In this mixed cohort of patients, comparison of the various CIPN assessment tools during chemotherapy cycles revealed significant correlations. This suggests the tools are useful in capturing the predominance of sensory symptoms and characterising the sensation of numbness and tingling. Further evaluation of this cohort along with additional studies may provide insight regarding the optimum choice of assessment tool(s).

Performance Assessment

2003 ◽  
pp. 210-224
Author(s):  
Apiwan D. Born ◽  
Carol M. Jessup
Keyword(s):  
Performance Assessment ◽  
Classroom Environment ◽  
Assessment Tool ◽  
Online Courses ◽  
Virtual Classroom ◽  
Assessment Tools ◽  
Future Research ◽  
Online Instructors ◽  
Traditional Assessment ◽  
Emerging Trends

This chapter examines a performance assessment tool (i.e., rubric) that is increasingly being utilized in online courses. The concept of performance assessment in the virtual classroom environment is discussed, including the proposition that using traditional assessment tools alone is not sufficient. Issues related to rubrics are identified, and guidelines are presented for online instructors and course developers. Emerging trends and future research opportunities are also provided.

Sign in / Sign up

Export Citation Format

Share Document