The journey towards the best quality of MCQs test, a success story in a medical school

2019 ◽  
Vol 1 (1) ◽  
Author(s):  
Bahaeldin Hassan ◽  
Abdulaziz Alamri ◽  
Muhammed T. Ahmad ◽  
Muhammed Abid
Keyword(s):  
Medical School ◽  
Training Program ◽  
Peer Review ◽  
Clinical Biochemistry ◽  
Item Analysis ◽  
Multiple Choice ◽  
P Value ◽  
Dramatic Improvement ◽  
Staff Members

Multiple choice question (MCQ) test is the most popular assessment tool that used in medical schools, faculty training on construction of multiple choice questions and peer review are the main factors that contributing to have a good quality MCQ test. This study aimed to evaluate the effect of these two factors on the quality of MCQ test. An extensive training program was conducted for the staff members who were participated in teaching and assessment of undergraduate medical students of clinical biochemistry and psychiatric courses. Post-test item analysis was used to compare pre-training with post-training item analysis parameters. Chi-square test was applied, level of significance was 5%. Regarding difficulty index, it showed great improvement in number of questions which lie in acceptable range of difficulty from 16%, 25% before the training program to 61%,42% after the training program for psychiatry and clinical biochemistry courses respectively and this is   significantly different (P0.05). In psychiatry course the number of questions with negatively discrimination power had been decreased from 36% before training program to 10% after it (P value 0.05). No significant improvement noticed in biochemistry course regarding this aspect. Both courses were showed improvement in the function of distracters, the percentage of test items where all distracters were functioning was increased from 11% to 41% in psychiatry course (P value=0.0002), and from 12% to 46%, in biochemistry course (P value=0.0001). The two courses which included in the study showed increased reliability after the training program and peer review processes, but this improvement is not statistically significant. A dramatic improvement was observed in the quality of MCQs after peer review and training program. We recommend adopting training program on construction of MCQs to be as teaching license for all medical school staff members in Saudi Arabia.

scholarly journals Item Analysis of Multiple-Choice Questions at the Department of Community Medicine, Wah Medical College, Pakistan

2020 ◽  
Vol 1 (2) ◽  
pp. 4
Author(s):  
Musarat Ramzan ◽  
Shezadi Sabah Imran ◽  
Sana Bibi ◽  
Khola Waheed Khan ◽  
Imrana Maqsood
Keyword(s):  
Pearson Correlation ◽  
Item Analysis ◽  
Multiple Choice ◽  
Community Medicine ◽  
P Value ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Significant Difference ◽  
Difficulty Index

Objective: The objective of the study was to assess the quality of multiple-choice questions (MCQs) of three different assessments in the subject of Community Medicine by computing the difficulty index, discrimination index and reliability and to estimate the relationship between difficulty and discrimination indices. Study Design: Retrospective observational study. Place and Duration of Study: Department of Community Medicine at Wah Medical College from August to December 2018. Materials and Methods: Three sets of MCQs were included in the study. Mean and standard deviation of difficulty and discrimination indices were calculated and one-way analysis of variance and Kruskal Wallis test were applied on difficulty and discrimination indices. The association was determined by Pearson correlation and considered significant at p value of < 0.05. Results: The mean difficulty index of first term, second term and send-up examination were 41.5, 48.8 and 51.9 respectively. Mean discrimination indices were 0.28, 0.27 and 0.26 and reliability were 0.83, 0.81 and 0.79. In the study, 72% MCQs of the first term, 61.5 % of the second term and 63% of the send-up examinations were in the range 30-70% of difficulty. There was a significant difference in the difficulty indices of the three question papers. The correlation between discrimination and difficulty indices was curvilinear and positively correlated. Conclusion: It is concluded that all three question papers have acceptable reliability, more than 65% MCQs have acceptable difficulty index and about 69% have good discriminatory power.

scholarly journals Item analysis to identify quality multiple choice questions/items in an assessment in Pharmacology of II MBBS students in Guntur Medical College of Andhra Pradesh, India

2018 ◽  
Vol 7 (8) ◽  
pp. 1517
Author(s):  
Vijaya K. Suryadevara ◽  
Zaheda Bano
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Correct Choice ◽  
Assessment Tools ◽  
P Value ◽  
Choice Response ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Difficulty Index

Background: In medical education, multiple choice questions/Items are the more frequently used assessment tools to assess the knowledge abilities and skills of medical students, for being their objectivity, wide coverage in less time. However only the Quality Items gives a valid and reliable assessment. The quality of an Item is determined by difficulty index (DIF I), Discrimination Index (DI) and Distractor efficiency (DE). Aim of the study was to know the quality of Items in pharmacology by Item analysis and to develop a MCQs bank with quality Items.Methods: The present study was conducted on 150 II MBBS students of Guntur Medical College, AP, India. A class test containing 50 Items with 150 distractors from topic chemotherapy was conducted. Item with the correct choice/response was awarded with one mark and with the wrong choice zero marks, no negative marks. Each test Item was analysed with DIF I, DI and DE and the results were tabulated and tested statistically, with unpaired "t" test.Results: Mean DIF I, DI, DE values with standard deviations in the present study are 44.72+17.63%, 0.30+0.12%, 84.48+24.65 respectively. DIF I of 32 (64%) items was good to excellent range (31%-60%) 9 (18%) Items were easy (>61%) and 9(18%) Items were difficult (>30%). DI of 10 (20%) Items was good (0.15 to 0.24.) 29 (58%) Items were excellent with DI > 0.25 and 11 (22%) Items were poor with DI <0.15. Among 150 distractors, 127 (85%) were functional distractors (FDs) and 23 (15%) were non-functional distractors (NFDs). DE of 33 (66%) items with nil NFDs was 100%, for 12 (24%) Items with one NFD, was 66.6%, for 4 (8%) items with 2 NFDs was 33.3% and for 1 (2%) Item with 3NFDs DE was 0%. When unpaired "t" test was applied to the means of "difficult" and "easy" Items, 96.22+11.33% SD, 51.44+29.31% SD respectively, the p-value obtained was 0.00058, which was highly significant.Conclusions: The study showed that Item analysis is a valid tool to identify quality Items, which assess, the students’ knowledge abilities and discriminate different levels of performance abilities of students effectively.

scholarly journals Examination of the Quality of Multiple-choice Items on Classroom Tests

2011 ◽  
Vol 2 (2) ◽  
Author(s):  
David DiBattista ◽  
Laura Kurzawa
Keyword(s):  
Nous Avons ◽  
Test Scores ◽  
Item Analysis ◽  
Multiple Choice ◽  
Discriminatory Power ◽  
Multiple Choice Tests ◽  
Choice Tests ◽  
Discrimination Coefficient ◽  
Multiple Choice Items

Because multiple-choice testing is so widespread in higher education, we assessed the quality of items used on classroom tests by carrying out a statistical item analysis. We examined undergraduates’ responses to 1198 multiple-choice items on sixteen classroom tests in various disciplines. The mean item discrimination coefficient was +0.25, with more than 30% of items having unsatisfactory coefficients less than +0.20. Of the 3819 distractors, 45% were flawed either because less than 5% of examinees selected them or because their selection was positively rather than negatively correlated with test scores. In three tests, more than 40% of the items had an unsatisfactory discrimination coefficient, and in six tests, more than half of the distractors were flawed. Discriminatory power suffered dramatically when the selection of one or more distractors was positively correlated with test scores, but it was only minimally affected by the presence of distractors that were selected by less than 5% of examinees. Our findings indicate that there is considerable room for improvement in the quality of many multiple-choice tests. We suggest that instructors consider improving the quality of their multiple-choice tests by conducting an item analysis and by modifying distractors that impair the discriminatory power of items. Étant donné que les examens à choix multiple sont tellement généralisés dans l’enseignement supérieur, nous avons effectué une analyse statistique des items utilisés dans les examens en classe afin d’en évaluer la qualité. Nous avons analysé les réponses des étudiants de premier cycle à 1198 questions à choix multiples dans 16 examens effectués en classe dans diverses disciplines. Le coefficient moyen de discrimination de l’item était +0.25. Plus de 30 % des items avaient des coefficients insatisfaisants inférieurs à + 0.20. Sur les 3819 distracteurs, 45 % étaient imparfaits parce que moins de 5 % des étudiants les ont choisis ou à cause d’une corrélation négative plutôt que positive avec les résultats des examens. Dans trois examens, le coefficient de discrimination de plus de 40 % des items était insatisfaisant et dans six examens, plus de la moitié des distracteurs était imparfaits. Le pouvoir de discrimination était considérablement affecté en cas de corrélation positive entre un distracteur ou plus et les résultatsde l’examen, mais la présence de distracteurs choisis par moins de 5 % des étudiants avait une influence minime sur ce pouvoir. Nos résultats indiquent que les examens à choix multiple peuvent être considérablement améliorés. Nous suggérons que les enseignants procèdent à une analyse des items et modifient les distracteurs qui compromettent le pouvoir de discrimination des items.

scholarly journals Quality of multiple choice questions in undergraduate pharmacology assessments in a teaching hospital of Kerala, India: an item analysis

2017 ◽  
Vol 6 (6) ◽  
pp. 1265
Author(s):  
Manju K. Nair ◽  
Dawnji S. R.
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Discrimination Index ◽  
Response Type ◽  
Multiple Choice Questions ◽  
Microsoft Excel ◽  
Best Response ◽  
Difficulty Index ◽  
Upper Third

Background: Carefully constructed, high quality multiple choice questions can serve as effective tools to improve standard of teaching. This item analysis was performed to find the difficulty index, discrimination index and number of non functional distractors in single best response type questions.Methods: 40 single best response type questions with four options, each carrying one mark for the correct response, was taken for item analysis. There was no negative marking. The maximum marks was 40. Based on the scores, the evaluated answer scripts were arranged with the highest score on top and the least score at the bottom. Only the upper third and lower third were included. The response to each item was entered in Microsoft excel 2010. Difficulty index, Discrimination index and number of non functional distractors per item were calculated.Results: 40 multiple choice questions and 120 distractors were analysed in this study. 72.5% items were good with a difficulty index between 30%-70%. 25% items were difficult and 2.5% items were easy. 27.5% items showed excellent discrimination between high scoring and low scoring students. One item had a negative discrimination index (-0.1). There were 9 items with non functional distractors.Conclusions: This study emphasises the need for improving the quality of multiple choice questions. Hence repeated evaluation by item analysis and modification of non functional distractors may be performed to enhance standard of teaching in Pharmacology.

scholarly journals AN ANALYSIS OF STUDENTS READING FINAL EXAMINATION BY USING ITEM ANALYSIS PROGRAM ON ELEVENTH GRADE OF SMA NEGERI 8 MEDAN

2019 ◽  
Vol 1 (1) ◽  
pp. 13-19
Author(s):  
Dewi Manalu ◽  
Kammer Tuahman Sipayung ◽  
Febrika Dwi Lestari
Keyword(s):  
Quantitative Research ◽  
Item Analysis ◽  
Multiple Choice ◽  
Final Examination ◽  
Multiple Choice Questions ◽  
Discrimination Power ◽  
Reliability Level ◽  
The Subject ◽  
Program Version

The purpose of this study was to determine the quality of the reading final examination in SMA N8 Medan grade eleventh in terms of reliability, level difficulty, discrimination power and level of distractor. This research is qualitative-quantitative research. The subject of research are the grade XI of SMA N8 Medan. Data is analyzed by ANATES program version 4.0.9. The analysis shows that: (1) items of multiple choice question that can be said as valid are 14 items ( 56%), while the invalid items amounted to 11 items ( 44%). (2) Items mutiple choice question can said as reliable because it equal 0,90 so it can said reliable. (3) items of multiple choice questions that categorized easy are 3 items (12%), satisfactory category 7 ( 28%) difficult category 2 (8%) and the other calculate categorized very easy 3 items (12%) and very difficult 3 (12%). (4) items of multiple choice questions that categorized poor are 12 items (48%),average category are 2 items (8%), good items category 1 items (4%) and excellent items are 8 items (32%).

scholarly journals Study on item and test analysis of multiple choice questions amongst undergraduate medical students

2017 ◽  
Vol 4 (5) ◽  
pp. 1562
Author(s):  
Abhijeet S. Ingale ◽  
Purushottam A. Giri ◽  
Mohan K. Doibale
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Test Methods ◽  
Discrimination Index ◽  
Multiple Choice Questions ◽  
Undergraduate Medical Students ◽  
Test Quality ◽  
Test Analysis ◽  
Difficulty Index

Background: Item analysis is the process of collecting, summarizing and using information from students’ response to assess the quality of test items. However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. There is more to writing good MCQs than writing good questions. The objectives of the study was to assess the item and test quality of multiple choice questions and to deal with the learning difficulties of students, identify the low achievers in the test. Methods: The hundred MBBS students from Government medical college were examined. A test comprising of thirty MCQs was administered. All items were analysed for Difficulty Index, Discrimination Index and Distractor Efficiency. Data entered in MS Excel 2007 and SPSS 21 analysed with statistical test of significance. Results: Majority 80% items difficulty index is within acceptable range. 63% items showed excellent discrimination Index. Distractor efficiency was overall satisfactory. Conclusions: Multiple choice questions with average difficulty and also having high discriminating power with good distracter efficiency should be incorporated into student’s examination. 

scholarly journals Standardization of Minimally Invasive Tissue Sampling Specimen Collection and Pathology Training for the Child Health and Mortality Prevention Surveillance Network

2019 ◽  
Vol 69 (Supplement_4) ◽  
pp. S302-S310 ◽  
Author(s):  
Natalia Rakislova ◽  
Fabiola Fernandes ◽  
Lucilia Lovane ◽  
Luisa Jamisse ◽  
Paola Castillo ◽  
...  
Keyword(s):  
Child Health ◽  
Minimally Invasive ◽  
Training Program ◽  
Tissue Sampling ◽  
Surveillance Network ◽  
Staff Members ◽  
Specimen Collection ◽  
Death Investigation ◽  
Health And Mortality

Abstract Background Minimally invasive tissue sampling (MITS) is a simplified postmortem examination technique that has shown to be an adequate approach for cause of death investigation in low-resource settings. It requires relatively low level of infrastructures and can be performed by health professionals with no background in pathology. A training program has been developed for the Child Health and Mortality Prevention Surveillance (CHAMPS) network to guarantee standardization of specimen collection techniques, procedures, and laboratory methods. Methods The training program has included assessment of the site capacities and training on a standardized protocol of MITS sampling and histological processing. The project has also introduced a program of training for trainers for the personnel from Mozambique. To guarantee the adequacy of the procedure in each site, a trainer accompanied the local teams when the activities started. Training outcomes were assessed by evaluating the quality of the samples obtained and the quality of the slides produced locally. Results Between June 2016 and October 2018, the laboratories of 7 sites (Bangladesh, Ethiopia, Kenya, Mali, Mozambique, Sierra Leone, and South Africa) have been evaluated and upgraded. Training has been delivered to 63 staff members from all sites. More than 600 MITS procedures have been performed. The quantity of tissue obtained in the MITS by the local teams was sufficient or abundant in 73%, and 87% of the slides were considered as technically acceptable or excellent. Conclusions Satisfactory standardization of MITS and histology procedures has been achieved across all CHAMPS sites through organized capacity-building plans.

scholarly journals Evaluation of an Internal Review Process for Grants And Manuscripts in the Canadian Critical Care Trials Group

2014 ◽  
Vol 21 (5) ◽  
pp. 283-286
Author(s):  
Karen EA Burns ◽  
Elaine Caon ◽  
Peter M Dodek
Keyword(s):  
Critical Care ◽  
Peer Review ◽  
Review Process ◽  
Multiple Choice ◽  
Likert Scale ◽  
Turnaround Time ◽  
Free Text ◽  
Internal Review ◽  
Present Evaluation

RATIONALE AND OBJECTIVES: All grants and manuscripts bearing the Canadian Critical Care Trials Group name are submitted for internal peer review before submission. The authors sought to formally evaluate authors’ and reviewers’ perceptions of this process.METHODS: The authors developed, tested and administered two electronic nine-item questionnaires for authors and two electronic 13-item questionnaires for reviewers. Likert scale, multiple choice and free-text responses were used.RESULTS: Twenty-one of 29 (72%) grant authors and 16 of 22 (73%) manuscript authors responded. Most author respondents were somewhat or very satisfied with the turnaround time, quality of the review and the review process. Two-thirds of grant (13 of 20 [65%]) and manuscript authors (11 of 16 [69%]) reported one or more successful submissions after review. Changes made to grants based on reviews were predominantly editorial and involved the background, rationale, significance/relevance and the methods/protocol sections. Twenty-one of 47 (45%) grant reviewers and 32 of 44 (73%) manuscript reviewers responded. Most reviewer respondents reported a good to excellent overall impression of the review process, good fit between their expertise and interests and the grants reviewed, and ample time to review. Although most respondents agreed with the current nonblinded review process, more grant than manuscript reviewers preferred a structured review format.CONCLUSIONS: The authors report a highly favourable evaluation of an existing internal review process. The present evaluation has assisted in understanding and improving the current internal review process.

scholarly journals Multiple-Choice Questions in Preclinical and Clinical Phase Examination for Undergraduate Medical Program in a Malaysian Medical School: A 2-Year Analysis

2020 ◽  
Author(s):  
Siti Khadijah Adam ◽  
Faridah Idris ◽  
Puteri Shanaz Jahn Kassim ◽  
Nor Fadhlina Zakaria ◽  
Rafidah Hod
Keyword(s):  
Clinical Examination ◽  
Item Analysis ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Discrimination Power ◽  
Clinical Phase ◽  
Preclinical Phase ◽  
Medical Program ◽  
Continuous Feedback

Abstract Background: Multiple-choice questions (MCQs) are used in measuring the student’s progress and post-examination analysis is usually done to guarantee the item’s appropriateness for question banking. Item analysis is a simple and effective method to determine the quality of MCQs by using three parameters; difficulty or passing index (PI), discrimination index (DI) and distractor efficiency (DE). Methods: This study analysed the MCQs in the preclinical and clinical examinations in Doctor of Medicine program of Universiti Putra Malaysia. Forty MCQs consisted of four options each in the preclinical examination and 80 MCQs with five options from the clinical examination paper in 2017 and 2018 were analysed and compared. Results: The mean DI was similar in all examinations, except a significant reduction in 2018 clinical examination. From 2017 and 2018, preclinical MCQs showed an increment in the number of ‘excellent’ and ‘good’ items. However, clinical papers showed reduction in DI due to high number of ‘poor’ questions. Comparing both years, there was an increase in the number of items with no non-functioning distractors in both examinations. Among all, preclinical MCQs in 2018 showed the highest mean of DE as compared to the others.Conclusion: Our findings suggested that item authors from preclinical phase showed an improvement in constructing good quality MCQs while clinical phase authors may need more training and continuous feedback. Higher number of options did not affect the level of difficulty of a question however the discrimination power and effectiveness of distractors might differ.

Sign in / Sign up

Export Citation Format

Share Document