scholarly journals Item Analysis of Multiple-Choice Questions at the Department of Community Medicine, Wah Medical College, Pakistan

2020 ◽  
Vol 1 (2) ◽  
pp. 4
Author(s):  
Musarat Ramzan ◽  
Shezadi Sabah Imran ◽  
Sana Bibi ◽  
Khola Waheed Khan ◽  
Imrana Maqsood
Keyword(s):  
Pearson Correlation ◽  
Item Analysis ◽  
Multiple Choice ◽  
Community Medicine ◽  
P Value ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Significant Difference ◽  
Difficulty Index

Objective: The objective of the study was to assess the quality of multiple-choice questions (MCQs) of three different assessments in the subject of Community Medicine by computing the difficulty index, discrimination index and reliability and to estimate the relationship between difficulty and discrimination indices. Study Design: Retrospective observational study. Place and Duration of Study: Department of Community Medicine at Wah Medical College from August to December 2018. Materials and Methods: Three sets of MCQs were included in the study. Mean and standard deviation of difficulty and discrimination indices were calculated and one-way analysis of variance and Kruskal Wallis test were applied on difficulty and discrimination indices. The association was determined by Pearson correlation and considered significant at p value of < 0.05. Results: The mean difficulty index of first term, second term and send-up examination were 41.5, 48.8 and 51.9 respectively. Mean discrimination indices were 0.28, 0.27 and 0.26 and reliability were 0.83, 0.81 and 0.79. In the study, 72% MCQs of the first term, 61.5 % of the second term and 63% of the send-up examinations were in the range 30-70% of difficulty. There was a significant difference in the difficulty indices of the three question papers. The correlation between discrimination and difficulty indices was curvilinear and positively correlated. Conclusion: It is concluded that all three question papers have acceptable reliability, more than 65% MCQs have acceptable difficulty index and about 69% have good discriminatory power.

scholarly journals Item analysis to identify quality multiple choice questions/items in an assessment in Pharmacology of II MBBS students in Guntur Medical College of Andhra Pradesh, India

2018 ◽  
Vol 7 (8) ◽  
pp. 1517
Author(s):  
Vijaya K. Suryadevara ◽  
Zaheda Bano
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Correct Choice ◽  
Assessment Tools ◽  
P Value ◽  
Choice Response ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Difficulty Index

Background: In medical education, multiple choice questions/Items are the more frequently used assessment tools to assess the knowledge abilities and skills of medical students, for being their objectivity, wide coverage in less time. However only the Quality Items gives a valid and reliable assessment. The quality of an Item is determined by difficulty index (DIF I), Discrimination Index (DI) and Distractor efficiency (DE). Aim of the study was to know the quality of Items in pharmacology by Item analysis and to develop a MCQs bank with quality Items.Methods: The present study was conducted on 150 II MBBS students of Guntur Medical College, AP, India. A class test containing 50 Items with 150 distractors from topic chemotherapy was conducted. Item with the correct choice/response was awarded with one mark and with the wrong choice zero marks, no negative marks. Each test Item was analysed with DIF I, DI and DE and the results were tabulated and tested statistically, with unpaired "t" test.Results: Mean DIF I, DI, DE values with standard deviations in the present study are 44.72+17.63%, 0.30+0.12%, 84.48+24.65 respectively. DIF I of 32 (64%) items was good to excellent range (31%-60%) 9 (18%) Items were easy (>61%) and 9(18%) Items were difficult (>30%). DI of 10 (20%) Items was good (0.15 to 0.24.) 29 (58%) Items were excellent with DI > 0.25 and 11 (22%) Items were poor with DI <0.15. Among 150 distractors, 127 (85%) were functional distractors (FDs) and 23 (15%) were non-functional distractors (NFDs). DE of 33 (66%) items with nil NFDs was 100%, for 12 (24%) Items with one NFD, was 66.6%, for 4 (8%) items with 2 NFDs was 33.3% and for 1 (2%) Item with 3NFDs DE was 0%. When unpaired "t" test was applied to the means of "difficult" and "easy" Items, 96.22+11.33% SD, 51.44+29.31% SD respectively, the p-value obtained was 0.00058, which was highly significant.Conclusions: The study showed that Item analysis is a valid tool to identify quality Items, which assess, the students’ knowledge abilities and discriminate different levels of performance abilities of students effectively.

scholarly journals Item Analysis of Multiple Choice Questions in Pharmacology

2020 ◽  
Vol 10 (2) ◽  
Author(s):  
Imtiaz Uddin ◽  
Iftikhar Uddin ◽  
Izaz Ur Rehman ◽  
Muhammad Siyar ◽  
Usman Mehboob
Keyword(s):  
Medical Education ◽  
Item Analysis ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Cross Sectional ◽  
Teaching Programme ◽  
Medical College ◽  
Item Writing ◽  
Difficulty Index

Background: MCQs type assessment in medical education is replacing old theory style. There are concerns regarding the quality of the Multiple Choice Questions.Objectives: To determine the quality of Multiple Choice Questions by item analysis. Material and Methods: Study was a cross sectional descriptive .Fifty Multiple Choice Questions in the final internal evaluation exams in 2015 of Pharmacology at Bacha khan Medical College were analyzed. The quality of each Multiple Choice Questions item was assessed by the Difficulty index (Dif.I), Discriminative Index (D.I) and Distracter Efficiency (D.E).Results: Multiple Choice Questions that were of moderate difficulty were 66%. Easy were 4% and high difficulty were 30%.Reasons for high difficult Multiple Choice Questions were analyzed as Item Writing Flaws 41%, Irreverent Difficulty 36% and C2 level 23%. Discrimination Index shows that majority of MCQs were of Excellent Level (DI greater than 0.25) i.e 52 , Good 32% . (DI=2.15-0.25), Poor 16%. MCQs Distracter Effectiveness (DE)= 4, 3,2,1 were 52%, 34%, 14%, and 0% respectively. Conclusion: Item analysis gives us different parameters with reasons to recheck MCQ pool and teaching programme. High proportions of difficult and sizable amount of poor discriminative indices MCQs were the finding in this study and need to be resolved

scholarly journals Analysis of multiple choice questions from a formative assessment of medical students of a medical college in Delhi, India

2018 ◽  
Vol 7 (1) ◽  
pp. 174
Author(s):  
Richa Garg ◽  
Vikas Kumar ◽  
Jyoti Maria
Keyword(s):  
Formative Assessment ◽  
Item Analysis ◽  
Multiple Choice ◽  
Medical Knowledge ◽  
Discrimination Index ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Best Response ◽  
Difficulty Index

Background: Assessment is a dominant motivator to direct and drive students learning. Different methods of assessment are used to assess medical knowledge in undergraduate medical education. Multiple choice questions (MCQs) are being used increasingly due to their higher reliability, validity, and ease of scoring. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), and distracter efficiency (DE).Methods: Students of second year MBBS appeared in a formative assessment test, that was comprised of 50 “One best response type” MCQs of 50 marks without negative marking. All MCQs were having single stem with four options including, one being correct answer and other three incorrect alternatives (distracter). Three question paper sets were prepared by disorganizing sequence of questions. One of the three paper sets was given to each student to avoid copying from neighboring students. Total 50 MCQs and 150 distracters were analyzed and indices like DIF I, DI, and DE were calculated.Results: Total Score of 87 students ranged from 17 to 48 (out of total 50). Mean for difficulty index (DIF I) (%) was 71.6+19.4. 28% MCQs were average and “recommended” (DIF I 30-70%). Mean for discrimination index (DI) was 0.3+0.17. 16% MCQs were “good” and 50% MCQs were in “excellent” criteria, while rests of the MCQs were “discard/poor” according to DI criteria. Mean for distracter efficiency (DE) (%) was 63.4+33.3. 90% of the items were having DE from 100 to 33%. It was found that MCQs with lower difficulty index (<70) were having higher distracter efficiency (93.8% vs. 6.2%, p=0.004).Conclusions: Item analysis provided necessary data for improvement in question formulation and helped in revising and improving the quality of items and test also. Questions having lower difficulty index (<70) were significantly associated with higher discrimination index (>0.15) and higher distractor efficiency.

scholarly journals Item analysis of multiple choice questions of undergraduate pharmacology examinations in a medical college in Belagavi, Karnataka, India

2018 ◽  
Vol 7 (10) ◽  
pp. 1917
Author(s):  
Netravathi B. Angadi ◽  
Amitha Nagabhushana ◽  
Nayana K. Hashilkar
Keyword(s):  
Simple Procedure ◽  
Item Analysis ◽  
Multiple Choice ◽  
Reliability And Validity ◽  
Incorrect Response ◽  
Test Construction ◽  
Multiple Choice Questions ◽  
Course Content ◽  
Medical College ◽  
Difficulty Index

Background: Multiple choice questions (MCQs) are a common method of assessment of medical students. The quality of MCQs is determined by three parameters such as difficulty index (DIF I), discrimination index (DI), and Distractor efficiency (DE). Item analysis is a valuable yet relatively simple procedure, performed after the examination that provides information regarding the reliability and validity of a test item. The objective of this study was to perform an item analysis of MCQs for testing their validity parameters.Methods: 50 items consisting of 150 distractors were selected from the formative exams. A correct response to an item was awarded one mark with no negative marking for incorrect response. Each item was analysed for three parameters such as DIF I, DI, and DE.Results: A total of 50 items consisting of 150 Distractor s were analysed. DIF I of 31 (62%) items were in the acceptable range (DIF I= 30-70%) and 30 had ‘good to excellent’ (DI >0.25). 10 (20%) items were too easy and 9 (18%) items were too difficult (DIF I <30%). There were 4 items with 6 non-functional Distractor s (NFDs), while the rest 46 items did not have any NFDs.Conclusions: Item analysis is a valuable tool as it helps us to retain the valuable MCQs and discard or modify the items which are not useful. It also helps in increasing our skills in test construction and identifies the specific areas of course content which need greater emphasis or clarity.

scholarly journals Quality of multiple choice questions in undergraduate pharmacology assessments in a teaching hospital of Kerala, India: an item analysis

2017 ◽  
Vol 6 (6) ◽  
pp. 1265
Author(s):  
Manju K. Nair ◽  
Dawnji S. R.
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Discrimination Index ◽  
Response Type ◽  
Multiple Choice Questions ◽  
Microsoft Excel ◽  
Best Response ◽  
Difficulty Index ◽  
Upper Third

Background: Carefully constructed, high quality multiple choice questions can serve as effective tools to improve standard of teaching. This item analysis was performed to find the difficulty index, discrimination index and number of non functional distractors in single best response type questions.Methods: 40 single best response type questions with four options, each carrying one mark for the correct response, was taken for item analysis. There was no negative marking. The maximum marks was 40. Based on the scores, the evaluated answer scripts were arranged with the highest score on top and the least score at the bottom. Only the upper third and lower third were included. The response to each item was entered in Microsoft excel 2010. Difficulty index, Discrimination index and number of non functional distractors per item were calculated.Results: 40 multiple choice questions and 120 distractors were analysed in this study. 72.5% items were good with a difficulty index between 30%-70%. 25% items were difficult and 2.5% items were easy. 27.5% items showed excellent discrimination between high scoring and low scoring students. One item had a negative discrimination index (-0.1). There were 9 items with non functional distractors.Conclusions: This study emphasises the need for improving the quality of multiple choice questions. Hence repeated evaluation by item analysis and modification of non functional distractors may be performed to enhance standard of teaching in Pharmacology.

scholarly journals Study on item and test analysis of multiple choice questions amongst undergraduate medical students

2017 ◽  
Vol 4 (5) ◽  
pp. 1562
Author(s):  
Abhijeet S. Ingale ◽  
Purushottam A. Giri ◽  
Mohan K. Doibale
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Test Methods ◽  
Discrimination Index ◽  
Multiple Choice Questions ◽  
Undergraduate Medical Students ◽  
Test Quality ◽  
Test Analysis ◽  
Difficulty Index

Background: Item analysis is the process of collecting, summarizing and using information from students’ response to assess the quality of test items. However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. There is more to writing good MCQs than writing good questions. The objectives of the study was to assess the item and test quality of multiple choice questions and to deal with the learning difficulties of students, identify the low achievers in the test. Methods: The hundred MBBS students from Government medical college were examined. A test comprising of thirty MCQs was administered. All items were analysed for Difficulty Index, Discrimination Index and Distractor Efficiency. Data entered in MS Excel 2007 and SPSS 21 analysed with statistical test of significance. Results: Majority 80% items difficulty index is within acceptable range. 63% items showed excellent discrimination Index. Distractor efficiency was overall satisfactory. Conclusions: Multiple choice questions with average difficulty and also having high discriminating power with good distracter efficiency should be incorporated into student’s examination. 

scholarly journals Evaluation of multiple choice questions using item analysis tool: a study from a medical institute of Ahmedabad, Gujarat

2017 ◽  
Vol 4 (6) ◽  
pp. 1876 ◽  
Author(s):  
Donald S. Christian ◽  
Arpit C. Prajapati ◽  
Bhavik M. Rana ◽  
Viral R. Dave
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Thinking Skills ◽  
Test Session ◽  
Discrimination Index ◽  
Medical Institute ◽  
P Value ◽  
Analysis Tool ◽  
Multiple Choice Questions ◽  
Difficulty Index

Background: Multiple choice question (MCQ) assessments are becoming popular means to assess knowledge for many screening examinations among several fields including Medicine. The single best answer MCQs may also test higher-order thinking skills. Hence, MCQs remain useful assessment gadget. Objectives: 1) To evaluate Multiple Choice Questions for testing their quality. 2) To explore the association between difficulty index (p-value) and discrimination indices (DI) with distractor efficiency (DE). 3) To study the occurrence of functioning distractors for MCQs. Methods: Total five MCQ test sessions were conducted among interns of a medical institute of Ahmedabad city Gujarat, between April 2016 to March 2017, as part of their compulsory rotating postings in the department. The average participation in each of the sessions was 17 interns, thus a total of 85 interns getting enrolled. For each test session, the questionnaire consisted of forty MCQs having 4 options including a single best answer. The MCQs were analyzed for difficulty index (DIF-I, p-value), discrimination index (DI), and distractor efficiency (DE). Results: Total 85 interns attended the tests consisting of total 200 MCQ items (questions) from four major medical disciplines namely - Medicine, Surgery, Obstetrics & Gynecology and Community Medicine. Mean test scores of each test ranged from 36.0% to 45.8%.The reliability of the tests, the Kuder Richardson (KR) 20, ranged from 0.29 to 0.52. The standard error of Measurement ranged from 2.59 to 2.79.Out of total 200 MCQs, seventy nine (n=79) had Discrimination index (DI) <0.15 (poor), and 61 had DI ≥0.35 (excellent). Easy items having average DE of all tests was 20.1%. Conclusions: Items having average difficulty and high discrimination with functioning distractors should be incorporated into tests to improve the validity of the assessment. 

scholarly journals DIFFICULTY AND DISCRIMINATION ANALYSIS OF END OF TERM MULTIPLE-CHOICE QUESTIONS AT COMMUNITY MEDICINE DEPARTMENT, WAH MEDICAL COLLEGE

2021 ◽  
Vol 71 (4) ◽  
pp. 1308-10
Author(s):  
Musarat Ramzan ◽  
Khola Waheed Khan ◽  
Saana Bibi ◽  
Shezadi Sabah Imran
Keyword(s):  
Multiple Choice ◽  
Cross Sectional Study ◽  
Community Medicine ◽  
Multiple Choice Questions ◽  
Cross Sectional ◽  
Medicine Department ◽  
Test Items ◽  
Medical College ◽  
The Mean ◽  
Difficulty Index

Objective: To perform post analysis of multiple-choice questions given in the 2nd term and send up examinations of the years 2016 to 2018, to establish relationship between difficulty (DF) and discrimination indices (DI) and to find out significant mean difference between the two. Study Design: Cross sectional study. Place and Duration of Study: Community Medicine Department, Wah Medical College, Wah, from Nov 2018 to Mar 2019. Methodology: A total of 390 Multiple-Choice Question of second term and send-up were taken for the study from the year 2016, 2017 and 2018. The response sheets were assessed by Optical Machine Reader (OMR) and the level of difficulty, power of discrimination and reliability were obtained. The data was entered in SPSS version 22. Results: A total of 315 test items were included. Results of the study showed that the reliability (KR20) for all the examined items was in the acceptable range i.e. ≥0.7 and there was no association was found between difficulty index and year p=0.310. The mean difficulty index was found to be 0.48 ± 0.22 and discrimination index as 0.24 ± 0.14. Conclusion: The analysis of 390 test items showed that most of the questions were acceptable in terms of difficulty and discrimination. There is still a need to modify and improve the testing ability of the MCQs with negative discrimination and higher difficulty index.

scholarly journals Item Analysis of Multiple-Choice Questions in Pharmacology in an Indian Medical School

2021 ◽  
Author(s):  
Manoor Narasimha Sachidananda Adiga ◽  
Swathi Acharya ◽  
Rajendra Holla
Keyword(s):  
Student Evaluation ◽  
Item Analysis ◽  
Student Assessment ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Medical Teacher ◽  
Internal Assessment ◽  
Medical College ◽  
Acceptable Range ◽  
Difficulty Index

Abstract Introduction Student assessment by multiple-choice questions (MCQs) is an integral part of student evaluation in medicine. The medical teacher should be trained to construct an item with proper stem and valid options. Periodic item analyses will make the process of assessment more meaningful. Hence, we conducted the study to analyze MCQs (item analysis) tested on a batch of MBBS students in pharmacology in their three internal assessment examinations. Methods The study was conducted in the Department of Pharmacology of a medical college in Mangaluru on 150 students. The MCQs of the three internal assessment examinations (20 each) respectively were analyzed. We analyzed each question for difficulty index (DI), discrimination index (DsI), and distracter efficacy or functionality and expressed the percentage results. Results The DI was in an acceptable range of 60, 75, and 90%, respectively, in the three internal assessments. The percentage of “too difficult” questions was 10, 20, and 10% and the average DsI was 0.32 ± 0.04, 0.28 ± 0.02, and 0.26 ± 0.02, respectively. In the second and third internal assessments, 95% of questions had functional distracters, while in the first internal assessment, only 60% of questions had functional distracters. Conclusion We conclude from our study that even though the items (MCQs) framed for the internal assessments were in the acceptable range of quality in terms of the parameters assessed, we must improve MCQ’s construction in selecting distracters in some topics.

scholarly journals Nonfunctional distractor analysis: An indicator for quality of Multiple choice questions

2020 ◽  
Vol 36 (5) ◽  
Author(s):  
Madiha Sajjad ◽  
Samina Iltaf ◽  
Rehan Ahmed Khan
Keyword(s):  
Item Analysis ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Medical College ◽  
Creative Commons ◽  
Irrelevant Distractors ◽  
Item Writing ◽  
Writing Errors ◽  
Distractor Analysis

Objectives: To analyze the low to medium distractor efficiency items in a multiple-choice question (MCQ) paper for item writing flaws. Methods: This qualitative study was conducted at Islamic International Medical College Rawalpindi, in October 2019. Archived item- analysis report from a midyear medium stakes MCQ paper of 2nd year MBBS class, was analyzed to determine the non-functional distractors (NFDs) and distractor efficiency (DE) of items, in a total of 181 MCQs. DE was categorized as low (3-4 NFDs), medium (1-2 NFDs) and high (0 NFD). Subsequently, qualitative document analysis of the MCQ paper whose item analysis report was assessed was conducted to investigate the item flaws in the low to medium DE items. The flaws identified were coded and grouped as, within option flaws, alignment flaws between options and stem/ lead-in and other flaws. Results: Distractor efficiency was high in 69 items (38%), moderate in 75 items (42%) and low in 37 items (20%). The item-writing flaws identified in low to moderate DE items within distractors included, non-homogenous length (1.8%), non-homogenous content (8%) and repeat in distractor (1.7%). Alignment flaws between distractors and stem/ lead-in identified were linguistic cues (10%), logic cues (12.5%) and irrelevant distractors (16%). Flaws unrelated to distractors were low cognitive level items (40%) and unnecessarily complicated stems (11.6%). Conclusions: Analyzing the low to medium DE items for item writing flaws, provides valuable information about item writing errors which negatively impact the distractor efficiency. doi: https://doi.org/10.12669/pjms.36.5.2439 How to cite this:Sajjad M, Iltaf S, Khan RA. Nonfunctional distractor analysis: An indicator for quality of Multiple choice questions. Pak J Med Sci. 2020;36(5):---------. doi: https://doi.org/10.12669/pjms.36.5.2439 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Sign in / Sign up

Export Citation Format

Share Document