Item analysis of biochemistry multiple choice questions - A retrospective Study & scope of improvement in method of assessment of MBBS students

Abstract Background:Multiple choice questions (MCQ) are used in the assessment of students in various fields. By this method of assessment it is possible to cover a wide range of topics in less amount of time. However the reliability of the test depends on the quality of the MCQ. The MCQ can be evaluated based on the Difficulty Index (DIFI), Discriminatory Index (Dl) and Distracter Efficiency (DE). Objectives:To evaluate the MCQs based on the Difficulty Index (DIFI), Discriminatory Index (Dl) & Distracter Efficiency (DE) and develop a valid pool of questions.Also to assess learner performance and discriminate between students of higher and lower abilities. Materials and Methods:A total of 120 students were assessed based on multiple choice questions in pathology. The number of items were 20 and the number of distracters were 60. Data was entered and analyzed in MS Excel 2007 and simple proportions, mean and standard deviations were calculated. Results:Mean and standard deviations for DIFI, Dl and DE were 57.8 ± 17.4%, 0.27 ± 0.17 and 84.98 ± 20.2 % respectively. Out of the 20 items, 11 items had good level of DIFI (31 - 60%), eight (8) items were considered easy (DIFI >61%) and one (1) item was considered difficult (DIFI < 30). Mean Dl in present study was 0.27 ± 0.17. Analysis of the Dl showed good discrimination power in eighteen (18) of the items. Out of the 60 distracters, nine (9) were non -functional distracters (NFD) and were seen in eight items. Out of these, seven items had one NFD each and one item had two NFD. Conclusions:The study emphasizes on the importance of use of item analysis in construction of good quality MCQs and also in the evaluation of learner performance.

Download Full-text

Item analysis of multiple choice questions of undergraduate pharmacology examinations in a medical college in Belagavi, Karnataka, India

International Journal of Basic & Clinical Pharmacology ◽

10.18203/2319-2003.ijbcp20183923 ◽

2018 ◽

Vol 7 (10) ◽

pp. 1917

Author(s):

Netravathi B. Angadi ◽

Amitha Nagabhushana ◽

Nayana K. Hashilkar

Keyword(s):

Simple Procedure ◽

Item Analysis ◽

Multiple Choice ◽

Reliability And Validity ◽

Incorrect Response ◽

Test Construction ◽

Multiple Choice Questions ◽

Course Content ◽

Medical College ◽

Difficulty Index

Background: Multiple choice questions (MCQs) are a common method of assessment of medical students. The quality of MCQs is determined by three parameters such as difficulty index (DIF I), discrimination index (DI), and Distractor efficiency (DE). Item analysis is a valuable yet relatively simple procedure, performed after the examination that provides information regarding the reliability and validity of a test item. The objective of this study was to perform an item analysis of MCQs for testing their validity parameters.Methods: 50 items consisting of 150 distractors were selected from the formative exams. A correct response to an item was awarded one mark with no negative marking for incorrect response. Each item was analysed for three parameters such as DIF I, DI, and DE.Results: A total of 50 items consisting of 150 Distractor s were analysed. DIF I of 31 (62%) items were in the acceptable range (DIF I= 30-70%) and 30 had ‘good to excellent’ (DI >0.25). 10 (20%) items were too easy and 9 (18%) items were too difficult (DIF I <30%). There were 4 items with 6 non-functional Distractor s (NFDs), while the rest 46 items did not have any NFDs.Conclusions: Item analysis is a valuable tool as it helps us to retain the valuable MCQs and discard or modify the items which are not useful. It also helps in increasing our skills in test construction and identifies the specific areas of course content which need greater emphasis or clarity.

Download Full-text

Inversions in true/false and in multiple choice questions - a new form of item analysis

Medical Education ◽

10.1111/j.1365-2923.1979.tb01201.x ◽

1979 ◽

Vol 13 (6) ◽

pp. 420-424 ◽

Cited By ~ 5

Author(s):

J. H. KOESLAG ◽

C. W. MELZER ◽

S. R. SCHACH

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Multiple Choice Questions ◽

New Form ◽

True False

Download Full-text

Development of multiple choice question bank in otorhinolaryngology by item analysis: a cross-sectional study

International Journal of Otorhinolaryngology and Head and Neck Surgery ◽

10.18203/issn.2454-5929.ijohns20190779 ◽

2019 ◽

Vol 5 (2) ◽

pp. 449

Author(s):

Ajeet Kumar Khilnani ◽

Rekha Thaddanee ◽

Gurudas Khilnani

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Cross Sectional Study ◽

Low Achievers ◽

Discrimination Index ◽

High Achievers ◽

Multiple Choice Questions ◽

Cross Sectional ◽

The Mean ◽

Difficulty Index

Background: Multiple choice questions (MCQs) are routinely used for formative and summative assessment in medical education. Item analysis is a process of post validation of MCQ tests, whereby items are analyzed for difficulty index, discrimination index and distractor efficiency, to obtain a range of items of varying difficulty and discrimination indices. This study was done to understand the process of item analysis and analyze MCQ test so that a valid and reliable MCQ bank in otorhinolaryngology is developed.Methods: 158 students of 7th Semester were given an 8 item MCQ test. Based on the marks achieved, the high achievers (top 33%, 52 students) and low achievers (bottom 33%, 52 students) were included in the study. The responses were tabulated in Microsoft Excel Sheet and analyzed for difficulty index, discrimination index and distractor efficiency. Results: The mean (SD) difficulty index (Diff-I) of 8 item test was 61.41% (11.81%). 5 items had a very good difficulty index (41% to 60%), while 3 items were easy (Diff-I >60%). There was no item with Diff-I <30%, i.e. a difficult item, in this test. The mean (SD) discrimination index (DI) of the test was 0.48 (0.15), and all items had very good discrimination indices of more than 0.25. Out of 24 distractors, 6 (25%) were non-functional distractors (NFDs). The mean (SD) distractor efficiency (DE) of the test was 74.62% (23.79%).Conclusions: Item analysis should be an integral and regular activity in each department so that a valid and reliable MCQ question bank is developed.

Download Full-text

Quality of multiple choice questions in undergraduate pharmacology assessments in a teaching hospital of Kerala, India: an item analysis

International Journal of Basic & Clinical Pharmacology ◽

10.18203/2319-2003.ijbcp20172001 ◽

2017 ◽

Vol 6 (6) ◽

pp. 1265

Author(s):

Manju K. Nair ◽

Dawnji S. R.

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Discrimination Index ◽

Response Type ◽

Multiple Choice Questions ◽

Microsoft Excel ◽

Best Response ◽

Difficulty Index ◽

Upper Third

Background: Carefully constructed, high quality multiple choice questions can serve as effective tools to improve standard of teaching. This item analysis was performed to find the difficulty index, discrimination index and number of non functional distractors in single best response type questions.Methods: 40 single best response type questions with four options, each carrying one mark for the correct response, was taken for item analysis. There was no negative marking. The maximum marks was 40. Based on the scores, the evaluated answer scripts were arranged with the highest score on top and the least score at the bottom. Only the upper third and lower third were included. The response to each item was entered in Microsoft excel 2010. Difficulty index, Discrimination index and number of non functional distractors per item were calculated.Results: 40 multiple choice questions and 120 distractors were analysed in this study. 72.5% items were good with a difficulty index between 30%-70%. 25% items were difficult and 2.5% items were easy. 27.5% items showed excellent discrimination between high scoring and low scoring students. One item had a negative discrimination index (-0.1). There were 9 items with non functional distractors.Conclusions: This study emphasises the need for improving the quality of multiple choice questions. Hence repeated evaluation by item analysis and modification of non functional distractors may be performed to enhance standard of teaching in Pharmacology.

Download Full-text

Relationship of text length of multiple-choice questions on item psychometric properties – A retrospective study

Saudi Journal for Health Sciences ◽

10.4103/sjhs.sjhs_76_20 ◽

2020 ◽

Vol 9 (2) ◽

pp. 84

Author(s):

Fawaz Pullishery ◽

DareenKhalid Aljehani ◽

OmerAbdelgadir Elfaki Osman ◽

BasemMohamed Abuzenada

Keyword(s):

Retrospective Study ◽

Psychometric Properties ◽

Multiple Choice ◽

Multiple Choice Questions ◽

Text Length ◽

Relationship Of

Download Full-text

The trials and tribulations of designing and utilising MCQs in HE and for assessing forensic practitioner competency

New Directions in the Teaching of Physical Sciences ◽

10.29311/ndtps.v0i7.474 ◽

2011 ◽

pp. 72-78

Author(s):

Claire Gwinnett ◽

John Cassella ◽

Mike Allen

Keyword(s):

Higher Education ◽

Science Education ◽

Deep Learning ◽

Forensic Science ◽

Cognitive Skills ◽

Multiple Choice ◽

Higher Order ◽

Multiple Choice Questions ◽

Funded Project ◽

Method Of Assessment

Multiple Choice Questions (MCQs) are a very well known, traditional and accepted method of assessment. The use of MCQs for testing students has produced numerous debates amongst academics concerning their effectiveness as they are viewed as practical and efficient but also perceived as possibly „too easy‟ and potentially unable to appropriately test the higher order cognitive skills that essay questions can assess.The use of MCQs in a forensic science context is currently being investigated, not only for use within forensic science education, but also for the testing of competency of qualified forensic practitioners. This paper describes a Higher Education Academy funded project that is investigating the design and the implementation of MCQs for testing forensic practitioners and the lessons that have been learnt so far, that will assist academics in the development of robust MCQ assessments within forensic science degrees to promote and assess deep learning.

Download Full-text

Item analysis as tool to validate multiple choice question bank in pharmacology

International Journal of Basic & Clinical Pharmacology ◽

10.18203/2319-2003.ijbcp20194106 ◽

2019 ◽

Vol 8 (9) ◽

pp. 1999

Author(s):

Amit P. Date ◽

Archana S. Borkar ◽

Rupesh T. Badwaik ◽

Riaz A. Siddiqui ◽

Tanaji R. Shende ◽

...

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Incorrect Response ◽

Cross Sectional Study ◽

Discrimination Index ◽

Multiple Choice Questions ◽

Medical Sciences ◽

Cross Sectional ◽

Lower Ability ◽

Difficulty Index

Background: Multiple choice questions (MCQs) are a common method for formative and summative assessment of medical students. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), distracter efficiency (DE). The objective of this study was to assess the quality of MCQs currently in use in pharmacology by item analysis and develop a MCQ bank with quality items.Methods: This cross-sectional study was conducted in 148 second year MBBS students at NKP Salve institute of medical sciences from January 2018 to August 2018. Forty MCQs twenty each from the two term examination of pharmacology were taken for item analysis A correct response to an item was awarded one mark and each incorrect response was awarded zero. Each item was analyzed using Microsoft excel sheet for three parameters such as DIF I, DI, and DE.Results: In present study mean and standard deviation (SD) for Difficulty index (%) Discrimination index (%) and Distractor efficiency (%) were 64.54±19.63, 0.26±0.16 and 66.54±34.59 respectively. Out of 40 items large number of MCQs has acceptable level of DIF (70%) and good in discriminating higher and lower ability students DI (77.5%). Distractor efficiency related to presence of zero or 1 non-functional distrator (NFD) is 80%.Conclusions: The study showed that item analysis is a valid tool to identify quality items which regularly incorporated can help to develop a very useful, valid and a reliable question bank.

Download Full-text

ITEM ANALYSIS BLOK IMUNOLOGI DAN NEOPLASMA TA 2015-2016 DI PROGRAM STUDI PENDIDIKAN DOKTER UNIVERSITAS MALIKUSSALEH

AVERROUS: Jurnal Kedokteran dan Kesehatan Malikussaleh ◽

10.29103/averrous.v1i2.398 ◽

2018 ◽

Vol 1 (2) ◽

pp. 15

Author(s):

Sri Wahyuni

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Multiple Choice Questions ◽

Microsoft Excel

Multiple-choice questions (MCQs) atau soal pilihan berganda sering digunakan dalam penilaian mahasiswa. Item analysis secara luas digunakan untuk memperbaiki kualitas ujian. Indeks kesulitan soal (p) dan indeks diskriminasi soal (D) adalah dua indikator objektif untuk evaluasi soal MCQ. Penelitian ini bertujuan untuk menilai kualitas soal dan menjadikan soal yang berkualitas sebagai bank soal. Sebanyak 70 soal MCQ Blok Imunologi dan Neoplasma dianalisis dan dihitung nilai p dan nilai D masing-masing soal menggunakan Microsoft Excel. Korelasi antara p dan D dianalisis menggunakan korelasi Pearson. Hasil penelitian menunjukkan rerata nilai mahasiswa adalah 33,5 ± 10,3. Rerata nilai p adalah 0.48 ± 0.21 dan rerata nilai D adalah 0.37 ± 0.23. Sebanyak 13 (19%) soal termasuk mudah, 15 (21%) soal termasuk sulit dan 42 (60%) masih acceptable. Sebanyak masing-masing 3 (4%) soal mempunyai nilai indeks diskriminasi negatif dan rendah (poor). Total 79% soal memiliki indeks diskriminasi acceptable-excellent. Indeks diskriminasi menunjukkan korelasi positif lemah dengan indeks kesulitan soal (r = 0,357) dan bermakna signifikan (p=0,02). Penelitian ini menyimpulkan bahwa soal MCQ sudah mempunyai kualitas yang baik dan dapat dipergunakan pada ujian selanjutnya sehingga memperbaiki kualitas ujian secara keseluruhan.

Download Full-text