How To Create A Good Test

Assessment is a very important part of a learning process. To conduct the assessment, teachers have to design a test. To retrieve a quality of a test, a physics teacher needs to do the items analysis. There are several ways of doing items analysis which including the analysis of difficulty index, discrimination index, and analysis of the validity and reliability. This research is a descriptive study that aims to describe systematically and accurate information on the actual situation in this case about the difficulty index, discrimination index, validity and reliability of the items. The variables analyzed were the quality of teacher-made tests physics. The results of this study indicate that physics teacher-made tests have low validity, reliability moderate or medium, high difficulty index, and poor discrimination index

Download Full-text

Quality of multiple choice questions in undergraduate pharmacology assessments in a teaching hospital of Kerala, India: an item analysis

International Journal of Basic & Clinical Pharmacology ◽

10.18203/2319-2003.ijbcp20172001 ◽

2017 ◽

Vol 6 (6) ◽

pp. 1265

Author(s):

Manju K. Nair ◽

Dawnji S. R.

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Discrimination Index ◽

Response Type ◽

Multiple Choice Questions ◽

Microsoft Excel ◽

Best Response ◽

Difficulty Index ◽

Upper Third

Background: Carefully constructed, high quality multiple choice questions can serve as effective tools to improve standard of teaching. This item analysis was performed to find the difficulty index, discrimination index and number of non functional distractors in single best response type questions.Methods: 40 single best response type questions with four options, each carrying one mark for the correct response, was taken for item analysis. There was no negative marking. The maximum marks was 40. Based on the scores, the evaluated answer scripts were arranged with the highest score on top and the least score at the bottom. Only the upper third and lower third were included. The response to each item was entered in Microsoft excel 2010. Difficulty index, Discrimination index and number of non functional distractors per item were calculated.Results: 40 multiple choice questions and 120 distractors were analysed in this study. 72.5% items were good with a difficulty index between 30%-70%. 25% items were difficult and 2.5% items were easy. 27.5% items showed excellent discrimination between high scoring and low scoring students. One item had a negative discrimination index (-0.1). There were 9 items with non functional distractors.Conclusions: This study emphasises the need for improving the quality of multiple choice questions. Hence repeated evaluation by item analysis and modification of non functional distractors may be performed to enhance standard of teaching in Pharmacology.

Download Full-text

Impact of distractors in item analysis of multiple choice questions

International Journal of Research in Medical Sciences ◽

10.18203/2320-6012.ijrms20191313 ◽

2019 ◽

Vol 7 (4) ◽

pp. 1136

Author(s):

Ismail Burud ◽

Kavitha Nagandla ◽

Puneet Agarwal

Keyword(s):

Quality Assurance ◽

Statistical Significance ◽

Item Analysis ◽

Validity And Reliability ◽

Multiple Choice Questions ◽

Test Items ◽

Difficulty Index ◽

The Individual ◽

Discriminatory Index

Background: Item analysis is a quality assurance of examining the performance of the individual test items that measures the validity and reliability of exams. This study was performed to evaluate the quality of the test items with respect to their performance on difficulty index (DFI), Discriminatory index (DI) and assessment of functional and non-functional distractors (FD and NFD).Methods: This study was performed on the summative examination undertaken by 113 students. The analyses include 120 one best answers (OBAs) and 360 distractors.Results: Out of the 360 distractors, 85 distractors were chosen by less than 5% with the distractor efficiency of 23.6%. About 47 (13%) items had no NFDs while 51 (14%), 30 (8.3%), and 4 (1.1%) items contained 1, 2, and 3 NFDs respectively. Majority of the items showed excellent difficulty index (50.4%, n=42) and fair discrimination (37%, n=33). The questions with excellent difficulty index and discriminatory index showed statistical significance with 1NFD and 2 NFD (p=0.03).Conclusions: The post evaluation of item performance in any exam in one of the quality assurance method of identifying the best performing item for quality question bank. The distractor efficiency gives information on the overall quality of item.

Download Full-text

Item and distracter analysis of multiple choice questions (MCQs) from a preliminary examination of undergraduate medical students

International Journal of Research in Medical Sciences ◽

10.18203/2320-6012.ijrms20175453 ◽

2017 ◽

Vol 5 (12) ◽

pp. 5351 ◽

Cited By ~ 1

Author(s):

Durgesh Prasad Sahoo ◽

Rakesh Singh

Keyword(s):

Multiple Choice ◽

Discrimination Index ◽

Difficulty Level ◽

Validity And Reliability ◽

Multiple Choice Questions ◽

Undergraduate Medical Students ◽

Test Items ◽

Preliminary Examination ◽

High Discrimination ◽

Difficulty Index

Background: Multiple choice questions (MCQs) or Items forms an important part to assess students in different educational streams. It is an objective mode of assessment which requires both the validity and reliability depending on the characteristics of its items i.e. difficulty index, discrimination index and distracter efficiency. To evaluate MCQs or items and build a bank of high-quality test items by assessing with difficulty index, discrimination index and distracter efficiency and also to revise/store or remove errant items based on obtained results.Methods: A preliminary examination of Third MBBS Part-1 was conducted by Department of Community Medicine undertaken for 100 students. Two separate papers with total 30 MCQs or items and 90 distractors each in both papers were analyzed and compared. Descriptive as well as inferential statistics were used to analyze the data.Results: The findings show that most of the items were falling in acceptable range of difficulty level however some items were rejected due to poor discrimination index. Overall paper I was found to be more difficult and more discriminatory, but its distractor efficiency was slightly low as compared to paper II.Conclusions: The analysis helped us in selection of quality MCQs having high discrimination and average difficulty with three functional distractors. This should be incorporated into future evaluations to improve the test score and properly discriminate among the students.

Download Full-text

The Quality of the Department of Library Science's Website by WebQual 4.0

Khizanah al-Hikmah Jurnal Ilmu Perpustakaan Informasi dan Kearsipan ◽

10.24252/kah.v8i2a11 ◽

2020 ◽

Vol 8 (2) ◽

pp. 218-226

Author(s):

Andi Ibrahim ◽

◽

Touku Umar ◽

Feri Ardinata

Keyword(s):

Information Quality ◽

Sampling Technique ◽

Descriptive Study ◽

Library Science ◽

Quality Information ◽

Validity And Reliability ◽

Science Study ◽

Study Program ◽

Interaction Quality

This research discusses the quality of the website of the Library Science Study Program Universitas Islam Negeri Alauddin Makassar as a medium of information using the WebQual 4.0 method. Three measurement variables are used, namely: Usability Quality, Information Quality, and Interaction Quality. This descriptive study uses a quantitative approach in presenting data. The data were obtained through online questionnaires and website observations. The population in this study was 448,440 the number of website visitors and a sample of 100 student respondents who were determined by incidental sampling technique. The validity and reliability tests used SPSS 23 software. The results showed that the quality of usability and information quality of the website of the Department of Library Science UIN Alauddin Makassar were in the good category with a percentage of 75.1% and 73.7%. Meanwhile, the interaction quality is in the very good category with a percentage of 81.5%. So it can be said that the quality of this website is in a good category.

Download Full-text

Study on item and test analysis of multiple choice questions amongst undergraduate medical students

International Journal of Community Medicine and Public Health ◽

10.18203/2394-6040.ijcmph20171764 ◽

2017 ◽

Vol 4 (5) ◽

pp. 1562

Author(s):

Abhijeet S. Ingale ◽

Purushottam A. Giri ◽

Mohan K. Doibale

Keyword(s):

Item Analysis ◽

Multiple Choice ◽

Test Methods ◽

Discrimination Index ◽

Multiple Choice Questions ◽

Undergraduate Medical Students ◽

Test Quality ◽

Test Analysis ◽

Difficulty Index

Background: Item analysis is the process of collecting, summarizing and using information from students’ response to assess the quality of test items. However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. There is more to writing good MCQs than writing good questions. The objectives of the study was to assess the item and test quality of multiple choice questions and to deal with the learning difficulties of students, identify the low achievers in the test. Methods: The hundred MBBS students from Government medical college were examined. A test comprising of thirty MCQs was administered. All items were analysed for Difficulty Index, Discrimination Index and Distractor Efficiency. Data entered in MS Excel 2007 and SPSS 21 analysed with statistical test of significance. Results: Majority 80% items difficulty index is within acceptable range. 63% items showed excellent discrimination Index. Distractor efficiency was overall satisfactory. Conclusions: Multiple choice questions with average difficulty and also having high discriminating power with good distracter efficiency should be incorporated into student’s examination.

Download Full-text

Pre and Post examination evaluation of MCQs and SEQs Items in Undergraduate MBBS mock examination.

The Professional Medical Journal ◽

10.29309/tpmj/2020.27.12.4474 ◽

2020 ◽

Vol 27 (12) ◽

pp. 2749-2754

Author(s):

Anila Jaleel ◽

Zohra Khanum

Keyword(s):

Item Analysis ◽

Descriptive Study ◽

Medical College ◽

Physiology And Biochemistry ◽

Item Writing ◽

Difficulty Index ◽

Blooms Taxonomy ◽

Discriminatory Index ◽

Cover Test

Objectives: To evaluate the quality of MCQs and SEQs pre and post Mock examination of physiology and biochemistry and correlation between the scores of both at private medical college Lahore. Study Design: Descriptive study and convenient sampling was done. Setting: Fatima Memorial College of Medicine and Dentistry, Lahore. Period: September 2016 to September 2017. Material & Methods: 149 students in Physiology and 143 in biochemistry took Mock examination. 45 MCQs and 9 SEQs each in biochemistry and physiology were prepared according to the table of specification provided by University of Health Sciences (UHS) Lahore, Pakistan. Item assessment according to Blooms taxonomy was done and item flaws identified with cover test done for structural validity of the paper before the exam by two medical educationists. Item analysis with difficulty index, discrimination index and distraction efficiency were done post examination. Results: 84.4% in physiology and 51.1% in biochemistry were of recall type (C1 level according to Blooms taxonomy), and 58% SEQs in physiology and 50% in biochemistry were C1. 20% and 28% MCQs in physiology and biochemistry respectively pass cover test and were without item writing flaws. Difficulty index shows that 53.3% in physiology and 48.8% in biochemistry needs modifications. 48.8% of MCQs in physiology and 15.5% in biochemistry need modification in discriminatory index. Similarly 59.5% and 64.0 % of MCQs had functional distractors in physiology and biochemistry respectively. Conclusion: The study concluded that Mock examination in subjects of physiology and biochemistry had item writing flaws, more MCQs were of C1 level and showed majority items in good difficulty, discriminatory index with two third functional distractors were present

Download Full-text

Assessment of MCQs in MBBS program in college of Medicine, King Khalid University Abha (Preprint)

10.2196/preprints.9508 ◽

2017 ◽

Author(s):

Abdulaziz Alamri ◽

Omer Abdelgadir Elfaki ◽

Karimeldin A Salih ◽

Suliman Al Humayed ◽

Fatmah Mohammed Ahmad Althebat ◽

...

Keyword(s):

Item Analysis ◽

Analysis Data ◽

Cross Sectional Study ◽

Discrimination Index ◽

Test Reliability ◽

Multiple Choice Questions ◽

Cross Sectional ◽

The Mean ◽

Difficulty Index

BACKGROUND Multiple choice questions represent one of the commonest methods of assessment in medical education. They believed to be reliable and efficient. Their quality depends on good item construction. Item analysis is used to assess their quality by computing difficulty index, discrimination index, distractor efficiency and test reliability. OBJECTIVE The aim of this study was to evaluate the quality of MCQs used in the college of medicine, King Khalid University, Saudi Arabia. METHODS Design: Cross sectional Study design Setting, Materials and methods Item analysis data of 21 MCQs exams were collected. Values for difficulty index, discrimination index, distractor efficiency and reliability coefficient were entered in MS excel 2010. Descriptive statistic parameters were computed. RESULTS Twenty one tests were analyzed. Overall, 7% of the items among all the tests were difficult, 35% were easy and 58% were acceptable. The mean difficulty of all the tests was in the acceptable range of 0.3-0.85. Items with acceptable discrimination index among all tests were 39%-98%. Negatively discriminating items were identified in all tests except one. All distractors were functioning in 5%-48%. The mean functioning distractors ranged from 0.77 to 2.25. The KR-20 scores lie between 0.47 and 0.97 CONCLUSIONS Overall, the quality of the items and tests was found to be acceptable. Some items were identified to be problematic and need to be revised. The quality of few tests of specific courses was questionable. These tests need to be revised and steps taken to improve this situation.

Download Full-text

Validity and reliability of the novel thyroid-specific quality of life questionnaire, ThyPRO

Acta Endocrinologica ◽

10.1530/eje-09-0521 ◽

2010 ◽

Vol 162 (1) ◽

pp. 161-167 ◽

Cited By ~ 80

Author(s):

Torquil Watt ◽

Laszlo Hegedüs ◽

Mogens Groenvold ◽

Jakob Bue Bjorner ◽

Åse Krogh Rasmussen ◽

...

Keyword(s):

Quality Of Life ◽

Life Questionnaire ◽

Clinical Validity ◽

Validity And Reliability ◽

Good Test ◽

Cross Sectional ◽

Retest Reliability ◽

Patient Reported ◽

Test Retest Reliability

BackgroundAppropriate scale validity and internal consistency reliability have recently been documented for the new thyroid-specific quality of life (QoL) patient-reported outcome (PRO) measure for benign thyroid disorders, the ThyPRO. However, before clinical use, clinical validity and test–retest reliability should be evaluated.AimTo investigate clinical (‘known-groups’) validity and test–retest reliability of the Danish version of the ThyPRO.MethodsFor each of the 13 ThyPRO scales, we defined groups expected to have high versus low scores (‘known-groups’). The clinical validity (known-groups validity) was evaluated by whether the ThyPRO scales could detect expected differences in a cross-sectional study of 907 thyroid patients. Test–retest reliability was evaluated by intra-class correlations of two responses to the ThyPRO 2 weeks apart in a subsample of 87 stable patients.ResultsOn all 13 ThyPRO scales, we found substantial and significant differences between the groups expected to have high versus low scores. Test–retest reliability was above 0.70 (range 0.77–0.89) for all scales, which is usually considered necessary for comparisons among patient groups, but below 0.90, which is the usual threshold for use in individual patients.ConclusionWe found support for the clinical validity of the new thyroid-specific QoL questionnaire, ThyPRO, and evidence of good test–retest reliability. The questionnaire is now ready for use in clinical studies of patients with thyroid diseases.

Download Full-text

METODE KUANTITATIF DENGAN PENDEKATAN KLASIK PADA APLIKASI ANALISIS BUTIR SOAL SEBAGAI MEDIA EVALUASI PENENTUAN SOAL YANG BERKUALITAS

Kilat ◽

10.33322/kilat.v7i1.104 ◽

2018 ◽

Vol 7 (1) ◽

pp. 15-23

Author(s):

Redaksi Tim Jurnal

Keyword(s):

High School ◽

Junior High School ◽

Quantitative Method ◽

Junior High ◽

Item Analysis ◽

Discrimination Index ◽

Classical Approach ◽

The Future ◽

Difficulty Index

This paper discusses the design of an application for the analysis on items using quantitative method with the classical approach as a media evaluation for teachers in determining the quality of questions in an exam or a test. The classical approach is the process of examining each item in the test based on the students' answers for calculating the Difficulty Index (Dif I) and the Discrimination Index (DI) of each question. The Difficulty Index is the ratio between the number of students who answered an item (or a question) correctly and the total number of students who participated in the test. The Discrimination Index is a measure of how well an item (or a question) is able to distinguish between clever students and students who are less clever. The data used in this research was obtained from class VII SMPN 10 (Junior High School) Makassar, especially on math (subject). The data includes math questions that have been tested, the answers given bythe students of class VII, and the data about the students. The results of the item analysis will provide information whether a question can be accepted, corrected or to be rejected. The application generated from this research is expected to be able to offer assistance for the teacher in performing item analysis. The results of the analysis can be used to help teachers in compiling more qualified questions in the future.

Download Full-text

Analysis of the Orthodontic Exams in Department of Orthodontics of Shahid Beheshti Dental School

Journal of Medical Education ◽

10.5812/jme.111566 ◽

2021 ◽

Vol 20 (1) ◽

Author(s):

Kazem Dalaei ◽

Ali Khorsand Nobahar ◽

Mohammad Behnaz ◽

Mahtab Nouri ◽

Fahimeh Anbari

Keyword(s):

Content Validity ◽

Main Tool ◽

Educational Process ◽

Discrimination Index ◽

Dental School ◽

Multiple Choice Questions ◽

Cross Sectional ◽

Structural Rules ◽

Difficulty Index

Background: Exams are an inseparable part of education, and they are the main tool to evaluate the results of educational process. If based on scientific rules, exams can show the level of achievement to educational goals. Objectives: This study aimed to evaluate the quality of orthodontic exams in Shahid Beheshti Dental School, Iran. Methods: In this cross-sectional study, the quality of all written and multiple-choice questions (MCQ) exams of orthodontic department in 2014 - 2015 were evaluated in terms of content validity, structural rules, discrimination index, difficulty index, and taxonomy. To collect the data, the Millman checklist, content analysis table, and standard formula were used. SPSS software version 21 was used to analyze the data. T-test, chi square, and independent sample test were used for statistical analysis. Results: Our results showed that 86.55% of exams were in MCQ form and 13.5% in written form. Content validity was considered in 94.8% of exams. Standard structural rules were observed in 93.61% of questions. Discriminative index and difficulty index were estimated to be 41.6 and 68.8%, respectively. Also, 43.7% of questions were categorized as taxonomy I, 44.38% as taxonomy II, and 11.58% as taxonomy III. Conclusions: Exams held in Shahid Beheshti Dental School in the 2014 - 2015 academic year were acceptable considering their content validity, structural rules, and difficulty index. However, their discrimination index was low, and they were in poor conditions in terms of taxonomy.

Download Full-text