Logical and Psychological Reasoning

1996 ◽  
Vol 83 (3_suppl) ◽  
pp. 1127-1138
Author(s):  
Horacio J. A. Rimoldi ◽  
Elsa Inés Bei de Libonatti
Keyword(s):  
Information Theory ◽  
Problem Solving ◽  
Multiple Choice ◽  
Logical Connectives ◽  
Multiple Choice Items ◽  
Psychological Reasoning ◽  
Exclusive Disjunction ◽  
Final Answer ◽  
Choice Approach

The performance of 30 subjects when solving problems built around logical connectives (Conjunction, Inclusive Disjunction, Exclusive Disjunction, Conditional, and Biconditional) was compared with the results obtained when the same logical connectives were presented using a multiple-choice approach. The processes followed by the subjects in solving 20 problems were evaluated in terms of information theory, making it possible to investigate (a) the processes followed by the subjects and (b) the quality of the final answer Analysis indicated that the problem-solving processes do not necessarily provide the same information as that obtained by the final answers. The knowledge obtained by examining the questions subjects ask is different from the knowledge obtained when examining the answers to multiple-choice items.

Experience Versus Potential of Multiple-Choice Tests in Marketing Courses

1979 ◽  
Vol 1 (2) ◽  
pp. 24-33 ◽  
Author(s):  
James R. McMillan
Keyword(s):  
Multiple Choice ◽  
Multiple Choice Questions ◽  
High Quality ◽  
Test Form ◽  
Multiple Choice Tests ◽  
Choice Tests ◽  
Classroom Evaluation ◽  
Quality Objective ◽  
Multiple Choice Items

Most educators agree that classroom evaluation practices need improvement. One way to improve testing is to use high-quality objective multiple-choice exams. Almost any understanding or ability which can be tested by another test form can also be tested by means of multiple-choice items. Based on a survey of 173 respondents, it appears that marketing teachers are disenchanted with multiple-choice questions and use them sparingly. Further, their limited use is largely in the introductory marketing course even though there are emerging pressures for universities to take a closer look at the quality of classroom evaluation at all levels.

scholarly journals Examination of the Quality of Multiple-choice Items on Classroom Tests

2011 ◽  
Vol 2 (2) ◽  
Author(s):  
David DiBattista ◽  
Laura Kurzawa
Keyword(s):  
Nous Avons ◽  
Test Scores ◽  
Item Analysis ◽  
Multiple Choice ◽  
Discriminatory Power ◽  
Multiple Choice Tests ◽  
Choice Tests ◽  
Discrimination Coefficient ◽  
Multiple Choice Items

Because multiple-choice testing is so widespread in higher education, we assessed the quality of items used on classroom tests by carrying out a statistical item analysis. We examined undergraduates’ responses to 1198 multiple-choice items on sixteen classroom tests in various disciplines. The mean item discrimination coefficient was +0.25, with more than 30% of items having unsatisfactory coefficients less than +0.20. Of the 3819 distractors, 45% were flawed either because less than 5% of examinees selected them or because their selection was positively rather than negatively correlated with test scores. In three tests, more than 40% of the items had an unsatisfactory discrimination coefficient, and in six tests, more than half of the distractors were flawed. Discriminatory power suffered dramatically when the selection of one or more distractors was positively correlated with test scores, but it was only minimally affected by the presence of distractors that were selected by less than 5% of examinees. Our findings indicate that there is considerable room for improvement in the quality of many multiple-choice tests. We suggest that instructors consider improving the quality of their multiple-choice tests by conducting an item analysis and by modifying distractors that impair the discriminatory power of items. Étant donné que les examens à choix multiple sont tellement généralisés dans l’enseignement supérieur, nous avons effectué une analyse statistique des items utilisés dans les examens en classe afin d’en évaluer la qualité. Nous avons analysé les réponses des étudiants de premier cycle à 1198 questions à choix multiples dans 16 examens effectués en classe dans diverses disciplines. Le coefficient moyen de discrimination de l’item était +0.25. Plus de 30 % des items avaient des coefficients insatisfaisants inférieurs à + 0.20. Sur les 3819 distracteurs, 45 % étaient imparfaits parce que moins de 5 % des étudiants les ont choisis ou à cause d’une corrélation négative plutôt que positive avec les résultats des examens. Dans trois examens, le coefficient de discrimination de plus de 40 % des items était insatisfaisant et dans six examens, plus de la moitié des distracteurs était imparfaits. Le pouvoir de discrimination était considérablement affecté en cas de corrélation positive entre un distracteur ou plus et les résultatsde l’examen, mais la présence de distracteurs choisis par moins de 5 % des étudiants avait une influence minime sur ce pouvoir. Nos résultats indiquent que les examens à choix multiple peuvent être considérablement améliorés. Nous suggérons que les enseignants procèdent à une analyse des items et modifient les distracteurs qui compromettent le pouvoir de discrimination des items.

scholarly journals The Advantages of Five-Option Multiple-Choice Items in Classroom Tests of Student Mastery

2020 ◽  
Vol 2 (4) ◽  
pp. p59
Author(s):  
Michael Joseph Wise
Keyword(s):  
Item Difficulty ◽  
Multiple Choice ◽  
Response Options ◽  
Course Content ◽  
Science Courses ◽  
The Past ◽  
Multiple Choice Items ◽  
Writing Tests ◽  
Do So

The effectiveness of multiple-choice (MC) items depends on the quality of the response options—particularly how well the incorrect options (“distractors”) attract students who have incomplete knowledge. It is often contended that test-writers are unable to devise more than two plausible distractors for most MC items, and that the effort needed to do so is not worthwhile in terms of the items’ psychometric qualities. To test these contentions, I analyzed students’ performance on 545 MC items across six science courses that I have taught over the past decade. Each MC item contained four distractors, and the dataset included more than 19,000 individual responses. All four distractors were deemed plausible in one-third of the items, and three distractors were plausible in another third. Each increase in plausible distractor led to an average of a 13% increase in item difficulty. Moreover, an increase in plausible distractors led to a significant increase in the discriminability of the items, with a leveling off by the fourth distractor. These results suggest that—at least for teachers writing tests to assess mastery of course content—it may be worthwhile to eschew recent skepticism and continue to attempt to write MC items with three or four distractors.

scholarly journals MULTIPLE CHOICE QUESTIONS;

2017 ◽  
Vol 24 (09) ◽  
pp. 1409-1414
Author(s):  
Muhammad Zafar Iqbal ◽  
Shumaila Irum ◽  
Muhammad Sohaib Yousaf
Keyword(s):  
Problem Solving ◽  
Descriptive Analysis ◽  
Multiple Choice ◽  
Development Program ◽  
Faculty Members ◽  
Multiple Choice Questions ◽  
Cross Sectional ◽  
Medical College ◽  
Item Writing

Objectives: The main objective of this study was to judge the quality of MCQs interms of their cognition level and item writing flaws, developed by the faculty of a public sectormedical college. Setting: This study was conducted in Sheikh Zayed Medical College, RahimYar Khan. Duration with Dates: Data was collected between June 2014 to March 2015 andthis study was completed in July 2016. Sample Size: A sample of 500 MCQs collected from25 faculty members were included in the study. Study Design: Quantitative method. StudyType: Cross sectional descriptive analysis. Material and Methods: This quantitative study wasconducted in Sheikh Zayed Medical College Rahim Yar Khan over six months period after theapproval of the study proposal. Every faculty member is supposed to write 25 MCQs in order tobecome supervisor. I collected 500 multiple choice questions from 25 faculty members readyfor submission to CPSP. The quality of all MCQs was checked in terms of item writing flawsand cognition level by panel of experts. Results: Absolute terms were observed in 10(2%),vague terms in 15(3%), implausible distracters in 75(15%), extra detail in correct option 15(3%),unfocused stem 63(12.6%), grammatical clues 39(7.8%), logical clues 18(3.6%), word repeats19(3.8%), >then one correct answer 21(4.2%), unnecessary information in stem 37(7.4%),lost sequence in data 15(3%), all of above16(3.2%), none of above 12(2.4%) and negativestem 23(4.6%). Cognition level l (recall) was observed in 363(72.6%), level ll (interpretation) in115(23%) and level lll (problem solving) in 22(4.4%) items. Total 378(75.6%) flaws were identifiedand four commonest flaws were implausible distracter 75(15%), unfocused stem 63(12.6%),grammatical clues 39(7.8%) and unnecessary information in stem 37(7.4%). Conclusion: It isconcluded that assessment of medical students is very demanding and need of the time. A wellconstructed,peer-reviewed single best type MCQ is best one to complete this task becauseof cost effectiveness, better reliability and computerized marking. It is very important to startfaculty development program in order to decrease the number of item writing flaws and improvecognition level towards problem solving and application of knowledge.

scholarly journals Analisis Kualitas Soal Try-Out Bahasa Inggris SMP di Kota Makassar Berdasarkan Teori Respon Butir

2022 ◽  
Author(s):  
Achmad Shabir
Keyword(s):  
High Schools ◽  
Item Response Theory ◽  
Item Response ◽  
Junior High ◽  
Item Difficulty ◽  
Multiple Choice ◽  
Junior High Schools ◽  
Response Theory ◽  
Multiple Choice Items

The aim of this study was to describe the quality of English testing intrument used in Try Out National Exam conducted by 40 Junior High Schools in Makassar-Sulawesi Selatan, using Item Response Theory (IRT) especially based on one (1PL), two (2PL), and three (3PL) parameters models. The data consist of 1.267 student’s answer sheets and the test has 50 multiple choice items. Results showed that the test is preferably good at both item difficulty and item dicrimination as suggest by 1PL and 2PL estimation. But at 3PL estimation, the test unable to discriminate students ability, while 38 % of the items were easy to guess.

scholarly journals Pengembangan Instrumen Asesmen Higher Order Thinking Skills (HOTS) pada Mata Pelajaran Bahasa Indonesia SMA dan SMK

2020 ◽  
Vol 3 (1) ◽  
pp. 102-113
Author(s):  
Sutami
Keyword(s):  
Item Difficulty ◽  
Multiple Choice ◽  
Thinking Skills ◽  
Assessment Instrument ◽  
Item Discrimination ◽  
Test Items ◽  
Tenth Grade ◽  
Multiple Choice Items ◽  
Essay Test

This research aims to produce a valid and reliable Indonesian language assessment instrument in form of HOTS test items and it describes the quality of HOTS test items to measure HOTS skill for the tenth grade of SMA and SMK students. This study was a research and development study adapted from Borg & Gall’s development model, including the following steps: research and information collection, planning, early product development, limited try out, revising the early product, field try out, and revising the final product. The research’s result shows that the HOTS assessment instrument in the form of HOTS test consists of 40 multiple choice items and 5 essay test items. Based on the judgment of the materials, construction, and language was valid and appropriate to be used. The reliability coefficients were 0.88 for the multiple-choice items, and 0.79 for essays. The multiple-choice items have the average difficulty 0.57 (average), the average of item discrimination 0.44 (good), and the distractors function well. The essay items have the average of item difficulty 0.60 (average) and the average of item discrimination 0.45 (good)

scholarly journals Kartlegging av læreres kunnskap er ikke enkelt

2015 ◽  
Vol 9 (1) ◽  
pp. 7 ◽  
Author(s):  
Reidar Mosvold ◽  
Janne Fauskanger
Keyword(s):  
Mathematical Knowledge ◽  
Multiple Choice ◽  
Mathematical Knowledge For Teaching ◽  
Measuring Instruments ◽  
Measurement Instruments ◽  
Knowledge For Teaching ◽  
Types Of Knowledge ◽  
Different Types ◽  
Multiple Choice Items

Læreres kunnskap har betydning for elevenes læring, og som et ledd i prosessen for styrking av kvaliteten i norsk skole har politikere i senere tid ytret ønske om å kartlegge læreres kunnskap. Kartlegging av læreres kunnskap er imidlertid ikke enkelt, og i denne artikkelen undersøker vi hvordan de samme lærernes undervisningskunnskap i matematikk kan se ut til å være forskjellig når den måles ved hjelp av oppgaver med ulikt format. Deltakerne i studien var 30 praktiserende lærere, og instrumentet besto av 28 flervalgsoppgaver med tilhørende åpne oppgaver. Sammenhengen mellom lærernes kunnskap slik den måles ved hjelp av flervalgsoppgavene og de åpne oppgavene er ikke alltid klar. Det er derfor nødvendig å tilnærme seg måling av læreres undervisnings-kunnskap i matematikk med stor varsomhet. Nærmere undersøkelser er nødvendig for å finne ut hva slags type kunnskap som kan måles med ulike typer oppgaver og hva slags måleinstrumenter som kan brukes til hvilke formål.Nøkkelord: Undervisningskunnskap i matematikk, kartlegging, måleinstrument-er, flervalgsoppgaver AbstractTeachers’ knowledge influence students’ learning, and as part of their attempts to increase the quality of Norwegian schools, politicians have announced their intentions to measure the knowledge of teachers. Measuring teacher knowledge is not straightforward, however, and this article investigates how a group of mathematics teachers’ knowledge might appear different when measured by items of different formats. The participants in this study were 30 in-service teachers, and the instrument consisted of 28 multiple-choice items with associated open-response items. The connection between the teachers’ know-ledge as measured by multiple-choice items and open-response items is not always straightforward. As an implication, it is necessary to approach measure-ment of teachers’ mathematical knowledge for teaching with great care. Further studies are necessary in order to investigate what types of knowledge can be measured by different types of items and how different kinds of measuring instruments can be used for particular purposes.Key words: Mathematical knowledge for teaching, measurement, instruments, multiple-choice items 

Sign in / Sign up

Export Citation Format

Share Document