Development of a multiple-choice problem-solving categorization test for assessment of student knowledge structure

Instructors in higher education frequently employ examinations composed of problem-solving questions to assess student knowledge and learning. But are student scores on these tests reliable? Surprisingly few have researched this question empirically, arguably because of perceived limitations in traditional research methods. Furthermore, many believe multiple choice exams to be a more objective, reliable form of testing students than any other type. We question this wide-spread belief. In a series of empirical studies in 8 classes (401 students) in a finance course, we used a methodology based on three key elements to examine these questions: A true experimental design, more appropriate estimation of exam score reliability, and reliability confidence intervals. Internal consistency reliabilities of problem-solving test scores were consistently high (all > .87, median = .90) across different classes, students, examiners, and exams. In contrast, multiple-choice test scores were less reliable (all < .69). Recommendations are presented for improving the construction of exams in higher education.

Download Full-text

Guessing in Multiple-Choice Tests as a Judgment and Choice Problem

PsycEXTRA Dataset ◽

10.1037/e683312011-118 ◽

1999 ◽

Cited By ~ 1

Author(s):

Yigal Attali

Keyword(s):

Multiple Choice ◽

Choice Problem ◽

Multiple Choice Tests ◽

Choice Tests

Download Full-text

Problems and potentialities of e-Learning for regular undergraduate courses in emergency medicine

Revista Brasileira de Educação Médica ◽

10.1590/s0100-55022010000300016 ◽

2010 ◽

Vol 34 (3) ◽

pp. 452-458 ◽

Cited By ~ 2

Author(s):

William Rafaelo Schlinkert ◽

Sandro Scarpelini ◽

Antonio Pazin-Filho

Keyword(s):

Medical Students ◽

Multiple Choice ◽

First Aid ◽

Student Knowledge ◽

Multiple Choice Tests ◽

Reading Material ◽

Learning Techniques ◽

Choice Tests ◽

E Learning ◽

The Impact

BACKGROUND: E-learning techniques are spreading at great speed in medicine, raising concerns about the impact of adopting them. Websites especially designed to host courses are becoming more common. There is a lack of evidence that these systems could enhance student knowledge acquisition. GOAL: To evaluate the impact of using dedicated-website tools over cognition of medical students exposed to a first-aid course. METHODS: Prospective study of 184 medical students exposed to a twenty-hour first-aid course. We generated a dedicated-website with several sections (lectures, additional reading material, video and multiple choice exercises). We constructed variables expressing the student's access to each section. The evaluation was composed of fifty multiple-choice tests, based on clinical problems. We used multiple linear regression to adjust for potential confounders. RESULTS: There was no association of website intensity of exposure and the outcome - beta-coeficient 0.27 (95%CI - 0.454 - 1.004). These findings were not altered after adjustment for potential confounders - 0.165 (95%CI -0.628 - 0.960). CONCLUSION: A dedicated website with passive and active capabilities for aiding in person learning had not shown association with a better outcome.

Download Full-text

MEASURING STUDENTS’ PROBLEM-SOLVING SKILLS ON DIRECT CURRENT CIRCUITS WITH MULTIPLE REPRESENTATIONS

Humanities & Social Sciences Reviews ◽

10.18510/hssr.2020.8378 ◽

2020 ◽

Vol 8 (3) ◽

pp. 725-736

Author(s):

Maria Dewati ◽

A. Suparmi ◽

Widha Sunarno ◽

Sukarmin ◽

C. Cari

Keyword(s):

Problem Solving ◽

Physics Education ◽

Multiple Choice ◽

Multiple Representations ◽

Assessment Instruments ◽

Multiple Choice Questions ◽

Multiple Representation ◽

Problem Solving Skills ◽

Electrical Circuits ◽

Physics Problems

Purpose of study: This study aims to measure the level of students' problem-solving skills, using assessment instruments in the form of multiple-choice tests based on the multiple representation approach on DC electrical circuits. Methodology: This research is a quantitative descriptive involving 46 students of physics education. Students are asked to solve the problem of DC electrical circuits on 12 multiple choice questions with open reasons, involving verbal, mathematical, and picture representations. Data were analyzed by determining means and standard deviations. Main findings: The results of the study showed that there were 3 levels of students' problem-solving skills, namely 7 (15%) students in the high category, 22 (48%) students in the medium category and 17 (37%) students in the low category. Applications of this study: The implication of this research is to continuously develop assessment instruments based on multiple representations in the form of various types of tests, to help students improve their conceptual understanding, so students can solve physics problems correctly. The novelty of this study: Researchers explain the right way to solve physics problems, 1) students are trained to focus on identifying problems, 2) students are accustomed to planning solutions using a clear approach, to build an understanding of concepts, 3) students are directed to solve problems accordingly with understanding the concepts they have built.

Download Full-text

Achievement in Online vs. Traditional Classes

Encyclopedia of Distance Learning, Second Edition ◽

10.4018/978-1-60566-198-8.ch004 ◽

2011 ◽

pp. 29-32 ◽

Cited By ~ 1

Author(s):

E. Lea Witta

Keyword(s):

Problem Solving ◽

Multiple Choice ◽

Extended Period ◽

Student Needs ◽

Class Session ◽

Universities And Colleges ◽

Working Classes ◽

Class Scheduling ◽

Time Frames ◽

Seat Time

Universities and colleges constantly attempt to address student needs by offering courses in various time frames. Because it is difficult for teachers to enroll and attend classes during the “normal” two- or three-day format for 15 weeks, graduate classes in education are typically offered as a one evening per week class. In summer, when teachers usually are not working, classes may be offered in an alternative format meeting for longer periods of time in each class session but for fewer weeks. There are, however, questions concerning the changes in class scheduling. Although the seat time in an eightweek extended period class is equivalent to a 15-week class, are the learning outcomes equivalent? According to Rayburn and Rayburn (1999), if only responses on multiple choice Accounting exams were considered, there was no effect of class length. If, however, problem solving was also considered, there was a statistically significant effect based on length of the class.

Download Full-text