scholarly journals Using process data to understand problem-solving strategies and processes for drag-and-drop items in a large-scale mathematics assessment

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Yang Jiang ◽  
Tao Gong ◽  
Luis E. Saldivia ◽  
Gabrielle Cayton-Hodges ◽  
Christopher Agard
Keyword(s):  
Problem Solving ◽  
Time Allocation ◽  
Large Scale ◽  
Solution Process ◽  
Process Data ◽  
Mathematics Assessments ◽  
Assessment Design ◽  
Eighth Grade Students ◽  
Problem Solving Strategies ◽  
Drag And Drop

AbstractIn 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to study the cognitive and behavioral processes that underlie test-takers’ performances in ways that are not otherwise possible with the response data alone. In this exploratory study, we investigated the problem-solving processes and strategies applied by the nation’s fourth and eighth graders by analyzing the process data collected during their interactions with two technology-enhanced drag-and-drop items (one item for each grade) included in the first digital operational administration of the NAEP’s mathematics assessments. Results from this research revealed how test-takers who achieved different levels of accuracy on the items engaged in various cognitive and metacognitive processes (e.g., in terms of their time allocation, answer change behaviors, and problem-solving strategies), providing insights into the common mathematical misconceptions that fourth- and eighth-grade students held and the steps where they may have struggled during their solution process. Implications of the findings for educational assessment design and limitations of this research are also discussed.

scholarly journals A Diagnostic Classification Analysis of Problem-Solving Competence using Process Data

2020 ◽  
Author(s):  
Peida Zhan ◽  
Xin Qiao
Keyword(s):  
Problem Solving ◽  
Student Assessment ◽  
Outcome Data ◽  
Diagnostic Classification ◽  
Process Data ◽  
Action Sequence ◽  
Problem Solving Skills ◽  
International Student Assessment ◽  
Latent Ability ◽  
Problem Solving Strategies

Process data refers to data recorded by computer-based assessments (CBA) that reflect respondents’ problem-solving processes and provide greater insight into how students solve problems, instead of merely how well they solve them. Using the rich information contained in process data, this study proposed an item-specific psychometric method for analyzing process data in order to comprehensively understand respondents’ problem-solving competence. By incorporating diagnostic classification into process data analysis, the proposed method cannot only estimate respondents’ problem-solving ability along a continuum, but can also classify respondents according to their problem-solving strategies. To illustrate the application and advantages of the proposed method, a Programme for International Student Assessment (PISA) problem-solving task was used. The results indicated that (a) the estimated latent classes provided more detailed diagnoses of respondents’ problem-solving strategies than the observed score classes; (b) although only one item was used, estimated higher-order latent ability reflected the respondents’ problem-solving ability more accurately than the estimated unidimensional latent ability taken from the outcome data; and (c) the interactions between problem-solving skills may follow the conjunctive condensation rule, which assumes that only when a respondent has mastered all the required problem-solving skills can the specific action sequence appear. Overall, the main conclusion drawn from this study was that using diagnostic classification is a feasible and promising method for analyzing process data.

Analyzing Process Data from Problem-Solving Items with N-Grams

2016 ◽  
pp. 750-777 ◽  
Author(s):  
Qiwei He ◽  
Matthias von Davier
Keyword(s):  
Problem Solving ◽  
Task Performance ◽  
Language Processing ◽  
Large Scale ◽  
Process Data ◽  
Action Sequence ◽  
Sequential Action ◽  
Item Development ◽  
Sequence Patterns ◽  
Computer Based

This chapter draws on process data recorded in a computer-based large-scale program, the Programme for International Assessment of Adult Competencies (PIAAC), to address how sequences of actions recorded in problem-solving tasks are related to task performance. The purpose of this study is twofold: first, to extract and detect robust sequential action patterns that are associated with success or failure on a problem-solving item, and second, to compare the extracted sequence patterns among selected countries. Motivated by the methodologies of natural language processing and text mining, we utilized feature selection models in analyzing the process data at a variety of aggregate levels and evaluated the different methodologies in terms of predictive power of the evidence extracted from process data. It was found that action sequence patterns significantly differed by performance groups and were consistent across countries. This study also demonstrated that the process data were useful in detecting missing data and potential mistakes in item development.

Metacognitive Feedback in Online Mathematical Discussion

2009 ◽  
pp. 794-806 ◽  
Author(s):  
Bracha Kramarski
Keyword(s):  
Problem Solving ◽  
Transfer Test ◽  
Online Discussion ◽  
Real Life ◽  
Solution Process ◽  
The Other ◽  
Mathematical Discussion ◽  
Eighth Grade Students ◽  
Practical Implications ◽  
Problem Solving Task

Effects of two online inquiry discussions in mathematics are compared: one inquiry was based on metacognitive feedback guidance (MFG) and the other with no such guidance (NG). The MFG students were exposed to the IMPROVE metacognitive questioning method that serves as cues for solving the problem and features of providing feedback (Kramarski & Mevarech, 2003). A total of80 eighth-grade students participated in the study. Students were asked to solve online a real-life task and provide feedback to their peers about the solution process. Results indicated that the MFG students significantly outperformed the NG students in online problem-solving task. The MFG students were engaged more in online discussion with respect to mathematical and metacognitive aspects. They also succeeded more on a delayed written mathematical transfer test. Theoretical and practical implications of the study are discussed.

The Teaching of Heuristic Problem-Solving Strategies in Elementary Calculus

1974 ◽  
Vol 5 (1) ◽  
pp. 36-46
Author(s):  
John F. Lucas
Keyword(s):  
Problem Solving ◽  
Solution Process ◽  
Problem Solver ◽  
Thought Processes ◽  
Heuristic Strategies ◽  
Efficient Performance ◽  
Elementary Calculus ◽  
Problem Solving Strategies ◽  
Human Problem ◽  
Heuristic Problem Solving

Research in the area of computer simulation of human thought processes indicates that the solution of every problem involves the element of search in a space of many alternatives (Newell, Shaw, & Simon, 1958). For human problem solving, the space may conceivably include infinitely many alternatives; however, humans appear to exhibit more efficient performance than exhaustive scanning or purely random trial and error. Human problem solving is characterized by heuristic strategies. These higher-order processes (heuristics) guide the search, enabling the problem solver to select from a reduced set of alternatives and to order his solution process in a sequence of steps; they are tentative rules of thumb that are based on experience or plausible assumptions and that apply generally to problems (Pylyshyn, 1963).

Scientific Problem Solving in a Virtual Laboratory: A Comparison Between Individuals and Pairs

2008 ◽  
Vol 67 (2) ◽  
pp. 71-83 ◽  
Author(s):  
Yolanda A. Métrailler ◽  
Ester Reijnen ◽  
Cornelia Kneser ◽  
Klaus Opwis
Keyword(s):  
Visual Search ◽  
Problem Solving ◽  
Process Data ◽  
Scientific Problem ◽  
Domain Specific ◽  
Scientific Problem Solving ◽  
Before And After ◽  
Verbal Data ◽  
Psychological Knowledge ◽  
Problem Solving Task

This study compared individuals with pairs in a scientific problem-solving task. Participants interacted with a virtual psychological laboratory called Virtue to reason about a visual search theory. To this end, they created hypotheses, designed experiments, and analyzed and interpreted the results of their experiments in order to discover which of five possible factors affected the visual search process. Before and after their interaction with Virtue, participants took a test measuring theoretical and methodological knowledge. In addition, process data reflecting participants’ experimental activities and verbal data were collected. The results showed a significant but equal increase in knowledge for both groups. We found differences between individuals and pairs in the evaluation of hypotheses in the process data, and in descriptive and explanatory statements in the verbal data. Interacting with Virtue helped all students improve their domain-specific and domain-general psychological knowledge.

IDENTIFICATION OF PROBLEM SOLVING STRATEGIES IN MATHEATICS AMONG HIGH SCHOOL STUDENTS

2017 ◽  
Vol 4 (36) ◽  
Author(s):  
J. Navaneetha Krishnan ◽  
P. Paul Devanesan
Keyword(s):  
High School ◽  
High School Students ◽  
Problem Solving ◽  
Sample Survey ◽  
Survey Method ◽  
Teaching Mathematics ◽  
School Students ◽  
Problem Solving Strategies ◽  
Achievement In Mathematics

The major aim of teaching Mathematics is to develop problem solving skill among the students. This article aims to find out the problem solving strategies and to test the students’ ability in using these strategies to solve problems. Using sample survey method, four hundred students were taken for this investigation. Students’ achievement in solving problems was tested for their Identification and Application of Problem Solving Strategies as a major finding, thirty one percent of the students’ achievement in mathematics is contributed by Identification and Application of Problem Solving Strategies.

Proses Berpikir Logis Siswa Sekolah Dasar Bertipe Kecerdasan Logis Matematis dalam Memecahkan Masalah Matematika

2014 ◽  
Vol 2 (2) ◽  
Author(s):  
Liska Yanti Pane ◽  
Kamid Kamid ◽  
Asrial Asrial
Keyword(s):  
Problem Solving ◽  
Learning Process ◽  
Qualitative Method ◽  
Logical Thinking ◽  
Think Aloud ◽  
Process Data ◽  
Thinking Process ◽  
Problem Solving Process ◽  
Problem Solving Strategy ◽  
Solving Strategy

This research aims to describe logical thinking process of a logical-mathematical intelligence student. We employ qualitative method to disclose the subject’s learning process. Data are collected by interview and modified think aloud methods. The results show that subject has capability to find and organize problems and data correctly. Subject describes conditions that are needed to do the steps of problem solving strategy. The steps are done systematically until the end of problem solving process.

Sign in / Sign up

Export Citation Format

Share Document