Assessment Design with Automated Scoring in Mind

2020 ◽  
pp. 29-48
Author(s):  
Kristen DiCerbo ◽  
Emily Lai ◽  
Ventura Matthew
Keyword(s):  
Automated Scoring ◽  
Assessment Design

scholarly journals Assessing Complex Patterns of Student Resources and Behavior in the Large Scale

2019 ◽  
Vol 683 (1) ◽  
pp. 233-249 ◽  
Author(s):  
Knut Neumann ◽  
Horst Schecker ◽  
Heike Theyßen
Keyword(s):  
Large Scale ◽  
Reliability And Validity ◽  
Performance Tests ◽  
Automated Scoring ◽  
Large Scale Data ◽  
Assessment Design ◽  
Simulation Based ◽  
And Behavior ◽  
Physics Experiments ◽  
Scale Data

Large-scale assessments still focus on those aspects of students’ competence that can be evaluated using paper-and-pencil tests (or computer-administered versions thereof). Performance tests are considered costly due to administration and scoring, and, more importantly, they are limited in reliability and validity. In this article, we demonstrate how a sociocognitive perspective provides an understanding of these issues and how, based on this understanding, an argument-based approach to assessment design, interpretation, and use can help to develop comprehensive, yet reliable and valid, performance-based assessments of student competence. More specifically, we describe the development of a computer-administered, simulation-based assessment that can reliably and validly assess students’ competence to plan, perform, and analyze physics experiments at a large scale. Data from multiple validation studies support the potential of adopting a sociocognitive perspective and assessments based on an argument-based approach to design, interpretation, and use. We conclude by discussing the potential of simulations and automated scoring methods for reliable and valid performance-based assessments of student competence.

Introduction to the Biomass Project: An Illustration of Evidence-Centered Assessment Design and Delivery Capability

2003 ◽  
Author(s):  
Linda S. Steinberg ◽  
Robert J. Mislevy ◽  
Russell G. Almond ◽  
Andrew B. Baird ◽  
Cara Cahallan ◽  
...  
Keyword(s):  
Assessment Design

Embedding employability context through assessment design

Planet ◽  
2009 ◽  
Vol 21 (1) ◽  
pp. 64-67 ◽  
Author(s):  
Zoe Robinson
Keyword(s):  
Assessment Design

Visualizing chemistry teachers’ enacted assessment design practices to better understand barriers to “best practices”

2021 ◽  
Author(s):  
Adam G. L. Schafer ◽  
Victoria M. Borland ◽  
Ellen J. Yezierski
Keyword(s):  
High School ◽  
Best Practices ◽  
Formative Assessments ◽  
Learning Goals ◽  
Specific Method ◽  
High School Chemistry ◽  
Chemistry Teachers ◽  
Assessment Design ◽  
Teachers Beliefs ◽  
Representational Level

Even when chemistry teachers’ beliefs about assessment design align with literature-cited best practices, barriers can prevent teachers from enacting those beliefs when developing day-to-day assessments. In this paper, the relationship between high school chemistry teachers’ self-generated “best practices” for developing formative assessments and the assessments they implement in their courses are examined. Results from a detailed evaluation of several high school chemistry formative assessments, learning goals, and learning activities reveal that assessment items are often developed to require well-articulated tasks but lack either alignment regarding representational level or employ only one representational level for nearly all assessment items. Implications for the development of a chemistry-specific method for evaluating alignment are presented as well as implications for high school chemistry assessment design.

scholarly journals Using process data to understand problem-solving strategies and processes for drag-and-drop items in a large-scale mathematics assessment

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Yang Jiang ◽  
Tao Gong ◽  
Luis E. Saldivia ◽  
Gabrielle Cayton-Hodges ◽  
Christopher Agard
Keyword(s):  
Problem Solving ◽  
Time Allocation ◽  
Large Scale ◽  
Solution Process ◽  
Process Data ◽  
Mathematics Assessments ◽  
Assessment Design ◽  
Eighth Grade Students ◽  
Problem Solving Strategies ◽  
Drag And Drop

AbstractIn 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to study the cognitive and behavioral processes that underlie test-takers’ performances in ways that are not otherwise possible with the response data alone. In this exploratory study, we investigated the problem-solving processes and strategies applied by the nation’s fourth and eighth graders by analyzing the process data collected during their interactions with two technology-enhanced drag-and-drop items (one item for each grade) included in the first digital operational administration of the NAEP’s mathematics assessments. Results from this research revealed how test-takers who achieved different levels of accuracy on the items engaged in various cognitive and metacognitive processes (e.g., in terms of their time allocation, answer change behaviors, and problem-solving strategies), providing insights into the common mathematical misconceptions that fourth- and eighth-grade students held and the steps where they may have struggled during their solution process. Implications of the findings for educational assessment design and limitations of this research are also discussed.

Assessment design decisions in practice: Profile identification in approaches to assessment design

2021 ◽  
pp. 1-16
Author(s):  
Javier Fernández Ruiz ◽  
Ernesto Panadero ◽  
Daniel García- Pérez ◽  
Leire Pinedo
Keyword(s):  
Design Decisions ◽  
Assessment Design

scholarly journals How Can Scientific Literature Support Decision-Making in the Renovation of Historic Buildings? An Evidence-Based Approach for Improving the Performance of Walls

2021 ◽  
Vol 13 (4) ◽  
pp. 2266
Author(s):  
Valentina Marincioni ◽  
Virginia Gori ◽  
Ernst Jan de Place Hansen ◽  
Daniel Herrera-Avellanosa ◽  
Sara Mauri ◽  
...  
Keyword(s):  
Energy Performance ◽  
Decision Makers ◽  
Cultural Value ◽  
Correct Selection ◽  
Evidence Based ◽  
Historic Buildings ◽  
Indoor Environmental Quality ◽  
Building Stock ◽  
Advantages And Disadvantages ◽  
Assessment Design

Buildings of heritage significance due to their historical, architectural, or cultural value, here called historic buildings, constitute a large proportion of the building stock in many countries around the world. Improving the performance of such buildings is necessary to lower the carbon emissions of the stock, which generates around 40% of the overall emissions worldwide. In historic buildings, it is estimated that heat loss through external walls contributes significantly to the overall energy consumption, and is associated with poor thermal comfort and indoor air quality. Measures to improve the performance of walls of historic buildings require a balance between energy performance, indoor environmental quality, heritage significance, and technical compatibility. Appropriate wall measures are available, but the correct selection and implementation require an integrated process throughout assessment (planning), design, construction, and use. Despite the available knowledge, decision-makers often have limited access to robust information on tested retrofit measures, hindering the implementation of deep renovation. This paper provides an evidence-based approach on the steps required during assessment, design, and construction, and after retrofitting through a literature review. Moreover, it provides a review of possible measures for wall retrofit within the deep renovation of historic buildings, including their advantages and disadvantages and the required considerations based on context.

scholarly journals High-throughput behavioral screen in C. elegans reveals Parkinson’s disease drug candidates

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Salman Sohrabi ◽  
Danielle E. Mor ◽  
Rachel Kaletsky ◽  
William Keyes ◽  
Coleen T. Murphy
Keyword(s):  
Neural Network ◽  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
High Throughput ◽  
Potential Candidate ◽  
Automated Scoring ◽  
C Elegans ◽  
Drug Candidates ◽  
Branched Chain ◽  
Approved Drugs

AbstractWe recently linked branched-chain amino acid transferase 1 (BCAT1) dysfunction with the movement disorder Parkinson’s disease (PD), and found that RNAi-mediated knockdown of neuronal bcat-1 in C. elegans causes abnormal spasm-like ‘curling’ behavior with age. Here we report the development of a machine learning-based workflow and its application to the discovery of potentially new therapeutics for PD. In addition to simplifying quantification and maintaining a low data overhead, our simple segment-train-quantify platform enables fully automated scoring of image stills upon training of a convolutional neural network. We have trained a highly reliable neural network for the detection and classification of worm postures in order to carry out high-throughput curling analysis without the need for user intervention or post-inspection. In a proof-of-concept screen of 50 FDA-approved drugs, enasidenib, ethosuximide, metformin, and nitisinone were identified as candidates for potential late-in-life intervention in PD. These findings point to the utility of our high-throughput platform for automated scoring of worm postures and in particular, the discovery of potential candidate treatments for PD.

Sign in / Sign up

Export Citation Format

Share Document