scholarly journals Improving the Precision of Ability Estimates Using Time-On-Task Variables: Insights From the PISA 2012 Computer-Based Assessment of Mathematics

2021 ◽  
Vol 12 ◽  
Author(s):  
Denise Reis Costa ◽  
Maria Bolsinova ◽  
Jesper Tijmstra ◽  
Björn Andersson
Keyword(s):  
Response Times ◽  
Time On Task ◽  
Additional Information ◽  
Ability Estimates ◽  
Pisa 2012 ◽  
Country Level ◽  
Item Responses ◽  
Notable Increase ◽  
Computer Based ◽  
Log File

Log-file data from computer-based assessments can provide useful collateral information for estimating student abilities. In turn, this can improve traditional approaches that only consider response accuracy. Based on the amounts of time students spent on 10 mathematics items from the PISA 2012, this study evaluated the overall changes in and measurement precision of ability estimates and explored country-level heterogeneity when combining item responses and time-on-task measurements using a joint framework. Our findings suggest a notable increase in precision with the incorporation of response times and indicate differences between countries in how respondents approached items as well as in their response processes. Results also showed that additional information could be captured through differences in the modeling structure when response times were included. However, such information may not reflect the testing objective.

scholarly journals An Attention-Based Diffusion Model for Psychometric Analyses

Psychometrika ◽  
2021 ◽  
Author(s):  
Udo Boehm ◽  
Maarten Marsman ◽  
Han L. J. van der Maas ◽  
Gunter Maris
Keyword(s):  
Diffusion Model ◽  
Cognitive Process ◽  
Functional Form ◽  
Response Times ◽  
Psychometric Tests ◽  
Psychometric Analyses ◽  
Item Responses ◽  
Computer Based ◽  
Psychometric Models ◽  
Source Of Information

AbstractThe emergence of computer-based assessments has made response times, in addition to response accuracies, available as a source of information about test takers’ latent abilities. The development of substantively meaningful accounts of the cognitive process underlying item responses is critical to establishing the validity of psychometric tests. However, existing substantive theories such as the diffusion model have been slow to gain traction due to their unwieldy functional form and regular violations of model assumptions in psychometric contexts. In the present work, we develop an attention-based diffusion model based on process assumptions that are appropriate for psychometric applications. This model is straightforward to analyse using Gibbs sampling and can be readily extended. We demonstrate our model’s good computational and statistical properties in a comparison with two well-established psychometric models.

Increased Behavioral Sensitivity to Repeated Experiences of Punishment in Children With ADHD: Experimental Studies Using the Matching Law

2021 ◽  
Vol 25 (12) ◽  
pp. 1665-1665
Author(s):  
Emi Furukawa ◽  
Brent Alsop ◽  
Shizuka Shimabukuro ◽  
Paula Sowerby ◽  
Stephanie Jensen ◽  
...  
Keyword(s):  
Response Times ◽  
Experimental Studies ◽  
The United States ◽  
Matching Law ◽  
Limited Attention ◽  
Time On Task ◽  
Children With Adhd ◽  
Behavioral Sensitivity ◽  
Operant Task ◽  
English Speaking

Background: Research on altered motivational processes in ADHD has focused on reward. The sensitivity of children with ADHD to punishment has received limited attention. We evaluated the effects of punishment on the behavioral allocation of children with and without ADHD from the United States, New Zealand, and Japan, applying the generalized matching law. Methods: Participants in two studies (Furukawa et al., 2017, 2019) were 210 English-speaking (145 ADHD) and 93 Japanese-speaking (34 ADHD) children. They completed an operant task in which they chose between playing two simultaneously available games. Rewards became available every 10 seconds on average, arranged equally across the two games. Responses on one game were punished four times as often as responses on the other. The asymmetrical punishment schedules should bias responding to the less punished alternative. Results: Compared with controls, children with ADHD from both samples allocated significantly more responses to the less frequently punished game, suggesting greater behavioral sensitivity to punishment. For these children, the bias toward the less punished alternative increased with time on task. Avoiding the more punished game resulted in missed reward opportunities and reduced earnings. English-speaking controls showed some preference for the less punished game. The behavior of Japanese controls was not significantly influenced by the frequency of punishment, despite slowed response times after punished trials and immediate shifts away from the punished game, indicating awareness of punishment. Conclusion: Punishment exerted greater control over the behavior of children with ADHD, regardless of their cultural background. This may be a common characteristic of the disorder. Avoidance of punishment led to poorer task performance. Caution is required in the use of punishment, especially with children with ADHD. The group difference in punishment sensitivity was more pronounced in the Japanese sample; this may create a negative halo effect for children with ADHD in this culture.

scholarly journals The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment.

2014 ◽  
Vol 106 (3) ◽  
pp. 608-626 ◽  
Author(s):  
Frank Goldhammer ◽  
Johannes Naumann ◽  
Annette Stelter ◽  
Krisztina Tóth ◽  
Heiko Rölke ◽  
...  
Keyword(s):  
Problem Solving ◽  
Task Difficulty ◽  
Large Scale ◽  
Time On Task ◽  
Large Scale Assessment ◽  
Scale Assessment ◽  
Task Effect ◽  
Computer Based

scholarly journals A Simple Diffusion Model for Psychometric Analyses

2019 ◽  
Author(s):  
Udo Boehm ◽  
Maarten Marsman ◽  
Han van der Maas ◽  
Gunter Maris
Keyword(s):  
Diffusion Model ◽  
Functional Form ◽  
Response Times ◽  
Simple Diffusion ◽  
Regression Framework ◽  
Psychometric Analyses ◽  
Computer Based ◽  
Simple Diffusion Model ◽  
Source Of Information ◽  
Latent Regression

The emergence of computer-based assessments has made response times, in addition to response accuracies, available as a source of information about test takers’ latent abilities. The predominant approach to jointly account for response times and accuracies are statistical models. Substantive approaches such as the diffusion model, on the other hand, have been slow to gain traction due to their unwieldy functional form. In the present work we show how a single simplifying assumption yields a highly tractable diffusion model. This simple diffusion model is straightforward to analyse using Gibbs sampling and can be readily extended with a latent regression framework. We demonstrate the superior computational efficiency of our model compared to the standard diffusion model in a simulation study and showcase the theoretical merit of our model in an example application.

The Effects of Information and Augmentative Communication Technique on Attitudes Toward Nonspeaking Individuals

1991 ◽  
Vol 34 (1) ◽  
pp. 19-26 ◽  
Author(s):  
Carole Wood Gorenflo ◽  
Daniel W. Gorenflo
Keyword(s):  
Physical Disabilities ◽  
Dependent Measure ◽  
Augmentative Communication ◽  
Factual Information ◽  
Additional Information ◽  
Communication Techniques ◽  
Communication Aid ◽  
Communication Technique ◽  
Computer Based ◽  
The Individual

This study investigated the effects of printed factual information and three augmentative communication techniques on attitudes of nondisabled individuals toward nonspeaking persons with physical disabilities. Employing a 3 (augmentative communication techniques) x 2 (presence/absence of information) factorial design, subjects viewed a videotape depicting a nonspeaking adult having a conversation with a normal-speaking individual. Subjects in Condition 1 viewed a tape depicting the nonspeaking individual using unaided communication techniques; Condition 2 subjects viewed the individual using an alphabet board; subjects in Condition 3 viewed the individual using a computer-based voice output communication aid (VOCA). A scale assessing attitudes toward nonspeaking persons, the Attitudes Toward Nonspeaking Persons Scale (ATNP), was developed and validated for purposes of this study and was employed as the dependent measure. Results revealed that subjects expressed more favorable attitudes when provided with the additional information concerning the nonspeaking individual. Attitude favorability also increased with the sophistication of the augmentative communication technique.

The Compassionate Use Advisory Committee (CompAC) for development of pre-approval access to investigational drugs.

2017 ◽  
Vol 35 (15_suppl) ◽  
pp. e18001-e18001
Author(s):  
Edith P. Mitchell ◽  
Arthur Caplan ◽  
Alison Bateman-House ◽  
Amrit Ray
Keyword(s):  
Clinical Information ◽  
Risk Profile ◽  
Alternative Therapies ◽  
Rapid Review ◽  
Prior Therapy ◽  
Additional Information ◽  
Country Level ◽  
Access Channel ◽  
Investigational Drugs ◽  
Patient Advocates

e18001 Background: In May 2015, CompAC was formed at NYU to provide guidance fairly and transparently on allocating a limited supply of , Daratumumab, to patients outside of clinical trials. Janssen, manufacturer of Daratumumab, was receiving numerous pre-approval requests for access to the drug. Daratumumab (Darzalex) was approved in the U.S. for treatment of patients with multiple myeloma who have received at least one prior therapy and approved by EMA for monotherapy. The SPR program was phased out based on country level approvals, with requests and submissions managed in a regulatory and legal environment.Methods: CompAC was 10 medical experts, bioethicists, and patient advocates; ethical and medical principles for review and process guidelines developed. Weekly meetings were held to consider SPR requests, with 3 members voting each week. After U.S. approval, requests for drug were considered only from countries where Janssen was seeking but had not gained approval. Recommendations were conveyed weekly for final decisions. Results: A total of 331 cases were received by Janssen of which 5 withdrew and 2 expired, leaving 324 cases for review. Of these, 180 were sent to CompAC and 163 were recommended by CompAC for treatment. Janssen approved all 163. CompAC recommended declining 17 of these cases; Janssen declined 15 (2 cases were approved based on additional information provided). Of the 144 patients excluded by Janssen before CompAC review, 27 were due to low benefit/risk profile, 17 provided incomplete clinical information, 42 for other variables, 46 received alternative therapies, 10 for referral to another pre-approval access channel, and 2 commercially available drug. Of the cases that CompAC recommended declining, 11 were due to low benefit/risk profile, for 3 alternative therapies available, and remaining 3 for miscellaneous reasons. Requests were received from 13 countries, the majority from outside the U.S.Conclusions: CompAC was successfully implemented and provided rapid review of SPR requests for pre-approval Daratumumab access. Janssen is extending CompAC to other relevant assets. Other companies are considering the model.

scholarly journals E-Mail-Related Problem-Solving Behaviors Across Age Groups: An Analysis of Log File Data

2020 ◽  
Vol 4 (Supplement_1) ◽  
pp. 410-411
Author(s):  
Taka Yamashita ◽  
Leah Saal ◽  
Roberto Millar ◽  
Shalini Sahoo ◽  
Phyllis Cummins
Keyword(s):  
Older Adults ◽  
Problem Solving ◽  
Odds Ratio ◽  
Related Problem ◽  
Age Groups ◽  
Online Communication ◽  
Software Environment ◽  
Problem Solving Skills ◽  
Computer Based ◽  
Log File

Abstract Email is one of the most common and useful online communication tools. However, older adults tend to have difficulties fully taking advantage of email. Organizing the information in the email software environment and sending a message to selected recipients are examples of common email-related problem-solving. To date, little data are available to understand the email-related problem-solving behaviors of older adults. Nationally representative survey data and log-file data of the U.S. adults (n = 1,341) are derived from the 2012 Program for International Assessment of Adult Competencies (PIAAC), which provides computer-based assessment data on problem-solving skills. The PIAAC respondents used the computer device and solved the problems in the simulated email environment. Descriptive summary showed that those 55 years and older took longer (169 seconds), referred to the help menu (15%) and used the cancel button (26%) more often than younger age groups (e.g., age 25-34; 103 seconds, 3% and 17%, respectively) in one of the tasks. Additionally, binary logistic regression showed that taking longer time (odds-ratio = 0.99, p < 0.05) and using the help menu (odds-ratio = 0.85, p < 0.05) were associated with the incorrect answer to the email problem-solving, although the findings varied across different types of problems. These unique findings from the combination of survey and log-file data analyses suggested that some older adults may benefit from the training for common email-related problems rather than teaching themselves. Detailed descriptions of computer-based assessment log file data and other results are also evaluated in this study.

scholarly journals Editorial

2011 ◽  
Vol 7 (1) ◽  
Author(s):  
Jonathan Darby ◽  
David Squires
Keyword(s):  
Teaching Practice ◽  
Response Times ◽  
Fruit Fly ◽  
Computer Based Learning ◽  
Lookup Tables ◽  
Biology I ◽  
Computer Based ◽  
Different Strains

My first encounter with computer-based learning was in 1975 when I trialled packages produced by the Chelsea Science Simulation Project while on teaching practice. Access to the minicomputer running the packages was via modem and teletype, and response times were measured in minutes rather than seconds. Three of the four packages were little more than computer-based lookup tables but the fourth, despite the severe limitations of the technology, inspired me to take computers seriously as tools for learning. The application was a simulation of crossing different strains of fruit fly, a technique beloved of geneticists but impractical for school biology. I was impressed by the use of a computer to achieve something highly educational that was impractical by any other means.DOI:10.1080/0968776990070101

Sign in / Sign up

Export Citation Format

Share Document