Proficiency-aware systems

In an increasingly digital world, intelligent systems support us in accomplishing many everyday tasks. With the proliferation of affordable sensing devices, inferring user states from collected physiological data paves the way to tailor-made adaptation. While estimating a user’s abilities is technically possible, such proficiency assessments are rarely employed to benefit the user’s task reflection. In our work, we investigate how to model and design for proficiency estimation as part of context-aware systems. In this paper, we present the definition and conceptual architecture of proficiency-aware systems. The concept is not only applicable to current adaptive systems but provides a stepping stone for systems which actively aid in developing user proficiency during interaction.

The Sensitivity of Value-Added Modeling to the Creation of a Vertical Score Scale

The purpose of this study was to evaluate the sensitivity of growth and value-added modeling to the way an underlying vertical score scale has been created. Longitudinal item-level data were analyzed with both student- and school-level identifiers for the entire state of Colorado between 2003 and 2006. Eight different vertical scales were established on the basis of choices made for three key variables: the item response theory modeling approach, the calibration approach, and the student proficiency estimation approach. Each scale represented a methodological approach that was psychometrically defensible. Longitudinal values from each scale were used as the outcome in a commonly used value-added model (the “layered model” popularized by William Sanders) as a means of estimating school effects. Our findings suggest that while the ordering of estimating school effects is insensitive to the underlying vertical scale, the precision of such value-added estimates can be quite sensitive to the combinations of choices made in the creation of the scale.