Generalizability theory and the fair and valid assessment of linguistic minorities

2013 ◽  
Vol 19 (2-3) ◽  
pp. 245-263 ◽  
Author(s):  
Guillermo Solano-Flores ◽  
Min Li
Author(s):  
Simon M. Breil ◽  
Boris Forthmann ◽  
Mitja D. Back

Abstract. Social skills (e.g., persuading others, showing compassion, staying calm) are of key importance in work and education settings. Accordingly, the goal of many selection processes is to identify candidates who excel in desired skills. For this, high-fidelity simulations such as assessment centers (ACs) are regarded as ideal procedures because they can be used to evoke, observe, and evaluate candidates’ actual behavior. However, research has repeatedly shown that observed performance differences in ACs are not sufficiently driven by the specific skill dimensions that are defined for assessment. Building on multiple speed assessments and incorporating insights from behavioral personality science, we offer an alternative approach for the reliable and valid assessment of distinct social skills. We hereby (a) selected skills on the basis of a bottom-up analysis of observable and distinguishable interpersonal behaviors and (b) specifically designed exercises around these skills (i.e., one skill per exercise, multiple exercises per skill). Here, we present the initial results of this newly developed procedure across three samples in a high-stakes selection context ( N = 589). Generalizability theory analyses showed that a substantial amount of variance in assessor ratings could be attributed to the selected skills. This underlines the importance of more behaviorally focused selection procedures.


2021 ◽  
Author(s):  
Simon Mats Breil ◽  
Boris Forthmann ◽  
Mitja Back

Social skills (e.g., persuading others, showing compassion, staying calm) are of key importance in work and education settings. Accordingly, the goal of many selection processes is to identify candidates who excel in desired skills. For this, high-fidelity simulations such as assessment centers (ACs) are regarded as ideal procedures because they can be used to evoke, observe, and evaluate candidates’ actual behavior. However, research has repeatedly shown that observed performance differences in ACs are not sufficiently driven by the specific skill dimensions that are defined for assessment. Building on multiple speed assessments and incorporating insights from behavioral personality science, we offer an alternative approach for the reliable and valid assessment of distinct social skills. We hereby (a) selected skills on the basis of a bottom-up analysis of observable and distinguishable interpersonal behaviors and (b) specifically designed exercises around these skills (i.e., one skill per exercise, multiple exercises per skill). Here, we present initial results of this newly developed procedure across three samples in a high-stakes selection context (N = 589). Generalizability theory analyses showed that a substantial amount of variance in assessor ratings could be attributed to the selected skills. This underlines the importance of more behaviorally focused selection procedures.


2020 ◽  
Vol 63 (6) ◽  
pp. 1947-1957
Author(s):  
Alexandra Hollo ◽  
Johanna L. Staubitz ◽  
Jason C. Chow

Purpose Although sampling teachers' child-directed speech in school settings is needed to understand the influence of linguistic input on child outcomes, empirical guidance for measurement procedures needed to obtain representative samples is lacking. To optimize resources needed to transcribe, code, and analyze classroom samples, this exploratory study assessed the minimum number and duration of samples needed for a reliable analysis of conventional and researcher-developed measures of teacher talk in elementary classrooms. Method This study applied fully crossed, Person (teacher) × Session (samples obtained on 3 separate occasions) generalizability studies to analyze an extant data set of three 10-min language samples provided by 28 general and special education teachers recorded during large-group instruction across the school year. Subsequently, a series of decision studies estimated of the number and duration of sessions needed to obtain the criterion g coefficient ( g > .70). Results The most stable variables were total number of words and mazes, requiring only a single 10-min sample, two 6-min samples, or three 3-min samples to reach criterion. No measured variables related to content or complexity were adequately stable regardless of number and duration of samples. Conclusions Generalizability studies confirmed that a large proportion of variance was attributable to individuals rather than the sampling occasion when analyzing the amount and fluency of spontaneous teacher talk. In general, conventionally reported outcomes were more stable than researcher-developed codes, which suggests some categories of teacher talk are more context dependent than others and thus require more intensive data collection to measure reliably.


Author(s):  
Steven M. Doettl

It has been widely accepted that the assessment of balance after concussion plays a large role in determining deficit. Qualitative balance assessments have been an established piece of the post-injury assessment as a clinical behavioral marker of concussion for many years. Recently more specific guidelines outlining the role of balance evaluation in concussion identification and management have been developed as part of concussion management tools. As part of the ongoing development of concussions protocols, quantitative assessment of balance function following concussion has also been identified to have an important role. Frequently imbalance and dizziness reported following concussion is assumed to be associated with post-concussion syndrome (PCS). While imbalance and dizziness are common complaints in PCS, they can also be a sign of additional underlying pathology. In cases of specific dizziness symptoms or limited balance recovery beyond the initial post-concussive period, a quantitative vestibular assessment may also be needed. Electronystagmography and videonystagmography (ENG/VNG), rotary chair testing (RCT), and vestibular evoked myogenic potentials (VEMPs) have all been identified as valid assessment tools for vestibular dysfunction following traumatic brain injury (TBI). The assessment of balance and dizziness following sports-related concussions is an integral piece of the puzzle for removal from play, assessment of severity, and management.


1981 ◽  
Vol 36 (10) ◽  
pp. 1078-1085 ◽  
Author(s):  
Esteban L. Olmedo

2007 ◽  
Author(s):  
Natalie Durand-Bush ◽  
John H. Salmela

Sign in / Sign up

Export Citation Format

Share Document