Authentic Project-Based Learning in Times of Crisis

The shift to an online learning environment during the Covid-19 outbreak has not been conducive enough for either active learning or fair assessment procedures that yield valid results. Accordingly, this research paper aims to investigate the extent to which Authentic Project-Based Learning engages students and hence provides a valid assessment of students’ genuine performance and proficiency during online learning in EFL communication and Intensive English courses. Using a mixed-methods design, the researchers employed the self-completion questionnaires and focus group interviews to collect their data from their 100-purposively –selected participants. The findings showed promising results in terms of engaging students in the learning process, curbing online cheating, and ensuring reliable and valid assessments of students’ performance. These results are significant in providing a pedagogical approach to be adopted by EFL educators during and after the pandemic.

Tool-Box- Standardized Tool-Box for the assessment of the stress-physiological consequences of digitalization for medical practitioners

A valid assessment of digitization-associated stress appears to be crucial for the successful and health-promoting design of digitization processes in the healthcare sector. The present Tool-Box aims to contribute to this goal by providing an overview of central methods for assessing stress and associated constructs in the context of digitization processes, as well as their advantages and disadvantages. The methods included in the Tool-Box were selected based on an extensive review of existing literature on assessment of stress and associated constructs in the context of digitalization processes and beyond. In addition, we conducted expert interviews and pilot studies for the final selection and shaping of the Tool-Box.The target groups of our Tool-Box are researchers as well as all individuals which are involved in the design and implementation of digital innovations within the health care sector: monitoring digitalization-associated stress throughout the entire implementation process with methodologically sound tools might be one of the key factors for successful digital transformations.

Quantifying ADHD Symptoms in Open-Ended Everyday Life Contexts With a New Virtual Reality Task

Objective: To quantify goal-directed behavior and ADHD symptoms in naturalistic conditions, we developed a virtual reality task, EPELI (Executive Performance in Everyday LIving), and tested its predictive, discriminant and concurrent validity. Method: We collected EPELI data, conventional neuropsychological task data, and parent-ratings of executive problems and symptoms in 38 ADHD children and 38 typically developing controls. Results: EPELI showed predictive validity as the ADHD group exhibited higher percentage of irrelevant actions reflecting lower attentional-executive efficacy and more controller movements and total game actions, both indicative of hyperactivity-impulsivity. Further, the five combined EPELI measures showed excellent discriminant validity (area under curve 88 %), while the correlations of the EPELI efficacy measure with parent-rated executive problems (r = .57) and ADHD symptoms (r = .55) pointed to its concurrent validity. Conclusion: We provide a proof-of-concept validation for a new virtual reality tool for ecologically valid assessment of ADHD symptoms.

Assessment of Oral Skills in Adolescents

There is broad consensus on the need to foster oral skills in middle school due to their inherent importance and because they serve as a tool for learning and acquiring other competences. In order to facilitate the assessment of communicative competence, we hereby propose a model which establishes five key dimensions for effective oral communication: interaction management; multimodality and prosody; textual coherence and cohesion; argumentative strategies; and lexicon and terminology. Based on this model, we developed indicators to measure the proposed dimensions, thus generating a self-report tool to assess oral communication in middle school. Following an initial study conducted with 168 students (mean age = 12.47 years, SD = 0.41), we selected 22 items with the highest discriminant power, while in a second study carried out with a sample of 960 students (mean age 14.11 years, SD = 0.97), we obtained evidence concerning factorial validity and the relationships between oral skills, emotional intelligence and metacognitive strategies related to metacomprehension. We concluded that the proposed model and its derived measure constitute an instrument with good psychometric properties for a reliable and valid assessment of students’ oral competence in middle school.

How Older Adults Redefine Items on the PROMIS-57 Profile Patient-Reported Outcome Measure

PROMIS® measures provide valid assessment of patient-reported outcomes, but have not been validated in older adults (especially aged 80+), including those with cognitive impairment. The objective of this project was to study how age-related role change effected the understanding of items on the PROMIS-57 Profile using cognitive interviews. Cognitive interviews were conducted with 38 adults, age 65+ with MoCA scores 10-30. Preliminary codes were created and then codes were added or modified as needed. Each interview was coded independently by two coders with differences resolved by consensus. The sample was 47% age 80+, 45% female, 18% African American, and 32% had a MoCA score between 10-17 (cognitively impaired). Thematic analysis of codes indicated that participants endorsed little or no impairment when they adapt to physical or cognitive disabilities by using economic means, instrumental support, physical aids, or by reducing activities. One respondent using grocery delivery services described no difficulty running errands or shopping. Another respondent reported no difficulty walking 15 minutes because they use a cane. Some reported no difficulty engaging in social roles when they restricted their activities due to disability or lack of appropriate social activities. Age-related changes effected responses on PROMIS-57 items. Findings suggest that age-related changes bias individuals to indicate less physical and cognitive impairment than their actual level of function. Physical functioning items show more bias for individuals with financial or instrument support, and social role items show more bias for those with restricted social networks.

Interactive Virtual Reality versus Vignette-Based Assessment of Children’s Aggressive Social Information Processing

This study examined whether interactive Virtual Reality (VR) provides a more ecologically valid assessment of children’s aggressive social information processing (SIP) and aggressive responses than a standard vignette-based assessment. We developed a virtual classroom where children could meet and play games with virtual peers. Participants were boys (N = 184; ages 7–13) from regular education and special education for children with disruptive behavior problems. They reported on their SIP in four scenarios (i.e., two instrumental gain and two provocation scenarios) presented through both interactive VR and vignettes. Teachers reported on children’s real-life aggressive behavior and reactive and proactive motives for aggression. Results demonstrated that children found the interactive VR assessment more emotionally engaging and immersive than the vignette-based assessment. Moreover, compared to vignettes, the interactive VR assessment evoked higher levels of aggressive SIP and responses in provocation scenarios only. Results supported the enhanced predictive validity of the interactive VR assessment of children’s aggressive SIP and responses, which predicted children’s real-life aggression above and beyond the vignette-based assessment with 2 to 12% additional explained variance. Similar results were found for children’s real-life reactive and proactive motives for aggression, with 3 to 12% additional variance explained by interactive VR above and beyond vignettes. Interactive VR did not, however, evoke larger individual differences (i.e., variances) in children’s aggressive SIP and responses than vignettes. Together, these findings suggest that interactive VR provides a more ecologically valid method to assess children’s aggressive SIP and responses than hypothetical vignettes.

Early Attachment Network to Multiple Caregivers: A Brief History of Research and Assessment Models

Early attachment has been commonly hypothesized to predict children’s future developmental outcomes, and robust evidence relying on assessments of single caregiver-child attachment patterns has corroborated this hypothesis. Nevertheless, most often children are raised by multiple caregivers, and they tend to form attachment bonds with more than one of them. In this paper, we briefly describe the conceptual and empirical roots underlying the notion of attachment network to multiple caregivers. We describe potential reasons for researchfocusing on a single caregiver (most often mothers, but recently also fathers) and the historical attempts to establish a more ecologically valid assessment of attachment to multiple caregivers. Finally, we describe a recently developed organizational framework that includes testable models on which future research may rely for assessing the predictive power of attachment networks to multiple caregivers on children’s developmental outcomes.

"These Questions Have Everything That Happens to me": Analysis of a Femicide Risk Assessment Tool for Abused Women in Brazil

Purpose The purpose of this mixed-methods triangulation study was to assess the face validity and comprehension of a femicide risk assessment tool, the Danger Assessment-Brazil (DA-Brazil) among women seeking care in a one stop center for abused women in Curitiba, Brazil. Our secondary aim was to assess professionals' perceptions of feasibility for using the DA-Brazil in the same setting. Method Fifty-five women experiencing relationship violence completed the instrument and participated in cognitive interviews about their experience; professionals attending survivors were also interviewed. Results The vast majority of women described the DA-Brazil instrument as being easy to comprehend (n = 41, 73.2%). Nearly half of participants (n = 26, 46.4%) had some kind of question regarding the DA-Brazil calendar, a tool to visualize abuse frequency and severity. Queries aligned with five categories: recollection of dates, scale, relationship status, terminology, and discomfort. Professionals reported that the DA-Brazil instrument would support referral decision-making. Conclusion The overall face validity and comprehension of the DA-Brazil appears to be high. The majority of challenges were around the calendar activity. Professional perceptions of the DA-Brazil suggest a high degree of feasibility for its use in Brazilian healthcare settings. In order for the DA-Brazil to effectively be administered with facilitated support there is a need for training on the best use of the instrument. Accurate assessment of femicide risk is critical in a country like Brazil with high rates of femicide. The DA-Brazil provides a valid assessment of femicide risk and has the potential to trigger early intervention for those at risk.

A comparative judgment approach to assessing Chinese Sign Language interpreting

The quality of sign language interpreting (SLI) is a gripping construct among practitioners, educators and researchers, calling for reliable and valid assessment. There has been a diverse array of methods in the extant literature to measure SLI quality, ranging from traditional error analysis to recent rubric scoring. In this study, we want to expand the terrain of SLI assessment, by exploring and evaluating a novel method, known as comparative judgment (CJ), to assess SLI quality. Briefly, CJ involves judges to compare two like objects/items and make a decision by choosing the one with higher quality. The binary outcomes from repeated comparisons by a group of judges are then modelled statistically to produce standardized estimates of perceived quality for each object/item. We recruited 12 expert judges to operationalize CJ via a computerized system to assess the quality of Chinese Sign Language interpreting produced by 36 trainee interpreters. Overall, our analysis of quantitative and qualitative data provided preliminary evidential support for the validity and utility of CJ in SLI assessment. We discussed these results in relation to previous SLI literature, and suggested future research to cast light on CJ’s usefulness in applied assessment contexts.