Reliability and Validity of Behavior Observation Coding Systems in Child Maltreatment Risk Evaluation: A Systematic Review

Performing child maltreatment risk assessments is a challenging task that calls for valid and reliable measures. In child protection proceedings, mental health professionals conduct maltreatment assessments that often form an important basis for judicial decision making. Because parent–child interaction is a key construct in maltreatment risk evaluations, observational assessment measures are crucial. This systematic review aims to identify observational coding systems of parent–child interaction that are applicable for psychological evaluations of the risk of child maltreatment. The goal is to examine the potential of observational coding systems to discriminate behavior of parents who have versus have not engaged in child maltreatment. A systematic literature search led to the inclusion of 13 studies published in the United States and Europe that were then analyzed in detail. Across the 13 studies, this review identified 11 unique observational coding systems. Results are summarized systematically for study characteristics and outcomes. Additionally, the main characteristics of the observational coding systems are identified and analyzed, including the age range of the child, observation tasks, measured constructs, and reliability. The discussion focuses on the strengths and weaknesses of the individual observational coding systems in the context of child maltreatment risk assessments.

Taking a “Whole” Classroom Perspective: Theorizing Classroom Typologies Using a Video-Based Observational Protocol

Inspired by a “whole child” framing, the current study takes a “whole classroom” perspective to consider classroom practice. Study aims included: (1) presenting a systematic video-based observational coding strategy to concurrently consider practice domains that have implications for learning—cognitive instruction, classroom management, and teacher–student relational interactions; (2) identifying distinct and interrelated classroom typologies based upon this coding strategy. The framework was developed through coding and analysis of 58 purposively sampled urban 4th–9th grade classrooms from the Measures of Effective Teaching study. Analyses revealed three overarching typologies: task-focused (52%), low stimulation (43%), and optimal (5%). We conclude by discussing implications for urban education.

Studying Children’s Eating at Home: Using Synchronous Videoconference Sessions to Adapt to COVID-19 and Beyond

The COVID-19 pandemic has disrupted many facets of developmental research, including research that measures children’s eating behavior. Here, children’s food intake is often measured by weighing foods that children are offered before and after in-person testing sessions. Many studies also examine children’s food ratings (the extent to which they like or dislike a food), assessed via picture categorization tasks or hedonic scales. This paper reviews existing research on different methods for characterizing children’s eating behavior (with a focus on food intake, preferences, and concepts) and presents a feasibility study that examined whether children’s eating behaviors at home (including their food intake and ratings) can be measured via live video-chat sessions. The feasibility analyses revealed that an observational feeding paradigm at home yielded a majority (more than 70%) of video-chat recordings that had a sufficient view of the child and adequate sound and picture quality required for observational coding for the majority of the session’s duration. Such positioning would enable behavioral coding of child food intake, parent food talk, and meal characteristics. Moreover, children were able to answer questions to stories and express their preferences via researcher screen-share methods (which can assess children’s self-reported food preferences and beliefs) with low rates of exclusion across studies. The article ends with a discussion on the opportunities and challenges of using online platforms to conduct studies on children’s eating behaviors in their home environments during the COVID-19 pandemic and beyond.

Toward effective and efficient treatment adherence monitoring : development of the cognitive behavioral therapy adherence measure

Despite the large body of research demonstrating the efficacy of evidence-based practices (EBPs) for the most common youth mental health problems, EBPs are rarely used in usual care (UC). When EBPs are implemented in UC settings, there is a drop in adherence, and sometimes a drop in outcomes. Feasible and practical adherence monitoring tools may support high quality EBP delivery in UC. The current study examined the psychometric properties of the Cognitive-Behavioral Therapy Adherence Measure (CBTAM), a suite of therapist-, parent-, and youth-reported measures of adherence to the core components of cognitive-behavioral therapy (CBT) for youth anxiety, depression, and behavior problems. Therapists, parents, and youths completed the CBTAM after every session throughout a six-session treatment. Coders coded a subset of treatment sessions using the CBTAM and other established observational coding systems for CBT adherence and alliance. Therapists were consistent with coders on their ratings of individual CBTAM items. Therapist, parent, and youth CBTAM item ratings were correlated with similar items on established observational measures of CBT adherence and alliance, and these correlations were higher than correlations between CBTAM items and dissimilar items on observational measures. CBTAM adherence scores predicted faster symptom improvement on some therapist- and parent-rated outcomes. Therapist and parent end-of-treatment ratings were consistent with observational coding measures of adherence and alliance. Findings demonstrate the promise of the CBTAM as a feasible adherence monitoring tool for routine care.

Factors affecting the documentation of spoken safety-netting advice in routine GP consultations

Background: Previous studies have reported how often safety-netting is documented in medical records, but it is not known how this compares to what is verbalised and what factors might influence the consistency of documentation. Aim: To compare spoken and documented safety-netting advice (SNA) and explore factors associated with documentation. Design and setting: Secondary analysis of GP consultations archive. Method: Observational coding involving classifying and quantifying medical record entries and comparison with spoken SNA in 295 video / audio recorded consultations. Associations were tested using logistic regression. Results: Two-thirds of consultations (192/295) contained spoken SNA which applied to less than half of problems assessed (242/516). Only one-third of consultations (94/295) had documented SNA which covered 20% of problems (105/516). The practice of GPs varied widely from those that did not document their SNA, to those that nearly always did so (86.7%). GPs were more likely to document their SNA for new problems (p=0.030), when only a single problem was discussed in a consultation (p=0.040) and when they gave specific, rather than generic SNA (p=0.007). In consultations where multiple problems were assessed (n=139), the frequency of spoken and documented SNA decreased the later a problem was assessed. Conclusion: GPs frequently do not document safety-netting advice they have given to patients which may have medico-legal implications in the event of an untoward incident. GPs should consider how safely they can assess and document more than one problem in a single consultation and this risk should be shared with patients to help manage expectations.

Variance Decomposition of CAID

The Continuous Assessment of Interpersonal Dynamics (CAID) is an observational coding method that enables continuous tracking of warmth and dominance in both members of a dyad as an interaction unfolds. Research using this tool has revealed dynamic patterns relevant to psychopathology and psychotherapy, suggesting considerable potential for clinical assessment and research. However, CAID data are sensitive to a variety of person and situational factors, and the way that these factors combine to influence CAID data is poorly understood. We examined data from ten raters using CAID to assess moment-to-moment warmth and dominance in opposite-sex married dyads (N = 137 couples) interacting in four distinct situations. Using Generalizability Theory methods, we decomposed sources of variance in CAID data and estimated multiple forms of between- and within-person reliability. Results revealed how spouses’ moment-to-moment behaviors varied as a function of person, sex, dyad, rater, situation, and relevant interactions between these factors. Based on these results, we provided guidelines for interpreting CAID data at different levels of aggregation in clinical research and practice.

Quantifying the Child–Therapist Interaction in ASD Intervention: An Observational Coding System

Background: Observational research plays an important part in developmental research due to its noninvasiveness. However, it has been hardly applied to investigate efficacy of the child–therapist interaction in the context of naturalistic developmental behavioral interventions (NDBI). In particular, the characteristics of child–therapist interplay are thought to have a significant impact in NDBIs in children with autism spectrum disorder (ASD). Quantitative approaches may help to identify the key features of interaction during therapy and could be translated as instruments to monitor early interventions. Methods: n = 24 children with autism spectrum disorder (ASD) were monitored from the time of the diagnosis (T0) and after about one year of early intervention (T1). A novel observational coding system was applied to video recorded sessions of intervention to extract quantitative behavioral descriptors. We explored the coding scheme reliability together with its convergent and predictive validity. Further, we applied computational techniques to investigate changes and associations between interaction profiles and developmental outcomes. Results: Significant changes in interaction variables emerged with time, suggesting that a favorable outcome is associated with interactions characterized by increased synchrony, better therapist’s strategies to successfully engage the child and scaffold longer, more complex and engaging interchanges. Interestingly, data models linked interaction profiles, outcome measures and response trajectories. Conclusion: Current research stresses the need for process measures to understand the hows and the whys of ASD early intervention. Combining observational techniques with computational approaches may help in explaining interindividual variability. Further, it could disclose successful features of interaction associated with better response trajectories or to different ASD behavioral phenotypes that could require specific dyadic modalities.