Age-Related Memory and Novel Object Avoidance Differences in Family Dogs: Measuring the Validity and Reliability of a Rapid Behaviour Test Battery

The prolonged lifespan of companion dogs has resulted in an increased occurrence of behavioural and physical challenges linked to old age. The development of behavioural tests for identifying and monitoring age-related differences has begun. However, standardised testing requires validation. The present study aimed to assess external validity, interobserver reliability, and test-retest reliability of an indoor test battery for the rapid assessment of age-related behavioural differences in dogs. Two experimenters tested young and old dogs on a first occasion and after two weeks. Our results found external validity for two subtests out of six. On both test occasions, old dogs committed more errors than young dogs in a memory test and showed more object avoidance when encountering a novel object. Interobserver reliability and test-retest reliability was high. We conclude that the Memory and Novel object tests are valid and reliable for monitoring age-related memory performance and object neophobic differences in dogs.

Machine learning algorithm to predict anterior cruciate ligament revision demonstrates external validity

Purpose External validation of machine learning predictive models is achieved through evaluation of model performance on different groups of patients than were used for algorithm development. This important step is uncommonly performed, inhibiting clinical translation of newly developed models. Machine learning analysis of the Norwegian Knee Ligament Register (NKLR) recently led to the development of a tool capable of estimating the risk of anterior cruciate ligament (ACL) revision (https://swastvedt.shinyapps.io/calculator_rev/). The purpose of this study was to determine the external validity of the NKLR model by assessing algorithm performance when applied to patients from the Danish Knee Ligament Registry (DKLR). Methods The primary outcome measure of the NKLR model was probability of revision ACL reconstruction within 1, 2, and/or 5 years. For external validation, all DKLR patients with complete data for the five variables required for NKLR prediction were included. The five variables included graft choice, femur fixation device, KOOS QOL score at surgery, years from injury to surgery, and age at surgery. Predicted revision probabilities were calculated for all DKLR patients. The model performance was assessed using the same metrics as the NKLR study: concordance and calibration. Results In total, 10,922 DKLR patients were included for analysis. Average follow-up time or time-to-revision was 8.4 (± 4.3) years and overall revision rate was 6.9%. Surgical technique trends (i.e., graft choice and fixation devices) and injury characteristics (i.e., concomitant meniscus and cartilage pathology) were dissimilar between registries. The model produced similar concordance when applied to the DKLR population compared to the original NKLR test data (DKLR: 0.68; NKLR: 0.68–0.69). Calibration was poorer for the DKLR population at one and five years post primary surgery but similar to the NKLR at two years. Conclusion The NKLR machine learning algorithm demonstrated similar performance when applied to patients from the DKLR, suggesting that it is valid for application outside of the initial patient population. This represents the first machine learning model for predicting revision ACL reconstruction that has been externally validated. Clinicians can use this in-clinic calculator to estimate revision risk at a patient specific level when discussing outcome expectations pre-operatively. While encouraging, it should be noted that the performance of the model on patients undergoing ACL reconstruction outside of Scandinavia remains unknown. Level of evidence III.

EXPRESS: The More You Ask, the Less You Get: When Additional Questions Hurt External Validity

Researchers and practitioners in marketing, economics, and public policy often use preference elicitation tasks to forecast real-world behaviors. These tasks typically ask a series of similarly-structured questions. The authors posit that every time a respondent answers an additional elicitation question, two things happen: (1) they provide information about some parameter(s) of interest, such as their time preference or the partworth for a product attribute, and (2) the respondent increasingly adapts to the task—i.e., using task-specific decision processes specialized for this task that may or may not apply to other tasks. Importantly, adaptation comes at the cost of potential mismatch between the task-specific decision process and real-world processes that generate the target behaviors, such that asking more questions can reduce external validity. The authors used mouse- and eye-tracking to trace decision processes in time preference measurement and conjoint choice tasks: Respondents increasingly relied on task-specific decision processes as more questions were asked, leading to reduced external validity for both related tasks and real-world behaviors. Importantly, the external validity of measured preferences peaked after as few as seven questions in both types of tasks. When measuring preferences, less can be more.

Sexual Victimization Involving Solo Female Offenders: A Crime Script Approach

This study aims to provide a theoretically grounded analysis of the crime-commission process of solo females involved in sexual offending, using crime scripts. The sample includes 93 cases of sexual assaults perpetrated by female offenders in an extrafamilial context. Latent class analysis was used to identify the scripts involved in female sexual offending as well as to explore the relationship between each step of the crime-commission process. Also, additional variables related to victim, offender, and location characteristics were used to test the external validity of the model. Results suggest four different scripts used by females: Daytime Indoor, Coercive Outdoor, Coercive Indoor, and Nighttime Indoor. Theoretical and practical implications are discussed.

Four points regarding reproducibility and external statistical validity: a comment on Walter et al.

Walter et al. (2021) present phase 1–2–3 trial data that show two doses of the BNT162b2 (Pfizer–BioN-Tech) Covid-19 vaccine were safe and effective in children aged 5–11 years. Given that millions of children in this age group are receiving the paediatric Pfizer COVID-19 vaccine, that there are potential risks, and that the balance of benefits over potential risks is more limited in children compared to adults due to low rates of serious disease (ATAGI 2021), gold standards ought to be applied to supporting data in terms of placebo-controlled disease endpoint efficacy trials, safety databases large enough to detect adverse events, and appropriate data sharing to enable reproduction and scrutiny of results. Four points are worthy of attention regarding the reproducibility and external statistical validity of the analysis reported in Walter et al. (2021). ‘External validity’ refers to the extent to which conclusions drawn from the data (and statistical tests thereof) are likely to correspond to, or be generalisable to, the real world (Campbell 1957). ‘Reproducibility’ refers to the ability of independent researchers to draw the same conclusions from the data (Kass et al. 2016).