Theory Testing and Process Evidence in Accounting Experiments

This paper discusses the role of process evidence in accounting research. We define process evidence broadly as data providing insight into how and why cause-effect relationships occur, and we provide a framework to guide the provision and evaluation of process evidence in accounting studies. Our definition allows for an expanded understanding of techniques for gathering process evidence. The framework highlights the importance of the study’s goals and theory in choosing how to provide process evidence as well as how much process evidence to provide. The paper also outlines the strengths and limitations of three approaches to providing process evidence: mediation, moderation, and multiple-study based designs. We provide recommendations for best practices for each approach to minimize threats to validity and maximize the value of process evidence.

Impact of potential modifications to Alzheimer’s disease clinical trials in response to disruption by COVID-19: a simulation study

Background The COVID-19 pandemic disrupted Alzheimer disease randomized clinical trials (RCTs), forcing investigators to make changes in the conduct of such trials while endeavoring to maintain their validity. Changing ongoing RCTs carries risks for biases and threats to validity. To understand the impact of exigent modifications due to COVID-19, we examined several scenarios in symptomatic and disease modification trials that could be made. Methods We identified both symptomatic and disease modification Alzheimer disease RCTs as exemplars of those that would be affected by the pandemic and considered the types of changes that sponsors could make to each. We modeled three scenarios for each of the types of trials using existing datasets, adjusting enrollment, follow-ups, and dropouts to examine the potential effects COVID-19-related changes. Simulations were performed that accounted for completion and dropout patterns using linear mixed effects models, modeling time as continuous and categorical. The statistical power of the scenarios was determined. Results Truncating both symptomatic and disease modification trials led to underpowered trials. By contrast, adapting the trials by extending the treatment period, temporarily stopping treatment, delaying outcomes assessments, and performing remote assessment allowed for increased statistical power nearly to the level originally planned. Discussion These analyses support the idea that disrupted trials under common scenarios are better continued and extended even in the face of dropouts, treatment disruptions, missing outcomes, and other exigencies and that adaptations can be made that maintain the trials’ validity. We suggest some adaptive methods to do this noting that some changes become under-powered to detect the original effect sizes and expected outcomes. These analyses provide insight to better plan trials that are resilient to unexpected changes to the medical, social, and political milieu.

A Causal Framework for Cross-Cultural Generalizability

Behavioral researchers increasingly recognize the need for more diverse samples that capture the breadth of human experience. Current attempts to establish generalizability across populations focus on threats to validity, constraints on generalization and the accumulation of large cross-cultural datasets. But for continued progress, we also require a framework that lets us determine which inferences can be drawn and how to make informative cross-cultural comparisons. We describe a generative causal modeling framework and outline simple graphical criteria to derive analytic strategies and implied generalizations. Using both simulated and real data, we demonstrate how to project and compare estimates across populations. We conclude with a discussion of how a formal framework for generalizability can assist researchers in designing more informative cross-cultural studies and thus provides a more solid foundation for cumulative and generalizable behavioral research.

Impact of Potential Modifications to Alzheimer’s Disease Clinical Trials in Response to Disruption by COVID-19

BackgroundThe COVID-19 pandemic disrupted Alzheimer disease randomized clinical trials (RCTs)forcing investigators to make changes in the conduct of such trials while endeavoring to maintain their validity. Changing ongoing RCTs carries risks for biases and threats to validity. To understand the impact of exigent modifications due to COVID-19 we examined several scenarios in symptomatic and disease modification trials that could be made.MethodsWe identified both symptomatic and disease modification Alzheimer disease RCTs as exemplars of those that would be affected by the pandemic and considered the types of changes that sponsors could make to each. We modeled three scenarios for each of the types of trialsusing existing datasets, adjusting enrollment, follow-ups, and dropouts to examine the potential effects COVID-19-related changes.Simulations were performed that accounted for completion and dropout patterns using linear mixed effects models, modeling time as continuous and categorical. The statistical power of the scenarios was determined.ResultsTruncating both symptomatic and disease modification trials, led to underpowered trials.By contrast, adapting the trials byextending the treatment period, temporarily stopping treatment, delaying outcomes assessments, and performing remote assessment allowed for increased statistical power nearly to the level originally planned.DiscussionThese analyses support the idea that disrupted trials under common scenarios are better continued and extended even in the face of dropouts, treatment disruptions, missing outcomes, and other exigencies, and that adaptations can be made that maintain the trials validity. We suggest some adaptive methods to do this noting that some changes become under-powered to detect theoriginal effect sizes and expected outcomes. These analyses provide insight to better plan trials that are resilient to unexpected changes to the medical, social, and political milieu.

Engineering Model-Based Software Testing of WIMP Interactive Applications

The goal of software testing is to detect defects with the objective of removing them at a later stage in the development process. Interactive software development follows the User Centered Design approach that promotes continuous involvement of users both at design and evaluation phases. This process is meant to produce usable interactive software by gathering functional and non-functional requirements related to both user needs and context of use. However, taking into account these potentially very-complex-to-implement requirements increases the complexity of the software that is likely, without appropriate methods and tools, to encompass a large number of defects. One of the limitations of UCD approaches is that it provides no guidance on the engineering of the interactive application, which thus usually embeds numerous defects resulting in failures at the origin of user frustrations and performance drops. Even though a classification of interactive application defects has been proposed, interactive application testers remain only superficially supported in detecting them. This paper defines a model-based approach to engineer the testing activity for interactive applications. It proposes a process that bridges the gap between UCD artefacts and interactive software implementation by the production of a dedicated formal model exploited for testing purposes only. The application of the process is demonstrated on an interactive cockpit WIMP application. Finally, threats to validity (capability of the approach to detect defects and to ensure an acceptable coverage testing of the interactive application) are addressed by a longitudinal study on 61 variants of a simple application developed by 61 different developers. ?

Evolution of the Unified State Exam in History: Poems and Prose of Teachers' Practices

This paper investigates the effect of сonstructed-response items in the Unified State Exam (ESE) in History on exam’s validity and the threats to validity. The Unified State Exam is the primary high-stakes examination for Russian students. Despite playing a vital role as an achievement and an admission test, this exam’s validity has not been looked into. The evolution of this exam is distinctly marked by a growing change in the number and weight of constructed-response items, which might be affecting the validity of test results in many ways. The research was focused on interviews with 36 history experts. Thematic analysis of transcripts helped to identify three main threats to validity: faulty criteria, task content and expert bias. The paper presents these results along with recommendations on improving the test.

Using an Experimental Framework to Support Variables Selection: An Exploratory Study

The selection of variables in a given experiment is crucial, since it is the theoretical foundation that guides how data should be collected and analyzed. However, selecting variables is an intricate activity, especially considering areas such as Software Engineering and Education, whose studies should also consider human-related variables in the design. In this scenario, we aim to investigate how a support mechanism helps in the variables selection activity of the experiment process. To do so, we conducted a preliminary study on the use of an experimental framework composed of a catalog of variables. We explored the domain of the integration of software testing into programming education. Participants were divided into two groups (ad hoc and framework support) and asked to select variables for a given experiment goal. We analyzed the results by identifying threats to validity in their experimental design drafts. Results show a significant number of threats of type inadequate explication of constructs for both groups. Nonetheless, the framework helped to increase the clarity of concepts selected as variables. The cause of most raised threats, even with the framework support, was an inaccuracy in selecting the values of such variables (i.e. treatments and fixed values).

The meaning of postpartum sexual health for women living in Spain: a phenomenological inquiry

Background Sexual health is a multidimensional phenomenon constructed by personal, social, and cultural factors but continues to be studied with a biomedical approach. During the postpartum period, a woman transitions to mother, as well as partner-to-parent and couple-to-family. There are new realities in life in the postpartum period, including household changes and new responsibilities that can impact the quality of sexual health. This phenomenon is understudied especially in the context of Spain. The purpose of this study was to describe the lived experience of postpartum sexual health among primiparous women giving birth in Catalonia (Spain). Methods This was a phenomenological study with a purposive sample of primiparous women. Data was collected through semi-structured interviews until saturation. Analysis followed Colaizzi’s seven-step process with an eighth translation step added to limit cross-cultural threats to validity. Also, the four dimensions of trustworthiness were established through strategies and techniques during data collection and analysis. Results Ten women were interviewed from which five themes emerged, including: Not feeling ready, inhibiting factors, new reality at home, socio-cultural factors, and the clinician within the health system. Returning to sexual health led women to engage in experiential learning through trial and error. Most participants reported reduced libido, experienced altered body image, and recounted resumption of sexual activity before feeling ready. A common finding was fatigue and feeling overloaded by the demands of the newborn. Partner support was described as essential to returning to a meaningful relationship. Discussions about postpartum sexual health with clinicians were described as taboo, and largely absent from the care model. Conclusion Evidence-based practices should incorporate the best evidence from research, consider the postpartum sexual health experiences and preferences of the woman, and use clinician expertise in discussions that include the topic of postpartum sexual health to make decisions. As such, human caring practices should be incorporated into clinical guidelines to recognize the preferences of women. Clinicians need to be authentically present, engage in active communication, and individualize their care. More qualitative studies are needed to understand postpartum sexual health in different contexts, cultures, and countries and to identify similarities and differences through meta-synthesis.

The meaning of postpartum sexual health for women living in Spain: A phenomenological inquiry.

Background: Sexual health is a multidimensional phenomenon constructed by personal, social, and cultural factors but continues to be studied with a biomedical approach. During the postpartum period, a woman transitions to mother, as well as partner-to-parent and couple-to-family. There are new realities in life in the postpartum period, including household changes and new responsibilities that can impact the quality of sexual health. This phenomenon is understudied especially in the context of Spain. The purpose of this study was to describe the lived experience of postpartum sexual health among primiparous women giving birth in Catalonia (Spain).Methods: This was a phenomenological study with a purposive sample of primiparous women. Data was collected through semi-structured interviews until saturation. Analysis followed Colaizzi's seven-step process with an eighth translation step added to limit cross-cultural threats to validity. Also, the four dimensions of trustworthiness were established through strategies and techniques during data collection and analysis.Results: Ten women were interviewed from which five themes emerged, including: Not feeling ready, inhibiting factors, new reality at home, socio-cultural factors, and the clinician within the health system. Returning to sexual health led women to engage in experiential learning through trial and error. Most participants reported reduced libido, experienced altered body image, and recounted resumption of sexual activity before feeling ready. A common finding was fatigue and feeling overloaded by the demands of the newborn. Partner support was described as essential to returning to a meaningful relationship. Discussions about postpartum sexual health with clinicians were described as taboo, and largely absent from the care model.Conclusion: Evidence-based practices should incorporate the best evidence from research, consider the postpartum sexual health experiences and preferences of the woman, and use clinician expertise in discussions that include the topic of postpartum sexual health to make decisions. As such, human caring practices should be incorporated into clinical guidelines to recognize the preferences of women. Clinicians need to be authentically present, engage in active communication, and individualize their care. More qualitative studies are needed to understand postpartum sexual health in different contexts, cultures, and countries and to identify similarities and differences through meta-synthesis.

Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines

ABSTRACTObservational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. These are now in planning amid the ongoing rollout of SARS-CoV-2 vaccines globally. While traditional case-control (TCC) and test-negative design (TND) studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here we review the theoretical basis for estimation of vaccine direct effects under TCC and TND frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs. Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, non-specific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The TCC may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The TND reduces but may not eliminate this confounding, for instance if individuals who receive vaccination seek care or testing for less-severe infection. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources. We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages.