Conducting Comparable Research in Representative Worlds

Field-based simulation research can be delayed or prevented due to restricted resources and other practical challenges. Although laboratory work is a feasible alternative, it is often criticized for a lack of generalizability. We faced this issue when investigating the impact of workplace interruptions on nurses’ work performance in the Intensive Care Unit (ICU). The potential relationship between interruptions and errors has been widely investigated in healthcare settings; however, much of the evidence is associative. Some evidence outside the healthcare domain points to a causal connection between interruptions and errors, but the studies are mostly laboratory based and interruptions are artificial to the situation. Our eventual aim is to carry out a high-fidelity randomized-controlled trial to test the hypothesis that interruptions cause errors in healthcare, which could have major implications for interventions and policy. However, there are considerable challenges and constraints to overcome when designing such an experiment; for example (a) a limited potential participant pool within the ICU, (b) constant changes to technology and procedures in the ICU, and (c) restricted access to hospital simulation rooms. There are various ways to address these issues but most options compromise the generalizability of the simulation to authentic situations. By adopting principles of Brunswick’s representative design, we designed an initial laboratory study to be used as a formal pilot study in a different, but parallel, context to healthcare. Representative design refers to the “arrangement of conditions of an experiment so that they represent the behavioural setting to which the results are intended to apply” (Araujo, Davids, & Passos, 2007, p. 72). Representing the ‘natural world’ with high fidelity is not crucial in representative design, but rather ensuring that the properties of the conditions to which the researcher wishes to generalize are adequately captured in the laboratory task (Hammond & Stewart, 2001). Accordingly, we have created a laboratory-based simulation on the basis of the properties we aim to capture in a healthcare-based simulation, so that we can generalize findings from the former to the latter. The laboratory component needed to involve a task that embodied the high-level properties of medication preparation and administration, with a specialized population who regularly perform the task. The task of cocktail making fits the above requirements because it has high-level properties similar to medication preparation and administration. Cocktail making is concerned with controlled liquids, it requires perceptual motor skills, it involves multi-step tasks carried out in a busy environment where there is a high demand on working memory, and it is performed by experts (bartenders). These similarities mean that cocktail making can be used as a laboratory analog of certain aspects of medication preparation and administration. First, we mapped the physical environment across the two domains to ensure the cocktail component possessed the same spatial properties as the medication component. Second, we designed the high-level structure of the cocktail scenarios to approximate the high-level structure of the medication component scenarios. Third, we designed the interrupting tasks and added them to the scenarios. Experiment 1 of the cocktail component was a condition with zero interruptions to provide a baseline error rate with which to calculate the required sample size for Experiment 2. Experiment 2 was a between-subjects study in which participants were randomly assigned to receive either 3 or 12 interruptions. All participants had at least one-month cocktail making experience in a licensed venue. Cocktail errors were the analog of clinical errors in healthcare (Westbrook et al., 2010). In Experiment 1, an average of 44% of cocktails made contained at least one error per scenario ( SD = 18%). To calculate the required sample size for Experiment 2, an effect size was calculated by integrating our baseline data with observational data from Westbrook et al. (2010) and a power analysis was performed. Data collection for Experiment 2 is now completed. The findings from Experiment 2 will be used to calculate the required sample size for the medication component and lessons learned from the cocktail component will help finalize the design of the medication component. The cocktail component of our study is not totally analogous to the medication component, but we have shown that principles of representative design can be used to design a simulation from which we can argue that generalizable findings can be gathered. A major advantage of our approach is that we have been able to design and test many formal aspects of the medication component of our study prior to stepping into the hospital simulator. Findings from the medication component will help to shape interventions and policies in the healthcare domain that reduce error rates and increase patient safety. The cocktail component will contribute to the interruptions literature because the interruptions are representative of those that would actually occur in the workplace – something that is rare in laboratory-based interruptions research.