scholarly journals Simulation and minimization: technical advances for factorial experiments designed to optimize clinical interventions

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Jocelyn Kuhn ◽  
Radley Christopher Sheldrick ◽  
Sarabeth Broder-Fingert ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Computer Simulation ◽  
Random Element ◽  
Optimization Strategy ◽  
Randomized Experiments ◽  
Factorial Experiments ◽  
Sample Sizes ◽  
Multiphase Optimization Strategy ◽  
Technical Advances ◽  
Healthcare Interventions ◽  
The Impact

Abstract Background The Multiphase Optimization Strategy (MOST) is designed to maximize the impact of clinical healthcare interventions, which are typically multicomponent and increasingly complex. MOST often relies on factorial experiments to identify which components of an intervention are most effective, efficient, and scalable. When assigning participants to conditions in factorial experiments, researchers must be careful to select the assignment procedure that will result in balanced sample sizes and equivalence of covariates across conditions while maintaining unpredictability. Methods In the context of a MOST optimization trial with a 2x2x2x2 factorial design, we used computer simulation to empirically test five subject allocation procedures: simple randomization, stratified randomization with permuted blocks, maximum tolerated imbalance (MTI), minimal sufficient balance (MSB), and minimization. We compared these methods across the 16 study cells with respect to sample size balance, equivalence on key covariates, and unpredictability. Leveraging an existing dataset to compare these procedures, we conducted 250 computerized simulations using bootstrap samples of 304 participants. Results Simple randomization, the most unpredictable procedure, generated poor sample balance and equivalence of covariates across the 16 study cells. Stratified randomization with permuted blocks performed well on stratified variables but resulted in poor equivalence on other covariates and poor balance. MTI, MSB, and minimization had higher complexity and cost. MTI resulted in balance close to pre-specified thresholds and a higher degree of unpredictability, but poor equivalence of covariates. MSB had 19.7% deterministic allocations, poor sample balance and improved equivalence on only a few covariates. Minimization was most successful in achieving balanced sample sizes and equivalence across a large number of covariates, but resulted in 34% deterministic allocations. Small differences in proportion of correct guesses were found across the procedures. Conclusions Based on the computer simulation results and priorities within the study context, minimization with a random element was selected for the planned research study. Minimization with a random element, as well as computer simulation to make an informed randomization procedure choice, are utilized infrequently in randomized experiments but represent important technical advances that researchers implementing multi-arm and factorial studies should consider.

scholarly journals Simulation and minimization: Technical advances for factorial experiments designed to optimize clinical interventions

2019 ◽  
Author(s):  
Jocelyn Lara Kuhn ◽  
Radley Christopher Sheldrick ◽  
Sarabeth Broder-Fingert ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Computer Simulation ◽  
Random Element ◽  
Optimization Strategy ◽  
Randomized Experiments ◽  
Factorial Experiments ◽  
Sample Sizes ◽  
Multiphase Optimization Strategy ◽  
Technical Advances ◽  
Healthcare Interventions ◽  
The Impact

Abstract Background: The Multiphase Optimization Strategy (MOST) is designed to maximize the impact of clinical healthcare interventions, which are typically multicomponent and increasingly complex. MOST often relies on factorial experiments to identify which components of an intervention are most effective, efficient, and scalable. When assigning participants to conditions in factorial experiments, researchers must be careful to select the assignment procedure that will result in balanced sample sizes and equivalence of covariates across conditions while maintaining unpredictability. Methods: In the context of a MOST optimization trial with a 2x2x2x2 factorial design, we used computer simulation to empirically test five subject allocation procedures: simple randomization, stratified randomization with permuted blocks, maximum tolerated imbalance (MTI), minimal sufficient balance (MSB), and minimization. We compared these methods across the 16 study cells with respect to sample size balance, equivalence on key covariates, and unpredictability. Leveraging an existing dataset to compare these procedures, we conducted 250 computerized simulations using bootstrap samples of 304 participants. Results: Simple randomization, the most unpredictable procedure, generated poor sample balance and equivalence of covariates across the 16 study cells. Stratified randomization with permuted blocks performed well on stratified variables but resulted in poor equivalence on other covariates and poor balance. MTI, MSB, and minimization had higher complexity and cost. MTI resulted in balance close to pre-specified thresholds and a higher degree of unpredictability, but poor equivalence of covariates. MSB had 19.7% deterministic allocations, poor sample balance and improved equivalence on only a few covariates. Minimization was most successful in achieving balanced sample sizes and equivalence across a large number of covariates, but resulted in 34% deterministic allocations. Small differences in proportion of correct guesses were found across the procedures. Conclusions: Computer simulation was highly useful for evaluating tradeoffs among randomization procedures. Based on the computer simulation results and priorities within the study context, minimization with a random element was selected for the planned research study. Minimization with a random element, as well as computer simulation to make an informed randomization procedure choice, are utilized infrequently in randomized experiments but represent important technical advances that researchers implementing multi-arm and factorial studies should consider.

scholarly journals Simulation and minimization: Technical advances for factorial experiments designed to optimize clinical interventions

2019 ◽  
Author(s):  
Jocelyn Lara Kuhn ◽  
Radley Christopher Sheldrick ◽  
Sarabeth Broder-Fingert ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Computer Simulation ◽  
Random Element ◽  
Optimization Strategy ◽  
Randomized Experiments ◽  
Factorial Experiments ◽  
Sample Sizes ◽  
Multiphase Optimization Strategy ◽  
Technical Advances ◽  
Healthcare Interventions ◽  
The Impact

Abstract Background: The Multiphase Optimization Strategy (MOST) is designed to maximize the impact of clinical healthcare interventions, which are typically multicomponent and increasingly complex. MOST often relies on factorial experiments to identify which components of an intervention are most effective, efficient, and scalable. When assigning participants to conditions in factorial experiments, researchers must be careful to select the assignment procedure that will result in balanced sample sizes and equivalence of covariates across conditions while maintaining unpredictability. Methods: In the context of a MOST optimization trial with a 2x2x2x2 factorial design, we used computer simulation to empirically test five subject allocation procedures: simple randomization, stratified randomization with permuted blocks, maximum tolerated imbalance (MTI), minimal sufficient balance (MSB), and minimization. We compared these methods across the 16 study cells with respect to sample size balance, equivalence on key covariates, and unpredictability. Leveraging an existing dataset to compare these procedures, we conducted 250 computerized simulations using bootstrap samples of 304 participants. Results: Simple randomization, the most unpredictable procedure, generated poor sample balance and equivalence of covariates across the 16 study cells. Stratified randomization with permuted blocks performed well on stratified variables but resulted in poor equivalence on other covariates and poor balance. MTI, MSB, and minimization had higher complexity and cost. MTI resulted in balance close to pre-specified thresholds and a higher degree of unpredictability, but poor equivalence of covariates. MSB had 19.7% deterministic allocations, poor sample balance and improved equivalence on only a few covariates. Minimization was most successful in achieving balanced sample sizes and equivalence across a large number of covariates, but resulted in 34% deterministic allocations. Small differences in proportion of correct guesses were found across the procedures. Conclusions: Based on the computer simulation results and priorities within the study context, minimization with a random element was selected for the planned research study. Minimization with a random element, as well as computer simulation to make an informed randomization procedure choice, are utilized infrequently in randomized experiments but represent important technical advances that researchers implementing multi-arm and factorial studies should consider.

scholarly journals Simulation and minimization: Technical advances for factorial experiments designed to optimize clinical interventions

2019 ◽  
Author(s):  
Jocelyn Lara Kuhn ◽  
Radley Christopher Sheldrick ◽  
Sarabeth Broder-Fingert ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Computer Simulation ◽  
Sample Size ◽  
Optimization Strategy ◽  
Randomized Experiments ◽  
Factorial Experiments ◽  
Sample Sizes ◽  
Assignment Method ◽  
Multiphase Optimization Strategy ◽  
Allocation Method ◽  
The Impact

Abstract Background The Multiphase Optimization Strategy (MOST) is designed to maximize the impact of clinical healthcare interventions, which are typically multicomponent and increasingly complex. The MOST framework often relies on factorial experiments to identify which components of an intervention are most effective, efficient, and scalable. When assigning participants to conditions in factorial experiments, researchers must be careful to select the assignment method that will result in balanced sample sizes and equivalence of covariates across conditions without being predictable. Historically, procedures most often include simple randomization and stratification with blocking; minimization is an increasingly utilized method that assigns participants to the condition that minimizes differences in covariates and sample size across study conditions. Methods In the context of a MOST optimization trial with a 2x2x2x2 factorial design (4 components, 16 cells), we used computer simulation to empirically test three subject assignment methods: simple randomization, stratification with blocking, and minimization. We compared these methods with respect to: sample size balance across condition, equivalence across conditions on key covariates, and unpredictability of assignments. Leveraging an existing dataset to compare three different allocation methods, we conducted 250 computerized simulations using bootstrap samples of 304 participants, which was the planned sample size for the proposed study. Result Simple randomization, the most unpredictable subject allocation method, generated the least balance of sample and equivalence of covariates across the 16 study cells. Stratification with blocking performed well on stratified variables, and resulted in similar sample balance and predictability as minimization. In contrast, minimization, which had a higher degree of complexity and cost, was most successful in achieving balanced sample sizes and equivalence across a large number of covariates. Conclusions Unlike simple randomization, minimization procedures and stratification with blocking are both methodologically sound options for factorial designs. Based on the computer simulation results and priorities within context of this MOST optimization trial, minimization was selected as the optimal subject allocation method. Minimization is utilized infrequently in randomized experiments but represents an important technical advance that should be considered by researchers implementing multi-arm and factorial studies.

Developing mHealth Interventions with Implementation in Mind: Application of the Multiphase Optimization Strategy (MOST) Preparation Phase to Diabetes Prevention Programming (Preprint)

2022 ◽  
Author(s):  
Megan MacPherson ◽  
Kohle Merry ◽  
Sean Locke ◽  
Mary Jung
Keyword(s):  
Phase I ◽  
Conceptual Model ◽  
Diabetes Prevention ◽  
Healthcare Providers ◽  
Intervention Development ◽  
Diabetes Prevention Program ◽  
Optimization Strategy ◽  
Mhealth Intervention ◽  
Multiphase Optimization Strategy ◽  
The Impact

UNSTRUCTURED With thousands of mHealth solutions on the market, patients and healthcare providers struggle to identify which solution to use/prescribe. The lack of evidence-based mHealth solutions may be due to limited research on intervention development and continued use of traditional research methods for mHealth evaluation. The Multiphase Optimization Strategy (MOST) is a framework which aids in developing interventions which are economical, affordable, scalable, and effective (EASE). MOST Phase I highlights the importance of formative intervention development, a stage often overlooked and rarely published. The aim of MOST Phase I is to identify candidate intervention components, create a conceptual model, and define the optimization objective. While MOST sets these three targets, the framework itself does not provide robust guidance on how to conduct quality research within Phase I, and what steps can be taken to identify potential intervention components, develop the conceptual model, and achieve intervention EASE with the implementation context in mind. To advance the applicability of MOST within the field of implementation science, this paper provides an account of the methods used to develop an mHealth intervention. Specifically, we provide a comprehensive example of how to achieve the goals of MOST Phase I by outlining the formative development of an mHealth prompting intervention within a diabetes prevention program. Additionally, recommendations are proposed for future researchers to conduct formative research on mHealth interventions with implementation in mind. Given its considerable reach, mHealth has the potential to positively impact public health by decreasing implementation costs and improving accessibility. MOST is well-suited for the efficient development and optimization of mHealth interventions. By using an implementation-focused lens and outlining the steps in developing an mHealth intervention using MOST Phase I, this work can may guide future intervention developers towards maximizing the impact of mHealth outside of the research laboratory.

scholarly journals Using the Multiphase Optimization Strategy (MOST) Framework to Test Intervention Delivery Strategies: A Study Protocol

2019 ◽  
Author(s):  
Sarabeth Broder-Fingert ◽  
Jocelyn Lara Kuhn ◽  
Radley Christopher Sheldrick ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Health Services ◽  
Factorial Design ◽  
Behavioral Health ◽  
Study Protocol ◽  
Optimization Strategy ◽  
Behavioral Health Services ◽  
Intervention Delivery ◽  
Multiphase Optimization Strategy ◽  
Delivery Strategies ◽  
The Impact

Abstract Background Delivery of behavioral interventions is complex, as the majority of interventions consist of multiple components used either simultaneously, sequentially, or both. The importance of clearly delineating delivery strategies within these complex interventions - and furthermore understanding the impact of each strategy on effectiveness - has recently emerged as an important facet of intervention research. Yet, few methodologies exist to prospectively test the effectiveness of delivery strategies and how they impact implementation. In the current paper, we describe a study protocol for a large randomized controlled trial in which we will use the Multiphase Optimization Strategy (MOST) – a novel framework developed to optimize interventions - to test the effectiveness of intervention delivery strategies using a factorial design. We apply this framework to delivery of Family Navigation (FN), an evidence-based care management strategy designed to reduce disparities and improve access to behavioral health services, and test four components related to its implementation. Methods/Design The MOST framework contains three distinct phases: Preparation, Optimization, and Evaluation. The preparation phase for this study occurred previously. The current study consists of the optimization and evaluation phases. Children ages three-to-twelve years-old who are detected as “at-risk” for behavioral health disorders (n=304) at a large, urban federally qualified community health center will be referred to a Family Partner – a bi-cultural, bi-lingual member of the community with training in behavioral health and systems navigation – who will perform FN. Families will then be randomized to one of 16 possible combinations of FN delivery strategies (2x2x2x2 factorial design). The primary outcome measure will be achieving a family-centered goal related to behavioral health services within 90 days of randomization. Implementation data on fidelity, acceptability, feasibility, and cost of each strategy will also be collected. Results from the primary and secondary outcomes will be reviewed by our team of stakeholders to optimize FN delivery for implementation and dissemination based on effectiveness, efficiency, and cost. Discussion In this protocol paper, we describe how the MOST Framework can be used to improve intervention delivery. These methods will be useful for future studies testing intervention delivery strategies and their impact on implementation.

scholarly journals Using the Multiphase Optimization Strategy (MOST) framework to test intervention delivery strategies: a study protocol

Trials ◽  
2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Sarabeth Broder-Fingert ◽  
Jocelyn Kuhn ◽  
Radley Christopher Sheldrick ◽  
Andrea Chu ◽  
Lisa Fortuna ◽  
...  
Keyword(s):  
Health Services ◽  
Factorial Design ◽  
Behavioral Health ◽  
Study Protocol ◽  
Optimization Strategy ◽  
Behavioral Health Services ◽  
Intervention Delivery ◽  
Multiphase Optimization Strategy ◽  
Delivery Strategies ◽  
The Impact

Abstract Background Delivery of behavioral interventions is complex, as the majority of interventions consist of multiple components used either simultaneously, sequentially, or both. The importance of clearly delineating delivery strategies within these complex interventions—and furthermore understanding the impact of each strategy on effectiveness—has recently emerged as an important facet of intervention research. Yet, few methodologies exist to prospectively test the effectiveness of delivery strategies and how they impact implementation. In the current paper, we describe a study protocol for a large randomized controlled trial in which we will use the Multiphase Optimization Strategy (MOST), a novel framework developed to optimize interventions, i.e., to test the effectiveness of intervention delivery strategies using a factorial design. We apply this framework to delivery of Family Navigation (FN), an evidence-based care management strategy designed to reduce disparities and improve access to behavioral health services, and test four components related to its implementation. Methods/design The MOST framework contains three distinct phases: Preparation, Optimization, and Evaluation. The Preparation phase for this study occurred previously. The current study consists of the Optimization and Evaluation phases. Children aged 3-to-12 years old who are detected as “at-risk” for behavioral health disorders (n = 304) at a large, urban federally qualified community health center will be referred to a Family Partner—a bicultural, bilingual member of the community with training in behavioral health and systems navigation—who will perform FN. Families will then be randomized to one of 16 possible combinations of FN delivery strategies (2 × 2 × 2× 2 factorial design). The primary outcome measure will be achieving a family-centered goal related to behavioral health services within 90 days of randomization. Implementation data on the fidelity, acceptability, feasibility, and cost of each strategy will also be collected. Results from the primary and secondary outcomes will be reviewed by our team of stakeholders to optimize FN delivery for implementation and dissemination based on effectiveness, efficiency, and cost. Discussion In this protocol paper, we describe how the MOST framework can be used to improve intervention delivery. These methods will be useful for future studies testing intervention delivery strategies and their impact on implementation. Trial registration ClinicalTrials.gov, NCT03569449. Registered on 26 June 2018.

scholarly journals Robots for Elderly Care in the Home: A Landscape Analysis and Co-Design Toolkit

2021 ◽  
Author(s):  
Gianluca Bardaro ◽  
Alessio Antonini ◽  
Enrico Motta
Keyword(s):  
Large Scale ◽  
Daily Life ◽  
Elderly Care ◽  
Real Life ◽  
Robotic Platform ◽  
Holistic View ◽  
Personal Assistants ◽  
Healthcare Interventions ◽  
The Impact ◽  
The Eu

AbstractOver the last two decades, several deployments of robots for in-house assistance of older adults have been trialled. However, these solutions are mostly prototypes and remain unused in real-life scenarios. In this work, we review the historical and current landscape of the field, to try and understand why robots have yet to succeed as personal assistants in daily life. Our analysis focuses on two complementary aspects: the capabilities of the physical platform and the logic of the deployment. The former analysis shows regularities in hardware configurations and functionalities, leading to the definition of a set of six application-level capabilities (exploration, identification, remote control, communication, manipulation, and digital situatedness). The latter focuses on the impact of robots on the daily life of users and categorises the deployment of robots for healthcare interventions using three types of services: support, mitigation, and response. Our investigation reveals that the value of healthcare interventions is limited by a stagnation of functionalities and a disconnection between the robotic platform and the design of the intervention. To address this issue, we propose a novel co-design toolkit, which uses an ecological framework for robot interventions in the healthcare domain. Our approach connects robot capabilities with known geriatric factors, to create a holistic view encompassing both the physical platform and the logic of the deployment. As a case study-based validation, we discuss the use of the toolkit in the pre-design of the robotic platform for an pilot intervention, part of the EU large-scale pilot of the EU H2020 GATEKEEPER project.

Using Behavioral Insights to Improve School Administrative Communications: The Case of Truancy Notifications

2021 ◽  
pp. 0013189X2110007
Author(s):  
Jessica Lasky-Fink ◽  
Carly D. Robinson ◽  
Hedy Nai-Lin Chang ◽  
Todd Rogers
Keyword(s):  
Parental Engagement ◽  
Randomized Experiment ◽  
Randomized Experiments ◽  
Parental Efficacy ◽  
Behavioral Barriers ◽  
The Impact

Many states mandate districts or schools notify parents when students have missed multiple unexcused days of school. We report a randomized experiment ( N = 131,312) evaluating the impact of sending parents truancy notifications modified to target behavioral barriers that can hinder effective parental engagement. Modified truancy notifications that used simplified language, emphasized parental efficacy, and highlighted the negative incremental effects of missing school reduced absences by 0.07 days in the 1 month following compared to the standard, legalistic, and punitively worded notification—an estimated 40% improvement over the standard truancy notification. This work illustrates how behavioral insights and randomized experiments can be used to improve administrative communications in education.

Sign in / Sign up

Export Citation Format

Share Document