Comparing Output from Two Methods of Participatory Design for Developing Implementation Strategies: Traditional Contextual Inquiry vs. Rapid Crowd Sourcing

BackgroundParticipatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies.MethodsWe concurrently used two separate participatory design methods: 1) field observations and qualitative interviews and 2) rapid crowd sourcing (an innovation tournament). Our goal was to generate information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison.ResultsObservations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into clusters from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. Strategies derived from observations and interviews were more specific than those from the innovation tournament. Nine strategies (23%) reflected content unique to observations and interviews and 4 (10%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used.ConclusionsThere was substantial overlap in content derived from observations and interviews and the innovation tournament. However, each yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider what each method may elicit and weigh the resources available to invest in the process.Trial RegistrationN/A

A pilot study of participatory and rapid implementation approaches to increase depression screening in primary care

Background Most individuals with depression go unidentified and untreated. In 2016 the US Preventive Services Task Force released guidelines recommending universal screening in primary care to identify patients with depression and to link them to treatment. Feasible, acceptable, and effective strategies to implement these guidelines are needed. Methods This three-phased study employed rapid participatory methods to design and test strategies to increase depression screening at Penn Medicine, a large health system with 90 primary care practices. First, researchers solicited ideas and barriers from stakeholders to increase screening using an innovation tournament—a crowdsourcing method that invites stakeholders to submit ideas to address a workplace challenge. Second, a panel of stakeholders and scientists deliberated over and ranked the tournament ideas. An instant runoff election was held to select the winning idea. Third, the research team piloted the winning idea in a primary care practice using rapid prototyping, an approach that quickly refines and iterates strategy designs. Results The innovation tournament yielded 31 ideas and 32 barriers from diverse stakeholders (12 primary care physicians, 10 medical assistants, 4 nurse practitioners, 2 practice managers, and 4 patient support assistants). A panel of 6 stakeholders and scientists deliberated on the ideas and voted for patient self-report (i.e., through tablet computers, text message, or an online patient portal) as the winning idea. The research team rapid prototyped tablets in one primary care practice with one physician over 5 five-hour shifts to examine the feasibility, acceptability, and effectiveness of the strategy. Most patients, the physician, and medical assistants found the tablets acceptable and feasible. However, patient support assistants struggled to incorporate them in their workflow and expressed concerns about scaling up the process. Depression screening rates were higher using tablets compared to usual care; follow-up was comparable between tablets and usual care. Conclusions Rapid participatory methods engaged and amplified the voices of diverse stakeholders in primary care. These methods helped design an acceptable and feasible implementation strategy that showed promise for increasing depression screening in a primary care setting. The next step is to evaluate the strategy in a randomized controlled trial across primary care practices.

A Pilot Study of Participatory and Rapid Implementation Approaches to Increase Depression Screening in Primary Care

Background: Most individuals with depression go unidentified and untreated. In 2016 the US Preventive Services Task Force released guidelines recommending universal screening in primary care to identify patients with depression and to link them to treatment. Feasible, acceptable, and effective strategies to implement these guidelines are needed. Methods: This three-phased study employed rapid participatory methods to design and test strategies to increase depression screening at Penn Medicine, a large health system with 90 primary care practices. First, researchers solicited ideas and barriers from stakeholders to increase screening using an innovation tournament—a crowdsourcing method that invites stakeholders to submit ideas to address a workplace challenge. Second, a panel of stakeholders and scientists deliberated over and ranked the tournament ideas. An instant runoff election was held to select the winning idea. Third, the research team piloted the winning idea in a primary care practice using rapid prototyping, an approach that quickly refines and iterates strategy designs. Results: The innovation tournament yielded 31 ideas and 32 barriers from diverse stakeholders (12 primary care physicians, 10 medical assistants, 4 nurse practitioners, 2 practice managers, and 4 patient support assistants). A panel of 6 stakeholders and scientists deliberated on the ideas and voted for patient self-report (i.e., through tablet computers, text message, or an online patient portal) as the winning idea. The research team rapid prototyped tablets in one primary care practice with one physician over 5 five-hour shifts to examine the feasibility, acceptability, and effectiveness of the strategy. Most patients, the physician, and medical assistants found the tablets acceptable and feasible. However, patient support assistants struggled to incorporate them in their workflow and expressed concerns about scaling up the process. Depression screening rates were higher using tablets compared to usual care; follow-up was comparable between tablets and usual care. Conclusions: Rapid participatory methods engaged and amplified the voices of diverse stakeholders in primary care. These methods helped design an acceptable and feasible implementation strategy that showed promise for increasing depression screening in a primary care setting. The next step is to evaluate the strategy in a randomized controlled trial across primary care practices.

Different eyes on the same prize: implications of entry timing heterogeneity and incentives for contestant effort in innovation tournament

PurposeThis paper examines an apparent contrast in organizing innovation tournaments; seekers offer contestant-agnostic incentives to elicit greater effort from a heterogeneous pool of contestants. Specifically, the study tests whether and how such incentives and the underlying heterogeneity in the contestant pool, assessed in terms of contestants' entry timing, are jointly associated with contestant effort. Thus, the study contributes to the prior literature that has looked at behavioral consequences of entry timing as well as incentives in innovation tournaments.Design/methodology/approachFor hypothesis testing, the study uses a panel dataset of submission activity of over 60,000 contestants observed in nearly 200 innovation tournaments. The estimation employs multi-way fixed effects, accounting for unobserved heterogeneity across contestants, tournaments and submission week. The findings remain stable across a range of robustness checks.FindingsThe study finds that, on average, late entrant tends to exert less effort than an early entrant (H1). Results further show that the effort gap widens in tournaments that offer higher incentives. In particular, the effort gap between late and early entrants is significantly wider in tournaments that have attracted superior solutions from several contestants (H2), offer gain in status (H3, marginally significant) or offer a higher monetary reward (H4).Originality/valueThe study's findings counter conventional wisdom, which suggests that incentives have a positive effect on contestant behavior, including effort. In contrast, the study indicates that incentives may have divergent implications for contestant behavior, contingent on contestants' entry timing. As the study discusses, these findings have several implications for research and practice of managing innovation tournaments.

Tournaments to Crowdsource Innovation: The Role of Moderator Feedback and Participation Intensity

Firms increasingly use innovation tournaments to crowdsource innovation ideas from customers. This article uncovers antecedents and consequences of customers’ participation intensity over the course of a tournament. More specifically, the authors theorize on the effects that the type and timing of moderating feedback have on tournament participants’ participation intensity, as well as the effect of the latter on idea quality. Through two longitudinal experiments using a commercial innovation tournament platform, the authors show that moderating feedback stimulates ideators’ participation intensity. They find that negative feedback increases participation intensity, as compared to no feedback and positive feedback. Moreover, negative feedback, either provided in isolation or together with positive feedback, is more effective during the early stages than in the later stages of a tournament. Using a large-scale managerial survey, the authors show that higher participation intensity leads to higher idea quality and better business performance. The effect of participation intensity on idea quality is stronger than the effect of number of ideas and as strong as the effect of number of participants on idea quality.

Subsidizing Technological Innovations in the Presence of R&D Spillovers

We analyze a situation where a principal wants to induce two firms to produce an output, for example electricity from renewable energy sources. Firms can undertake non-contractible investments to reduce production cost of the output. Part of these investments spills over and also reduces production cost of the other firm. Comparing a general price subsidy and an innovation tournament, we find that the principal’s expected cost of implementing a given expected output is always higher under the tournament, even though this scheme may lead to more innovation.