Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the COVID-19 pandemic

The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6–8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.

A Study on Near Real-Time Carbon Emission of Roads in Urban Agglomeration of China to Improve Sustainable Development under the Impact of COVID-19 Pandemic

In order to achieve the goal of carbon neutrality and explore the impact of COVID-19 on urban road carbon emission, this study applied and improved a near real-time road carbon emission estimation method for typical Chinese urban agglomeration to improve the rapid evaluation of sustainable development. As a result, we recorded the daily road carbon emission for 12 cities in the Beijing–Tianjin–Hebei (JJJ) region under the impact of the epidemic, exploring the road carbon reduction effect caused by COVID-19. Singular value decomposition method was used to analyze the temporal and spatial characteristics of road carbon emission changes among cities and to explore the urban resilience oriented to public events. The results show: (1) In the JJJ region, the carbon reduction effect caused by COVID-19 is significant, but it lasted for a short time. In the three periods—before the epidemic, strict lockdown period, and post-lockdown period for prevention and control—the total daily road carbon emissions in the 12 cities were 170,000–190,000 tons, 90,000–110,000 tons, and 160,000–180,000 tons, respectively. (2) Cities in the JJJ region showed different road carbon reduction potential under short-term administrative control. During the “strict lockdown period” (23 January–25 February 2020), the average change rate of road carbon emissions in Beijing was −78.72%, which had great potential for reduction. However, the average change rates of Xingtai and Zhangjiakou were only −7.53% and −8.66%, respectively. (3) There are spatiotemporal differences in carbon emissions of urban roads in the JJJ region under the impact of the epidemic. During the gradual reduction of COVID-19 restrictions, great differences between cities on weekends and holidays arise, showing the road carbon emissions in Beijing on weekends and holidays are far lower than that in other cities. (4) In the face of public emergencies, the larger the city is and the more complex the function of the city is, the more difficult for the city is to maintain a steady state. This study not only provides an idea for the dynamic monitoring of urban carbon emissions to improve the rapid evaluation of urban sustainable development in post- and pre-lockdown but also fills the gap in the research on the differences in the response of cities to sudden security incidents from the perspective of road carbon emissions.

Deaminative chlorination of aminoheterocycles

Selective modification of heteroatom-containing aromatic structures is in high demand as it permits rapid evaluation of molecular complexity in advanced intermediates. Inspired by the selectivity of deaminases in nature, herein we present a simple methodology that enables the NH2 groups in aminoheterocycles to be conceived as masked modification handles. With the aid of a simple pyrylium reagent and a cheap chloride source, C(sp2)‒NH2 can be converted into C(sp2)‒Cl bonds. The method is characterized by its wide functional group tolerance and substrate scope, allowing the modification of >20 different classes of heteroaromatic motifs (five- and six-membered heterocycles), bearing numerous sensitive motifs. The facile conversion of NH2 into Cl in a late-stage fashion enables practitioners to apply Sandmeyer- and Vilsmeier-type transforms without the burden of explosive and unsafe diazonium salts, stoichiometric transition metals or highly oxidizing and unselective chlorinating agents.

Rapid Evaluations of Innovations: A Scoping Review

BackgroundThere is increasing demand for more rapid evaluation of innovation in health and social care, to support timely decision-making about service redesign. These pressures have increased during the COVID-19 pandemic. Making evaluations more rapid raises challenges in terms of ensuring rigour and the effective use of resources, but assessment of rapid evaluation methodologies has been lacking. MethodsWe conducted a scoping review to map the developing field of methods of rapid evaluation of innovation in health and social care, to describe the existing literature, categorise different approaches to rapid evaluation, and identify knowledge gaps. We searched multiple databases and websites of key organisations. We prioritised studies with relevance to the context of the NHS in England. We extracted information to enable us to classify and map existing studies on key characteristics. We undertook a narrative synthesis to identify the evidence and the gaps; focussing on the different approaches to conducting rapid evaluation in primary research. ResultsWe identified 14069 records from our searches of which 352 explored rapid evaluations of innovations, methods for rapid evaluation or rapid evaluation of implementation. Our scoping review identified four main approaches used for rapid evaluation: (1) Use of a methodology designed specifically for rapid evaluation;(2) Increasing rapidity by doing less or using a less time-intensive methodology;(3) Use of alternative technologies and/or data to increase the speed of an existing evaluation method; (4) Adaptation of part of a non-rapid evaluation DiscussionThis scoping review identified a lack of clarity about ‘rapid evaluation’ but identified some useful preliminary categories. There is very little comparative research on the impact of using rapid rather than standard evaluation. There is a need for clarity and consistency in terms of what constitutes rapid evaluation, the development of specific methodologies for making evaluation more rapid, and assessment of the advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.