Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the COVID-19 pandemic

2022 ◽  
pp. 1035719X2110576
Author(s):  
Milbert Gawaya ◽  
Desiree Terrill ◽  
Eleanor Williams
Keyword(s):  
Service Delivery ◽  
Large Scale ◽  
Evaluation Method ◽  
Evaluation Methods ◽  
Lessons Learned ◽  
Evaluation Framework ◽  
Rapid Evaluation ◽  
Evaluation Approach ◽  
Response Planning ◽  
The Impact

The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6–8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.

scholarly journals A virtual hydrological framework for evaluation of stochastic rainfall models

2018 ◽  
Author(s):  
Bree Bennett ◽  
Mark Thyer ◽  
Michael Leonard ◽  
Martin Lambert ◽  
Bryson Bates
Keyword(s):  
Evaluation Method ◽  
South Australia ◽  
Evaluation Methods ◽  
Evaluation Framework ◽  
Streamflow Prediction ◽  
Direct Evaluation ◽  
Rainfall Model ◽  
Rainfall Statistics ◽  
The Impact

Abstract. Stochastic rainfall modelling is a commonly used technique for evaluating the impact of flooding, drought or climate change in a catchment. While considerable attention is given to the development of stochastic rainfall models, significantly less attention is given to performance evaluation methods. Typical evaluation methods employ a variety of rainfall statistics. However, they give limited understanding about which rainfall characteristics are most important for reliable streamflow prediction whenever the simulated rainfall are poor. To address this issue a new evaluation method for rainfall models is introduced, with three key features: (i) streamflow-based – to give a direct evaluation of modelled streamflow performance, (ii) virtual – to avoid the issue of confounding errors in hydrological models or data, and (iii) targeted – to isolate the source of errors according to specific sites and months. The virtual hydrologic evaluation framework is applied to a case study of 22 sites in South Australia. The framework demonstrated that apparently good modelled rainfall can produce poor streamflow predictions, whilst poor modelled rainfall may lead to good streamflow predictions, as catchment processes can dampen or amplify rainfall errors when converted to streamflow. The framework identified the importance of rainfall in the wetting-up months of the catchment cycle (May and June in this case study) for providing reliable predictions of streamflow over the entire year despite their low monthly flow volume. This insight would not have been found using existing methods and highlights the importance of the virtual hydrological evaluation framework for stochastic rainfall model evaluation.

Disaster impacts in a port-city; learning from Beirut's  Port explosion

2021 ◽  
Author(s):  
Steluta topalov
Keyword(s):  
Large Scale ◽  
Focal Point ◽  
Transportation System ◽  
The Philippines ◽  
Lessons Learned ◽  
Social Dimension ◽  
Global Network ◽  
Port Cities ◽  
Residential Units ◽  
The Impact

<p>On 4 august 2020, one of the biggest non-nuclear explosions the world has seen in recent times took place in the Port of Beirut. Caused by the detonation of 2,750 tons of ammonium nitrate, inadequate stored in a warehouse in the port, the blast destroyed much of the city’s port and the surrounding infrastructure and severly  damaged the dense residential and commercial areas within 5 km of the explosion site. The impact of the explosion, which registered as a 3.3 magnitude earthquake according to the U.S. Geological Survey, was felt as far away as the island of Cyprus.</p><p>Athough the event was an technological hazard, the impact of the explosion is similar to a standardised natural disaster.</p><p>According to UNDP, a total of 200 000 residential units were affected with an estimated of 40 000 buildings damaged; 200 people lost their lives, around 6 000 individuals were injuried and around 300 000 people were displaced.</p><p>Such figure are comparable to other large-scale disasters such as Cyclone Vayu in India, which occured in June 2019 or the displacement caused by the Typhoon Vongfong, in the Philippines.</p><p>The frequent increase of the natural disasters  puts pressure on the critical infrastructure of the cities. The disruption of the transportation system,  which is vital for the sustainable daily operations, are having a big impact on the economical, enviromental and social dimension of a city system. Among the various types of transportation system, ports are a focal point because of its strategic role for the economic growth of cities,regions and  global network. In addition, they are nodal points for the social and economical activity of the inhabitants.</p><p>Although the ports have played a key role in the development of their host cities, they are also vulnerable to a broad range of risks and threats because of a particular spatial character: the location at the intersection of land and sea.  </p><p>The study of the Beirut’s Port explosion examines the impact of port failures on the host urban enviroment and the relationship between hazards, vulnerability and the impact. The vulnerability of the port to disasters results  to the vulnerability of its host city. A context –based understanding  of the impact of the disaster and the elements at risk is essential to identify appropriate risk management strategies. The location of the port within the urban environment, in densely populated area, as in case of Beirut are some of the characteristics of the port cities that can magnify the impact of disasters to which they are prone.  The study will focus on a collection of data that records the impact and allows visualisation of the complex patterns of the disaster risk reduction.</p><p>The impact caused by the Beirut’s port explosion reminds us about the important role of the ports in their host cities and how fundamental is to identify the port’s infrastructure  exposure to hazards and risks.  Lessons learned from such event may be useful to reduce disaster risks in the port cities.</p>

scholarly journals Recruitment and Retention Strategies for Community-Based Longitudinal Studies in Urban, Minority Neighborhoods: Lessons Learned from the PARCS Study (Preprint)

2020 ◽  
Author(s):  
Emily B Ferris ◽  
Katarzyna Wyka ◽  
Kelly R. Evenson ◽  
Joan M Dorn ◽  
Lorna Thorpe ◽  
...  
Keyword(s):  
Longitudinal Studies ◽  
Low Income ◽  
Lessons Learned ◽  
Natural Experiments ◽  
Retention Strategies ◽  
Recruitment And Retention ◽  
Four Dimensions ◽  
Evaluation Approach ◽  
Measurement And Evaluation ◽  
The Impact

UNSTRUCTURED Longitudinal, natural experiments provide an ideal evaluation approach to better understand the impact of built environment interventions on community health outcomes, particularly heath disparities. As there are many recruitment and retention challenges inherent to the design of longitudinal, natural experiments, adaptive and iterative recruitment and retention strategies are critical to the success of a study. This paper documents lessons learned from the Physical Activity and Redesigned Community Spaces (PARCS) Study. The PARCS Study, while ongoing, has developed several approaches to improve the recruitment and retention protocols by prioritizing the following four dimensions: 1) building trust with communities; 2) adapting the study protocol to meet participants’ needs and to reflect their capacity for participation; 3) operational flexibility; and 4) measurement and evaluation systems. These strategies may help researchers more successfully recruit and retain participants, particularly in low-income, minority neighborhoods, into longitudinal studies.

scholarly journals The Stopit! programme to reduce bullying and undermining behaviour in hospitals

2018 ◽  
Vol 32 (3) ◽  
pp. 428-443 ◽  
Author(s):  
Graham Benmore ◽  
Steven Henderson ◽  
Joanna Mountfield ◽  
Brian Wink
Keyword(s):  
Evaluation Method ◽  
Realist Evaluation ◽  
Evaluation Framework ◽  
Content Type ◽  
Training Interventions ◽  
Research Findings ◽  
Past Experiences ◽  
The Impact ◽  
Q Sort ◽  
Practical Implications

Purpose The impact of bullying and undermining behaviours on the National Health Service on costs, patient safety and retention of staff was well understood even before the Illing report, published in 2013, that reviewed the efficacy of training interventions designed to reduce bullying and harassment in the outputs. The purpose of this paper is to provide an example of a good programme well evaluated. Design/methodology/approach The methodology follows a broad realist approach, by specifying the underlying programme assumptions and intention of the designers. Three months after the event, Q-sort methodology was employed to group participants into one of three contexts – mechanism – output groups. Interviews were then undertaken with members of two of these groups, to evaluate how the programme had influenced each. Findings Q-sort identified a typology of three beneficiaries from the Stopit! workshops, characterised as professionals, colleagues and victims. Each group had acted upon different parts of the programme, depending chiefly upon their current and past experiences of bullying in hospitals. Research limitations/implications The paper demonstrates the effectiveness of using Q-sort method to identify relevant CMOs in a realist evaluation framework. Practical implications The paper considers the effectiveness of the programme to reduce bullying, rather than teach victims to cope, and how it may be strengthened based upon the research findings and Illing recommendations. Social implications Workplace bullying is invariably implicated in scandals concerning poor hospital practice, poor patient outcomes and staff illness. All too frequently, the sector responds by offering training in resilience, which though helpful, places the onus on the victim to cope rather than the employer to reduce or eliminate the practice. This paper documents and evaluates an attempt to change workplace practices to directly address bullying and undermining. Originality/value The paper describes a new programme broadly consistent with Illing report endorsements. Second, it illustrates a novel evaluation method that highlights rigorously the contexts, mechanisms and outcomes at the pilot stage of an intervention identifies contexts and mechanisms via factor analysis using Q-sort methodology.

The impact of technological innovation upon servitization

2019 ◽  
Vol 30 (7) ◽  
pp. 1097-1114 ◽  
Author(s):  
Bang-Ning Hwang ◽  
Mu-Yen Hsu
Keyword(s):  
Service Delivery ◽  
Technological Innovation ◽  
Large Scale ◽  
Manufacturing Firms ◽  
Multivariate Logistic Regression Model ◽  
Technological Innovations ◽  
Manufacturing Firm ◽  
Content Type ◽  
The Impact ◽  
The Relationship

Purpose For most manufacturing firms, technological innovations are usually the key strategies to gain their competitive advantages. However, competing strategically through service provision is becoming an important strategy for most industries. A growing demand for packaged product and service delivery is blurring the traditional boundaries between manufacturing and service firms. This trend is called “servitization.” Prior research had different perspectives on the relationship between technological innovations and servitization. Some argued that as servitization exerts the innovative convergence of products and services, the possession of appropriate readiness and absorption capacity through technological innovations for a manufacturing firm is critical to the success of servitization. In contrast, some argued that the knowledge gained from developing technological innovations cannot be applied to the creation of services due to the fundamental difference between technology and service. These contradicting arguments motivated the authors to study the relationship between technological innovations and servitization a step further. The paper aims to discuss these issues. Design/methodology/approach To address the research gap, the authors conducted an empirical study based on the large-scale samples from the second Taiwan Community Innovation Survey (Taiwan CIS). A multivariate logistic regression model was applied in the research. Findings The authors found that different types of technological innovations, namely product innovation and process innovation, have different impacts on servitization. The innovativeness level of the technological innovation moderates the relationship between technological innovation and servitization. Based on the above findings, this research specifically explains the causes of the contradictory results of the prior research. Originality/value The values of this research are twofold. Its academic contribution rests on bridging the literature of innovation and servitization, and on providing a model to clarify the relationships among technological innovation type, level of innovativeness and servitization. Its practical contribution lies in its establishment of a guideline that illuminates manufacturing firms reinforcing service delivery through their existing technological innovation trajectory.

scholarly journals Simulation-Based Evaluation of the Performance of Broadband over Power Lines with Multiple Repeaters in Linear Topology of Distribution Substations

2020 ◽  
Vol 10 (19) ◽  
pp. 6879
Author(s):  
Petr Musil ◽  
Petr Mlynek ◽  
Jan Slacik ◽  
Jiri Pokorny
Keyword(s):  
Smart Grids ◽  
Large Scale ◽  
Field Measurements ◽  
Lessons Learned ◽  
Power Lines ◽  
Multiple Streams ◽  
Linear Topology ◽  
Performance Improvements ◽  
Multiple Data ◽  
The Impact

Broadband over Power Lines (BPL) is considered a promising communication technology in the concept of Smart Grids. This paper evaluates networks based on BPL, with a focus on the impact of repeaters in the linear topology of distribution substations. In large-scale Smart Grids network planning, positions of repeaters have to be carefully chosen. This article should help to determine such positions and limitations of BPL linear topology networks. Laboratory and on-field measurements and their results are presented in this article. Results show the impact of repeater’s deployment for different testing methodologies also with regard to other already presented studies. Measured values and the determined impacts of repeaters are later used as input data for simulation of the linear BPL topology in terms of network throughput with multiple streams and bottlenecks. These occur especially on lines shared by multiple communicating nodes. Furthermore, the simulation investigates the balancing time of multiple data streams throughput. The simulation shows that the throughput balancing can occupy a significant time slot, up to tens of seconds before the throughput of different streams balances. Also, the more data is generated, the more time the balancing time takes. Additionally, the throughput drop caused by a repeater is determined into the range of 35–60%. Based on the measurement and simulation results, lessons learned are presented, and possible performance improvements are discussed.

Constraining the model representation of the aerosol life cycle in relation to sources and sinks.

2020 ◽  
Author(s):  
Paul Kim ◽  
Daniel Partridge ◽  
James Haywood
Keyword(s):  
Climate Change ◽  
Life Cycle ◽  
Large Scale ◽  
Climate Model ◽  
Global Climate ◽  
Global Climate Model ◽  
Evaluation Framework ◽  
Microphysical Properties ◽  
The Impact ◽  
Source And Sink

<p>Global climate model (GCM) ensembles still produce a significant spread of estimates for the future of climate change which hinders our ability to influence policymakers. The range of these estimates can only partly be explained by structural differences and varying choice of parameterisation schemes between GCMs. GCM representation of cloud and aerosol processes, more specifically aerosol microphysical properties, remain a key source of uncertainty contributing to the wide spread of climate change estimates. The radiative effect of aerosol is directly linked to the microphysical properties and these are in turn controlled by aerosol source and sink processes during transport as well as meteorological conditions.</p><p>A Lagrangian, trajectory-based GCM evaluation framework, using spatially and temporally collocated aerosol diagnostics, has been applied to over a dozen GCMs via the AeroCom initiative. This framework is designed to isolate the source and sink processes that occur during the aerosol life cycle in order to improve the understanding of the impact of these processes on the simulated aerosol burden. Measurement station observations linked to reanalysis trajectories are then used to evaluate each GCM with respect to a quasi-observational standard to assess GCM skill. The AeroCom trajectory experiment specifies strict guidelines for modelling groups; all simulations have wind fields nudged to ERA-Interim reanalysis and all simulations use emissions from the same inventories. This ensures that the discrepancies between GCM parameterisations are emphasised and differences due to large scale transport patterns, emissions and other external factors are minimised.</p><p>Preliminary results from the AeroCom trajectory experiment will be presented and discussed, some of which are summarised now. A comparison of GCM aerosol particle number size distributions against observations made by measurement stations in different environments will be shown, highlighting the difficulties that GCMs have at reproducing observed aerosol concentrations across all size ranges in pristine environments. The impact of precipitation during transport on aerosol microphysical properties in each GCM will be shown and the implications this has on resulting aerosol forcing estimates will be discussed. Results demonstrating the trajectory collocation framework will highlight its ability to give more accurate estimates of the key aerosol sources in GCMs and the importance of these sources in influencing modelled aerosol-cloud effects. In summary, it will be shown that this analysis approach enables us to better understand the drivers behind inter-model and model-observation discrepancies.</p>

Lessons Learned From the Operation of Large-Scale Reprocessing-Recycling Facilities in France: What Is in It for China?

2017 ◽  
Author(s):  
Jean-Pierre Gros
Keyword(s):  
Nuclear Power ◽  
Power Plants ◽  
Large Scale ◽  
Central Government ◽  
Fuel Cycle ◽  
Public Information ◽  
Lessons Learned ◽  
Mitigation Measures ◽  
Spent Fuel ◽  
The Impact

AREVA has been running since decades nuclear reprocessing and recycling installations in France. Several industrial facilities have been built and used to this aim across the time. Following those decades and with the more and more precise monitoring of the impact of those installations, precise data and lessons-learned have been collected that can be used for the stakeholders of potential new facilities. China has expressed strong interest in building such facilities. As a matter of fact, the issue of accumulation of spent fuel is becoming serious in China and jeopardizes the operation of several nuclear power plants, through the running out of space of storage pools. Tomorrow, with the extremely high pace of nuclear development of China, accumulation of spent fuel will be unbearable. Building reprocessing and recycling installations takes time. A decision has to be taken so as to enable the responsible development of nuclear in China. Without a solution for the back end of its nuclear fuel cycle, the development of nuclear energy will face a wall. This is what the Chinese central government, through the action of its industrial CNNC, has well understood. Several years of negotiations have been held with AREVA. Everybody in the sector seems now convinced. However, now that the negotiation is coming to an end, an effort should be done towards all the stakeholders, sharing actual information from France’s reference facilities on: safety, security, mitigation measures for health protection (of the workers, of the public), mitigation measures for the protection of the environment. Most of this information is public, as France has since years promulgated a law on Nuclear transparency. China is also in need for more transparency, yet lacks means to access this public information, often in French language, so let’s open our books!

Sign in / Sign up

Export Citation Format

Share Document