scholarly journals Funnel Plots for Assessing Institutional Performance

2021 ◽  
Author(s):  
◽  
Amy Jennings
Keyword(s):  
Random Effects ◽  
Postpartum Haemorrhage ◽  
Missing At Random ◽  
Performance Outcomes ◽  
District Health ◽  
Funnel Plot ◽  
Birth Characteristics ◽  
Funnel Plots ◽  
Minimum Dataset ◽  
Control Limits

<p>This thesis presents a comparison of maternal outcomes for births in New Zealand District Health Boards (DHBs).This is carried out through analysis of the National Minimum Dataset collected by the Ministry of Health for 2007.  The outcome compared is postpartum haemorrhage (PPH) the results are displayed using funnel plots, a useful tool for displaying unbiased information on performance outcomes when comparing institutions.   Exploration of the data found that there are differences in the demographics, maternal and birth characteristics among DHBs. The rates of PPH are different and the population mixes are made up of a range of different proportions of ethnic groups, ages and deprivation indexes. The exploratory analysis found that a large number of factors are associated with PPH. And that birth weight, parity and gestation had a large number of missing observations. These factors are not missing at random and require imputing prior to constructing the funnel plots.  Results show that there is divergence amongst DHBs in the postpartum haemorrhage rate. First a raw PPH rate was plotted and the results indicated there were differences among DHBs. As there are many potential predictors for PPHa logistic regression model was applied to find the most important factors related to PPH. This allows us to apply an adjusted rate for the funnel plot. The risk adjusted funnel plot also indicated differences among DHBs.  Two approaches are taken to account for the overdispersion. A winsorised estimate and a winsorised estimate with a random effects term are applied to the data. The approaches produced different results. The winsorised estimate widened the control limits and the random effects term narrowed the control limits. All four plots identified an extreme outlier and this was later removed from the analysis and the winsorisation funnel plots were rerun. The influential outlier made a difference and from this we can concluded that 2 out 20 DHBs lie outside the 95% control limits. These two DHBs could be stated as having a very low rate of PPH.</p>

scholarly journals Funnel Plots for Assessing Institutional Performance

2021 ◽  
Author(s):  
◽  
Amy Jennings
Keyword(s):  
Random Effects ◽  
Postpartum Haemorrhage ◽  
Missing At Random ◽  
Performance Outcomes ◽  
District Health ◽  
Funnel Plot ◽  
Birth Characteristics ◽  
Funnel Plots ◽  
Minimum Dataset ◽  
Control Limits

<p>This thesis presents a comparison of maternal outcomes for births in New Zealand District Health Boards (DHBs).This is carried out through analysis of the National Minimum Dataset collected by the Ministry of Health for 2007.  The outcome compared is postpartum haemorrhage (PPH) the results are displayed using funnel plots, a useful tool for displaying unbiased information on performance outcomes when comparing institutions.   Exploration of the data found that there are differences in the demographics, maternal and birth characteristics among DHBs. The rates of PPH are different and the population mixes are made up of a range of different proportions of ethnic groups, ages and deprivation indexes. The exploratory analysis found that a large number of factors are associated with PPH. And that birth weight, parity and gestation had a large number of missing observations. These factors are not missing at random and require imputing prior to constructing the funnel plots.  Results show that there is divergence amongst DHBs in the postpartum haemorrhage rate. First a raw PPH rate was plotted and the results indicated there were differences among DHBs. As there are many potential predictors for PPHa logistic regression model was applied to find the most important factors related to PPH. This allows us to apply an adjusted rate for the funnel plot. The risk adjusted funnel plot also indicated differences among DHBs.  Two approaches are taken to account for the overdispersion. A winsorised estimate and a winsorised estimate with a random effects term are applied to the data. The approaches produced different results. The winsorised estimate widened the control limits and the random effects term narrowed the control limits. All four plots identified an extreme outlier and this was later removed from the analysis and the winsorisation funnel plots were rerun. The influential outlier made a difference and from this we can concluded that 2 out 20 DHBs lie outside the 95% control limits. These two DHBs could be stated as having a very low rate of PPH.</p>

Funnel plot control limits to identify poorly performing healthcare providers when there is uncertainty in the value of the benchmark

2016 ◽  
Vol 25 (6) ◽  
pp. 2670-2684 ◽  
Author(s):  
Bradley N Manktelow ◽  
Sarah E Seaton ◽  
T Alun Evans
Keyword(s):  
Statistical Methods ◽  
Binomial Distribution ◽  
Healthcare Providers ◽  
Prediction Intervals ◽  
Tolerance Intervals ◽  
Funnel Plot ◽  
Funnel Plots ◽  
Control Limits

There is an increasing use of statistical methods, such as funnel plots, to identify poorly performing healthcare providers. Funnel plots comprise the construction of control limits around a benchmark and providers with outcomes falling outside the limits are investigated as potential outliers. The benchmark is usually estimated from observed data but uncertainty in this estimate is usually ignored when constructing control limits. In this paper, the use of funnel plots in the presence of uncertainty in the value of the benchmark is reviewed for outcomes from a Binomial distribution. Two methods to derive the control limits are shown: (i) prediction intervals; (ii) tolerance intervals. Tolerance intervals formally include the uncertainty in the value of the benchmark while prediction intervals do not. The probability properties of 95% control limits derived using each method were investigated through hypothesised scenarios. Neither prediction intervals nor tolerance intervals produce funnel plot control limits that satisfy the nominal probability characteristics when there is uncertainty in the value of the benchmark. This is not necessarily to say that funnel plots have no role to play in healthcare, but that without the development of intervals satisfying the nominal probability characteristics they must be interpreted with care.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

scholarly journals Incidence of Herpes Zoster in HIV-Infected Patients Undergoing Antiretroviral Therapy: A Systematic Review and Meta-analysis

2021 ◽  
Vol 10 (11) ◽  
pp. 2300
Author(s):  
Han-Chang Ku ◽  
Yi-Tseng Tsai ◽  
Sriyani-Padmalatha Konara-Mudiyanselage ◽  
Yi-Lin Wu ◽  
Tsung Yu ◽  
...  
Keyword(s):  
Systematic Review ◽  
Herpes Zoster ◽  
Antiretroviral Therapy ◽  
Random Effects ◽  
Meta Analysis ◽  
People Living With Hiv ◽  
Funnel Plots ◽  
Sex With Men ◽  
Treat All ◽  
Living With Hiv

The incidence of herpes zoster (HZ) in patients infected with HIV is higher than that of the general population. However, the incidence of HZ in HIV patients receiving antiretroviral therapy (ART) remains unclear. This meta-analysis aimed to estimate the pooled incidence rate and risk factors for HZ in the post-ART era. We identified studies assessing the incidence of HZ in the post-ART era between 1 January 2000 and 28 February 2021, from four databases. Pooled risk ratios were calculated from 11 articles using a random-effects model. The heterogeneity of the included trials was evaluated by visually inspecting funnel plots, performing random-effects meta-regression and using I2 statistics. Of the 2111 studies screened, we identified 11 studies that were eligible for final inclusion in the systematic review and 8 studies that were eligible for a meta-analysis. The pooled incidence of HZ in the post-ART era (after the introduction of ART in 1997) was 2.30 (95% confidence interval (CI): 1.56–3.05) per 100 person years (PYs). The risks of incidence of HZ among people living with HIV included male sex (AOR: 4.35 (95% CI: 054–2.41)), men who have sex with men (AOR: 1.21 (95% CI: −0.76–1.13)), CD4 count < 200 cells/μL (AOR: 11.59 (95% CI: 0.53–4.38)) and not receiving ART (AOR: 2.89 (95% CI: −0.44–2.56)). The incidence of HZ is substantially lower among HIV infected patients receiving ART than those not receiving ART. Initiating ART immediately after diagnosis to treat all HIV-positive individuals is crucial to minimize the disease burden of HZ.

scholarly journals Dissections of regional lymph nodes for treatment of skin cancer: predicting annual caseloads that will optimise outcomes

2015 ◽  
Vol 97 (1) ◽  
pp. 52-55 ◽  
Author(s):  
JK Dickson ◽  
A Davies ◽  
S Rahman ◽  
C Sethu ◽  
JRO Smith ◽  
...  
Keyword(s):  
Skin Cancer ◽  
Lymph Nodes ◽  
Service Provision ◽  
Regional Lymph Nodes ◽  
Funnel Plot ◽  
Patient Demographics ◽  
South West ◽  
High Prevalence ◽  
Funnel Plots ◽  
Lower Limits

Introduction Dissection of regional lymph nodes (RLNs) can lead to significant morbidity and a high prevalence of complications. Published guidance states that these procedures should be carried out by surgeons who are members of a specialist skin multidisciplinary team who carry out a combined minimum of 15 axillary/groin dissections per year. However, there is little evidence to guide this minimum figure of procedures. We report on the burden of service provision and prevalence of complications across the South West of England and Wales. Methods A 12-month review of dissections of RLNs for skin cancer was undertaken covering five Plastic Surgery Units with a collective catchment of 8.4 million people. Detailed data were collected on patient demographics, pathology, timing of surgery, and prevalence of complications. Results A total of 163 dissections were carried out. Forty-three per cent of patients experienced one or more complication. In that 12-month period, an average of 8 axillary/groin dissections was carried out per surgeon. A funnel plot demonstrated that the prevalence of complications for individual surgeons was within the limit of the plot but, in many cases, this was based only on a relatively small number of procedures per consultant. If surgeons carried out 10 procedures per year, the upper and lower limits on the plot were 73% and 11%, respectively. Conclusions Funnel plots can provide a useful guide as to whether the prevalence of complications for procedures for individual surgeons lies within acceptable limits. Based on these results, 10 procedures per consultant per year should be sufficient to enable meaningful assessment of the prevalence of complications.

scholarly journals META ANALISIS: PENGARUH ANEMIA IBU HAMIL TERHADAP BERAT BAYI LAHIR RENDAH

2020 ◽  
Vol 11 (2) ◽  
pp. 163-177
Author(s):  
Aditianti Aditianti ◽  
Sri Poedji Hastoety Djaiman
Keyword(s):  
Pregnant Women ◽  
Young Women ◽  
Odds Ratio ◽  
Health Workers ◽  
Meta Analysis ◽  
Random Effect ◽  
Forest Plot ◽  
Funnel Plot ◽  
Funnel Plots

Abstract Background: The prevalence of low birth weight (LBW) in Indonesia shows a decrease, but the risk factor for anemia in pregnant women has increased sharply and this has an impact on increasing the prevalence of LBW. Objective: This study aimed to determine the risk of anemia in pregnant women to the prevalence of LBW in several countries. Methods: This study was a meta-analysis using PRISMA. Eleven of the 122,000 studies met criteria for the analysis. Presentation of the data used a forest plot with a random effect statistical model. Results: The combined odds ratio (OR) showed that the effect of anemia in pregnant women on LBW was 1.49 times higher than that of non-anemia mothers (95% CI: 1.26-4.60; p <0.001). The variance was 53,7%. The results of the funnel plots from 11 studies were not evenly distributed so that the information obtained was homogeneous, focusing more on the middle value. Conclusion: There was an effect of anemia in pregnant women with the prevalence of LBW. Detection of anemia in pregnant women needs to be done as early as possible by involving the role of health workers and cadres. Outreach activities for young women at schools and Posyandu must be carried out regularly and continuously.   Keywords: Anemia, LBW, Pregnancy     Abstrak Latar belakang: Prevalensi berat bayi lahir rendah (BBLR ) di Indonesia menunjukkan penurunan namun faktor risiko anemia pada ibu hamil meningkat tajam dan hal ini berdampak pada peningkatan kejadian BBLR. Tujuan: Studi ini bertujuan untuk mengetahui besarnya risiko ibu hamil anemia terhadap kejadian BBLR di beberapa negara. Metode: Studi ini merupakan meta analisis menggunakan PRISMA. Sebelas dari 122.000 studi masuk dalam kriteria untuk dianalisis. Penyajian data menggunakan forest plot dengan model statistik random effect. Hasil: Besar odds ratio (OR) gabungan menunjukkan bahwa pengaruh ibu hamil anemia terhadap BBLR 1,49 kali lebih tinggi dibandingkan ibu yang tidak anemia (95%CI: 1,26-4,60; p<0,001). Besarnya varian 53,7 persen. Hasil funnel plot dari 11 studi ini tidak tersebar secara merata sehingga informasi yang diperoleh homogen, lebih fokus pada nilai tengah. Kesimpulan: Terdapat pengaruh anemia pada ibu hamil dengan kejadian BBLR. Deteksi anemia pada ibu hamil perlu dilakukan sedini mungkin dengan melibatkan peran tenaga keseharan dan kader. Penyuluhan bagi remaja putri di sekolah dan posyandu harus dilakukan secara berkala dan berkesinambungan   Kata kunci: Anemia, BBLR, Kehamilan

scholarly journals Enhancing feedback on performance measures: the difference in outlier detection using a binary versus continuous outcome funnel plot and implications for quality improvement

2020 ◽  
Vol 30 (1) ◽  
pp. 38-45 ◽  
Author(s):  
Laurien Kuhrij ◽  
Erik van Zwet ◽  
Renske van den Berg-Vos ◽  
Paul Nederkoorn ◽  
Perla J Marang-van de Mheen
Keyword(s):  
Intravenous Thrombolysis ◽  
Sensitivity Analyses ◽  
Binary Outcomes ◽  
Continuous Variables ◽  
Continuous Outcome ◽  
Feedback Information ◽  
Funnel Plot ◽  
Care Processes ◽  
Funnel Plots ◽  
The Difference

BackgroundHospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values.MethodsWe examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot.ResultsThe median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses.ConclusionUsing funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives.

Knowledge, Attitude and Practice of Health workers in Using Health Information Management System in Volta and Eastern Regions, Ghana: Findings from the Pre- and Post-Surveys (Preprint)

2021 ◽  
Author(s):  
Young-Ji Lee ◽  
Seohyun Lee ◽  
SeYeon Kim ◽  
Wonil Choi ◽  
Yoojin Jeong ◽  
...  
Keyword(s):  
Data Management ◽  
Random Effects ◽  
Health Workers ◽  
New Technology ◽  
Information Management System ◽  
District Health ◽  
Ordered Logistic Regression ◽  
Knowledge And Practice ◽  
Knowledge Attitude And Practice ◽  
Mch Data

BACKGROUND Despite the increasing attention to electronic health record (EHR) system in the field of global health, most African countries still depend on inconvenient and inaccurate paper-based system. Good Neighbors International and Evaluate 4 Health have recently supported Ghana Health Service (GHS) on rollout of EHR system called e-Tracker in two Regions in Ghana -- Volta (recently renamed to Oti and Volta Regions), and Eastern Regions. The e-Tracker is an Android-based tracker capture app that collects maternal and child health (MCH) data electronically and transmit the data into the District Health Information Management System (DHIMS)-2 managed by District Health Offices in Ghana. The GHS has implemented this new mHealth program in the Community Health Planning and Services (CHPS) in the three regions. OBJECTIVE This study aims to evaluate the improved capacity and behavioral change of health workers in using an EHR technology, the e-Tracker system to deliver MCH services. Specifically, this study assesses the changes in Knowledge, Attitude, and Practice (KAP) of the health workers towards MCH data management by comparing the pre- and post-survey results. METHODS To assess behavioral change towards the e-Tracker system, KAP of frontline health workers were measured through self-administered surveys before and after implementing the e-Tracker system. The surveys were conducted among participants of the e-Tracker system training sessions, with a purposive sampling scheme. A total of 1,124 health workers from Volta and Eastern Regions participated in both pre- and post-surveys. This study used McNemar's Chi-Square test and Wilcoxon signed-rank test for a pre-post comparison analysis. Random-effects ordered logistic regression analysis and random effects panel analysis were also conducted to identify factors associated with the level of KAP. RESULTS The results of the pre-post comparison analysis showed significant improvement in the Knowledge and Practice levels, but the positive response to the Attitude(acceptability) towards electronic data management were reduced compared to the pre-survey. The result of random-effects ordered logistic regression showed that 'days of overwork' was significantly associated with health worker’s Attitude towards this emergent e-Tacker system. CONCLUSIONS In Ghana, the e-Tracker system is planned to be scaled up nationally, increasing the need to evaluate its impact on MCH data management, and user's Attitude for sustainable utilization of the new technology. This study provides empirical evidence that the e-Tracker system has a positive impact on health workers' KAP in managing MCH data. The findings imply that using a tablet computer-based e-Tracker system could enhance Knowledge and Practice on MCH data management. However, efforts to increase acceptability of the new technology among health workers are necessary for sustainable usage of the e-Tracker system and its scale-up.

Training leads to improved performance of Health Unit Management Committees in south western Uganda manuscript

2020 ◽  
Author(s):  
Teddy Kyomuhangi ◽  
Kimberly Manalili ◽  
Jerome Kabakyenga ◽  
Samuel Maling ◽  
George Muganga ◽  
...  
Keyword(s):  
Health Care ◽  
Health Services ◽  
Management Training ◽  
Performance Outcomes ◽  
Training Methods ◽  
Health Unit ◽  
District Health ◽  
Unit Management ◽  
Southwestern Uganda ◽  
Delivery Of Health Services

Abstract Background: A quality health workforce is critical for the development of health systems and effective delivery of health services. In southwestern Uganda, Health Unit Management Committees (HUMCs) are central to the delivery of health care. They also play a key role in facilitating links between health centres and the community, as they comprised of community members. While these teams took part in planning and management training between 2012-2015, no analysis had been done with regards to the outcomes of these training. This study sought, therefore, to determine whether HUMC members saw increased performance outcomes as a result of their training. Methods: The study followed a cross sectional evaluation design and adopted qualitative methods, including Focus Group Discussions (FGDs), Key Informant Interviews (KIIs) and In-Depth Interviews with health unit In-charges (managers), district health team members and project intervention staff. Evaluation was conducted in July 2016 in Bushenyi district in southwestern Uganda. Evaluation was completed in all levels of health care centers and in both urban and rural settings. Data was collected by members of the research team in both Runyankole and English, and translated into English. Results: Findings revealed that HUMCs reported to be more capable of handling issues at the facility as a result of knowledge and skills acquired during trainings. HUMCs identified several key learning themes, including: conflict resolution, strengthened relationships between members and increased community engagement. The training also resulted in several initiatives for increased health care outcomes, including saving schemes for emergency transportation of referrals, construction of placenta pit and canteen, and beautification projects. Overall there were positive feelings towards the training and its relevance for HUMCs’ job performance. Discussion: In examining the results of the study, conclusions can be drawn that training for HUMCs, which had been the first of their kind in this area, increased performance outcomes in health centers. This aligns with similar research, which identified management training for health care management teams as an important factor for improving the delivery of health services.

Detecting and correcting for publication bias in meta-analysis – A truncated normal distribution approach

2016 ◽  
Vol 27 (9) ◽  
pp. 2722-2741 ◽  
Author(s):  
Qiaohao Zhu ◽  
KC Carriere
Keyword(s):  
Normal Distribution ◽  
Publication Bias ◽  
Random Effects ◽  
Effect Size ◽  
Meta Analysis ◽  
Truncated Normal Distribution ◽  
Random Effects Models ◽  
Funnel Plot ◽  
Fixed And Random Effects ◽  
Truncated Normal

Publication bias can significantly limit the validity of meta-analysis when trying to draw conclusion about a research question from independent studies. Most research on detection and correction for publication bias in meta-analysis focus mainly on funnel plot-based methodologies or selection models. In this paper, we formulate publication bias as a truncated distribution problem, and propose new parametric solutions. We develop methodologies of estimating the underlying overall effect size and the severity of publication bias. We distinguish the two major situations, in which publication bias may be induced by: (1) small effect size or (2) large p-value. We consider both fixed and random effects models, and derive estimators for the overall mean and the truncation proportion. These estimators will be obtained using maximum likelihood estimation and method of moments under fixed- and random-effects models, respectively. We carried out extensive simulation studies to evaluate the performance of our methodology, and to compare with the non-parametric Trim and Fill method based on funnel plot. We find that our methods based on truncated normal distribution perform consistently well, both in detecting and correcting publication bias under various situations.

Sign in / Sign up

Export Citation Format

Share Document