Performance of propensity score matching to estimate causal effects in small samples

2020 ◽  
Vol 29 (3) ◽  
pp. 644-658 ◽  
Author(s):  
Anais Andrillon ◽  
Romain Pirracchio ◽  
Sylvie Chevret
Keyword(s):  
Propensity Score ◽  
Propensity Score Matching ◽  
Treatment Effect ◽  
Marrow Transplant ◽  
Small Sample ◽  
Average Treatment Effect ◽  
Sensitivity Analyses ◽  
Small Samples ◽  
Allogeneic Bone ◽  
Matching Methods

Propensity score (PS) matching is a very popular causal estimator usually used to estimate the average treatment effect on the treated (ATT) from observational data. However, opting for this estimator may raise some efficiency issues when the sample size is limited. Therefore, we aimed to evaluate the performance of propensity score matching in this context. We started with a motivating example based on a cohort of 66 children with sickle cell anemia who received either allogeneic bone-marrow transplant or chronic transfusion. We found substantial differences in the ATT estimate according to the model selected for propensity score estimation and subsequent matching. Then, we assessed the performance of the different propensity score matching methods and post-matching analyses to estimate the ATT using a simulation study. Although all selected propensity score matching methods were based of previous recommendations, we found important discrepancies in the estimation of treatment effect between them, underlining the importance of thorough sensitivity analyses when using propensity score matching in the context of small sample sizes.

Impact of mobile money access on internal remittances, consumption expenditure and household welfare in Ghana

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Priscilla Twumasi Baffour ◽  
Wassiuw Abdul Rahaman ◽  
Ibrahim Mohammed
Keyword(s):  
Propensity Score ◽  
Propensity Score Matching ◽  
Treatment Effect ◽  
Average Treatment Effect ◽  
Consumption Expenditure ◽  
Content Type ◽  
Mobile Money ◽  
Money Transfer ◽  
Per Capita ◽  
Transfer Services

PurposeThe purpose of this study is to examine the impact of mobile money access on internal remittances received, per capita consumption expenditure and welfare of household in Ghana.Design/methodology/approachThe study used data from the latest round of the Ghana Living Standards Survey (GLSS 7) and employed the propensity score matching technique to estimate average treatment effect between users and non-users of mobile money transfer services.FindingsThe study finds that using mobile money is welfare enhancing, particularly for poor households and the channel by which it impacts on welfare is through higher internal remittances received and per capita expenditure. The results from the average treatment effect indicate that mobile money users receive significantly higher remittances and consequently spend averagely higher on consumption than non-users.Research limitations/implicationsAlthough the data employed in this study is limited to one country, the findings support the financial inclusion role and developmental impact of mobile money transfer services. Hence, mobile money transfer services should be promoted and facilitated by the telecommunication and financial sector regulators.Originality/valueIn addition to making original contribution to the literature on the welfare impact of mobile money, the study's use of the propensity score matching is unique.

scholarly journals Estimating Treatment Effects of Unit-Based Pricing of Household Solid Waste Disposal

2018 ◽  
Vol 48 (1) ◽  
pp. 21-43
Author(s):  
Christopher Wright ◽  
John M. Halstead ◽  
Ju-Chin Huang
Keyword(s):  
Propensity Score ◽  
Municipal Solid Waste ◽  
Solid Waste ◽  
Propensity Score Matching ◽  
Waste Disposal ◽  
Treatment Effect ◽  
Treatment Effects ◽  
Average Treatment Effect ◽  
Solid Waste Disposal ◽  
Annual Reduction

Propensity score matching is used to estimate treatment effects when data are observational. Results presented in this study demonstrate the use of propensity score matching to evaluate the average treatment effect of unit-based pricing of household trash for reducing municipal solid waste disposal. Average treatment effect of the treated for 34 New Hampshire communities range from an annual reduction of 631 pounds per household to 823 pounds per household. This represents an annual reduction of 42 percent to 54 percent from an average of 1530 pounds per household if a town did not adopt municipal solid waste user fees.

Non-governmental Organisations and Democracy: An Empirical Analysis

2015 ◽  
Vol 21 (4) ◽  
pp. 489-496 ◽  
Author(s):  
Tania Masi
Keyword(s):  
Developing Countries ◽  
Propensity Score ◽  
United Nations ◽  
Propensity Score Matching ◽  
Empirical Analysis ◽  
Treatment Effect ◽  
Average Treatment Effect ◽  
Government Accountability ◽  
The United Nations ◽  
The Impact

AbstractThe United Nations established in 2005 the United Nations Democracy Fund (UNDEF), whose objective is to support projects submitted by national NGOs aimed at increasing government accountability. The purpose of this paper is to investigate the impact of NGOs activity on democracy exploiting the UNDEF database. An empirical analysis based on a propensity score matching (PSM) method is implemented on a sample of 102 developing countries. The findings indicate that the average treatment effect on the treated (ATT) is positive and significant only when countries receive UNDEF-funded NGOs projects for three rounds or more. In this case the Polity IV indicator improves by an average of 1.08 points with respect to the level of 2005.

scholarly journals Impact of Training of Small-Scale Dairy Farmers on Milk Production and Income in Ethiopia

2020 ◽  
Vol 24 (3) ◽  
pp. 1-8
Author(s):  
Wongel Getachew Seble ◽  
Kubota Satoko ◽  
Kanayama Toshihisa ◽  
Tiana Navalona Randrianantoandro ◽  
Hiroichi Kono
Keyword(s):  
Propensity Score ◽  
Propensity Score Matching ◽  
Milk Production ◽  
Treatment Effect ◽  
Smallholder Farmers ◽  
Average Treatment Effect ◽  
Dairy Farmers ◽  
Average Treatment ◽  
Management Condition ◽  
Smallholder Dairy

This paper examined dairy husbandry training impact on milk production and milk income under smallholder farmers’ management condition. A cross-sectional survey was conducted in two districts in Ethiopia and the data was collected from a total of 180 smallholder dairy farmers (60 of the participants were trained on dairy husbandry practices). Propensity Score Matching (PSM) technique was employed to construct suitable comparable group and to calculate the average treatment effect on the treated sample. The average treatment effect on the treated shows that dairy husbandry training increased milk production, volume of milk processed and milk income by about 21.7%, 56.5% and 22.5% respectively. This study confirms that training on dairy husbandry plays great role to bring change in dairy technology adoption which further enhance milk production and milk income under smallholder farmers’ management condition. Keywords: milk income; milk production; Ethiopia; propensity score matching; smallholder dairy farmers, training

scholarly journals Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Daniele Bottigliengo ◽  
Ileana Baldi ◽  
Corrado Lanera ◽  
Giulia Lorenzoni ◽  
Jonida Bejko ◽  
...  
Keyword(s):  
Propensity Score ◽  
Sample Size ◽  
Propensity Score Matching ◽  
Treatment Effect ◽  
Mean Squared Error ◽  
Small Sample Size ◽  
Small Sample ◽  
Squared Error ◽  
Matching Procedure ◽  
Nominal Coverage

Abstract Background Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings. Methods We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature. Results Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement. Conclusions The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.

Implications of Small Samples for Generalization: Adjustments and Rules of Thumb

2016 ◽  
Vol 41 (5) ◽  
pp. 472-505 ◽  
Author(s):  
Elizabeth Tipton ◽  
Kelly Hallberg ◽  
Larry V. Hedges ◽  
Wendy Chan
Keyword(s):  
Observational Studies ◽  
Small Sample ◽  
Average Treatment Effect ◽  
Small Samples ◽  
Sample Sizes ◽  
Random Samples ◽  
Rules Of Thumb ◽  
Large Populations ◽  
Small Sample Sizes ◽  
Combine Information

Background: Policy makers and researchers are frequently interested in understanding how effective a particular intervention may be for a specific population. One approach is to assess the degree of similarity between the sample in an experiment and the population. Another approach is to combine information from the experiment and the population to estimate the population average treatment effect (PATE). Method: Several methods for assessing the similarity between a sample and population currently exist as well as methods estimating the PATE. In this article, we investigate properties of six of these methods and statistics in the small sample sizes common in education research (i.e., 10–70 sites), evaluating the utility of rules of thumb developed from observational studies in the generalization case. Result: In small random samples, large differences between the sample and population can arise simply by chance and many of the statistics commonly used in generalization are a function of both sample size and the number of covariates being compared. The rules of thumb developed in observational studies (which are commonly applied in generalization) are much too conservative given the small sample sizes found in generalization. Conclusion: This article implies that sharp inferences to large populations from small experiments are difficult even with probability sampling. Features of random samples should be kept in mind when evaluating the extent to which results from experiments conducted on nonrandom samples might generalize.

Sign in / Sign up

Export Citation Format

Share Document