Evaluating Free-Choice Climate Education Interventions Applying Propensity Score Matching

Background and objectives. The majority of environmental education takes place in informal settings, of which so-called free-choice learning is typical. What is understood by this is a kind of learning which is self-determined and driven by the needs and interests of the learner. The voluntariness of participation in interventions and the fact that they take place in turbulent action settings lead to formidable challenges for impact evaluation, particularly because often no randomized control trials (RCTs) or before-and-after measurements can be carried out. In this article, the evaluation of five different interventions from a large-scale program on consumer climate education provides the empirical background for illustrating the problems confronting quasi-experimental impact analyses in a free-choice context and presenting a possible solution based on propensity score matching (PSM). Method. In a quasi-experimental control-group design, intervention participants and nonparticipants filled out questionnaires featuring the same behavioral intentions. The challenges due to selection processes were met with radius matching on the basis of sociodemographic characteristics as covariates. Sensitivity analyses on the basis of Rosenbaum Bounds and the Hodges-Lehmann point estimator were used for assessing the robustness of treatment effects against unobserved confounding variables. Results. The analyses show that all the interventions under study positively influenced intentions to seek further advice or information on the topics covered. Furthermore, for all the interventions, significant positive effects on the intentions relating directly to climate-friendly behavior could be identified. In this context, PSM and sensitivity analyses proved to be effective methods. However, there were several limitations due to conceptual and methodological issues, and these are discussed below.