Using Social Psychology to Constructively Involve Colleagues in Internal Evaluation

Internal evaluators can lead positive, helpful, promotive interactions among colleagues to make evaluation more appropriate, understandable, and accessible. As the motivations and abilities of non-evaluator colleagues are highly varied, interpersonal skills help facilitate cooperation and promote evaluation. Social interdependence theory from social psychology can assist with understanding teamwork and developing interpersonal competence. While the literature links social interdependence theory with evaluation, there is limited understanding of its application in practice. In this article, we share our internal evaluation experiences in three Australian nonprofit organizations to elucidate how we harnessed social connections and interpersonal dynamics to achieve cooperative goals. The purpose of this article is to share our process for using theory and literature to assist with reflecting, planning, and acting, around the way we interact with colleagues. Evaluators may find this process useful when considering how to personalize professional development around interpersonal competencies for evaluation.

Building Evaluation Capacity in Youth-Serving Organizations Through Evaluation Advisory Boards

Youth-serving organizations seek effective and cost-efficient solutions to build evidence and advance their impact. Some common challenges include choosing data systems or assessments, budgeting and planning for 3rd-party studies, and refining measurement and outcomes when programs expand or change. Evaluation advisory boards (EABs) are a low-cost solution to add evaluation capacity and can be mutually beneficial to both youth-serving organizations and evaluation experts. Previous research suggests that EABs may encourage meaningful use of data, support internal evaluators, and/or facilitate difficult conversations among stakeholders. However, there are very few examples of successful EABs in practice. This paper shares the perspectives of EAB members and organizational evaluation leaders from a large national after-school program, After-School All-Stars (ASAS), including (a) a description of the benefits of EABs, (b) how EABs may be especially helpful with the context of the COVID-19 pandemic, and (c) examples of youth-serving organizations’ EABs. The experiences and lessons learned by ASAS and its EAB are generalizable to other non-profit youth development programs. Recommendations for structuring EABs based on organizational goals are provided.

Teacher evaluation by different internal evaluators: Head of departments, teachers themselves, peers and students

<p>This study analyzed teacher evaluation in school, through involving different internal evaluators, in order to determine the extent to which they evaluate teacher performance accurately and objectively. Evaluation survey instruments are used in this study, which are designed based the criteria of existing teacher evaluation system in the context, along with other criteria for evaluating teachers. The sample of this study included teachers, heads of departments and students from high schools in four different districts in Kuwait, received responses as 100 from heads of department, 100 from teachers ‘self-evaluation’, 100 from peer and 912 from students. The findings show that there is no significant difference between teachers’ self-evaluation and heads of departments’ evaluation. On the other hand, this study finds that subjectivism and competition may have an effect on peer evaluation and students may over-evaluate their teachers’ performance as attempt to draw a better picture of their teachers in front of evaluators.</p>

Evaluation Literacy: Perspectives of Internal Evaluators in Non-Government Organizations

While there is an abundance of literature on evaluation use, there has been little discussion regarding internal evaluators’ role in promoting evaluation use. Evaluation can be undervalued if context is not taken into consideration. Evalua­tion literacy is needed to make evaluation more appropriate, understandable, and accessible, particularly in non-government organizations (NGOs) where there is a growing focus on demonstrable outcomes. Evaluation literacy refers to an individ­ual’s understanding and knowledge of evaluation and is an essential component of embedding evaluation into organizational culture. In recognition of the value of the internal perspective, a small exploratory exercise was undertaken to reveal internal evaluator roles and ways of engaging with colleagues around evaluation. Th e exercise examined a key question: What is the role of evaluation literacy in internal evalua­tion in the non-government sector? Three Australian auto-narrative examples from internal evaluators highlight evaluation literacy and locate it among the multiplicity of roles required for optimal evaluation uptake. Analysis of the narratives revealed the underlying issues affecting evaluation use in NGOs and the skills needed to motivate and enable others to access, understand, and use evaluation information. Responding to the call for expanded research into internal evaluation from a practice perspective, the authors hope that the findings will stimulate a wider conversation and further advance understanding of evaluation literacy.

Participatory Guarantee Systems in Peru: Two Case Studies in Lima and Apurímac and the Role of Capacity Building in the Food Chain

Participatory guarantee systems (PGS) are an emerging strategy in the organic farming and agroecology movement for ensuring the sustainable origin of food. This study focused on the perspectives of stakeholders involved in PGS in Peru (Lima and Apurímac) in order to acquire a greater understanding of how these PGS operate and the context in which they are embedded. Qualitative and quantitative approaches were used for data collection in 2016. PGS in Peru have a fairly centralized pyramid structure, with non-governmental organizations and regional farmers’ associations the main driving forces behind PGS implementation. Improved access to markets and additional commercialization channels are major motivations for farmers to participate in PGS, but major difficulties in these two areas are still being encountered in both regions. There is a high demand among farmers for technical training. Farmers acting as internal evaluators play a special role in their local nuclei and are crucial in the PGS process. The PGS in Lima and Apurímac are an important tool in the agro-ecological movement in Peru and offer considerable potential for the support of small-scale farmers. However, there is a need for official recognition and support together with improvements in internal organization and communication for PGS to be able to maintain their principles.

Developing a Cultural Protocol for Evaluation

Evaluation is understood to be important for ensuring programs and organisations are effective and relevant. Evaluation findings, however, can be potentially inappropriate or not useful if those who have an in-depth understanding of the context are not involved in guidance, direction or implementation. The Fred Hollows Foundation's Indigenous Australia Program (IAP), with more than half of its employees identifying as Aboriginal and/or Torres Strait Islander, has developed a cultural protocol for evaluation to strengthen the quality of its program evaluations, whether they are carried out by internal staff or external evaluators. The development of the protocol was initiated after an evaluation capacity building appraisal identified the potential benefits of increased external support to undertake evaluation activities, and the requirement for this external support to be undertaken in a culturally appropriate manner. The protocol was developed by combining IAP's experience and knowledge with contemporary evaluation and research approaches, particularly those developed for use in cross-cultural settings, with the aim of producing a meaningful and locally relevant resource. The protocol aims to assist staff and external evaluators to ensure that evaluation activities are undertaken with the appropriate respect for, and participation of, Aboriginal and Torres Strait Islander individuals and communities. Consistent with IAP principles, those involved in the process of developing the protocols sought to ensure that engagement between staff, evaluators and evaluation participants occurs in culturally-appropriate ways. IAP believes that the protocol will contribute to stronger evaluation practices, deeper understanding and thus, more useful outcomes. This article describes the process of engaging IAP staff with contextual evidence and the literature around cultural protocols to create a meaningful tool that is useful in our particular context. The process of development described will be useful for: organisations undertaking initiatives that source external evaluators; internal evaluators engaging with external expertise; or evaluators linking with organisations working in a cross-cultural setting.

The World Bank's Doing Business Report: A last fling for the Washington Consensus?

The Washington Consensus policies of privatisation and deregulation promoted by the international financial institutions (IFIs) became increasingly controversial during the 1990s, and in 2004 the World Bank's president declared the consensus to be ‘dead’. However, a new push for across-the-board deregulation, notably in the area of workers’ protection, started in 2003 through an annual World Bank publication, Doing Business, which proclaimed a wide range of labour regulations to be nothing more than a hindrance to investment. The IFIs used it to pressure dozens of developing countries to do away with workers’ protection rules, contending that deregulation was necessary to stimulate employment growth, even though the Bank's own internal evaluators were unable to corroborate the claimed link between the Doing Business labour indicator and positive economic outcomes. Faced with mounting pressure from unions, the ILO and elected officials, the Bank finally instructed its staff in 2009 to stop using the indicator and removed it as a conditionality criterion, declaring that the global economic crisis justified adopting a different policy approach.

Evaluation Capacity-building: A Tale of Value-adding

Evaluation capacity-building entails not only developing the expertise needed to undertake robust and useful evaluations; it also involves creating and sustaining a market for that expertise by promoting an organisational culture in which evaluation is a routine part of ‘the way we do things around here’. A challenge for evaluators is to contribute to evaluation capacity-building while also fulfilling their key responsibilities to undertake evaluations. A key strategy is to focus on both discerning value and adding value for clients/commissioners of evaluations. This paper takes as examples two related internal evaluation projects conducted for the Queensland Police Service that have added value for the client and, in doing so, have helped to promote and sustain an evaluation culture within the organisation. It describes key elements of these evaluations that contributed to evaluation capacity-building. The paper highlights the key role that evaluators themselves, especially internal evaluators, can take in evaluation capacity-building, and proposes that internal evaluators can, and should, integrate evaluation capacity-building into their routine program evaluation work.

A Fundamental Choice: Internal or External Evaluation?

An organisation wishes to evaluate one of its programs. It can ask a staff member or hire someone outside the organisation. Which should it choose? Surprisingly little guidance is available for this common scenario. A review of 30 texts dealing with organisational performance and evaluation shows that too often the issue is assumed one way or the other. Management texts aimed at business and organisational audiences tend to presume that evaluation is conducted by internal evaluators, usually managers. By contrast the specialist evaluation literature almost always proceeds from the opposite assumption: that evaluation is undertaken by external evaluators. This paper proposes a series of measures for comparing the strengths and weaknesses of internal and external evaluators. These include cost, knowledge, flexibility, objectivity, accountability, willingness to criticise, ethics and utilisation of results. A set of guidelines is offered to assist organisations in choosing between internal and external evaluation in each particular case.