Using Training Evaluation to Improve Practice

By providing you with a detailed description of our upgrade training case, we hope you would be able to (a) relate and compare our case descriptions, environment, processes to your organization and environment, (b) Identify similarities and difference between our cases, and (c) Develop possible action plan based on our lesson learn to carry out your training and evaluations needs. In our lesson learned, look for tools that work best for you depending on your environment to conduct training and evaluations (Beebe et al., 2012). Locate or create your training and evaluation data because they are very useful in improving your training for your learners (Phillips, 1997). Following-up extreme feedback after training evaluation (Goad, 2010) either directly in person or through related channel allows you to evaluate and gain more insight into learner's reactions.

Finding the Critical Few

The purpose of this chapter is to describe a successful case study of evaluation and learning transfer in a Global CRO (ICON) located in Dublin, Ireland. The case study focuses on a management development program entitled Management Essentials Two. The evaluation element of the exercise engaged the theories of Kirkpatrick and Phillips as a fundamental approach. The outcomes of the program that were measured included Levels one, two and three of both models. Post program behavior change was selected as a measure of transfer. In a departure from standard evaluation practice the case study employed an original inventory, “System of Training Transfer Assessment Report” (STAR), to measure the factors leading to transfer (Donovan & Darcy, 2011). Regression analysis was used to identify four factors that were critical to transfer of learning at ICON. Lessons for ICON, practitioners, and researchers are drawn.

Align and Redesign

This chapter emphasizes the design and evaluation of a system redesign initiative which included a unique professional development component for program staff within a comprehensive, multi-year initiative of Wyoming Adult Education and Family Literacy (AEFL) programs. Align and Redesign's (A & R) goals in Wyoming were transforming AEFL services to a new type of delivery system, with greater student persistence, improved academic gains, increased social capital skills and successful transitions to college or careers. A & R involved intense professional development for all program directors and instructors. In short, the system was refocused and rebuilt to redesign services. A key question coming out of the substantial training investments Wyoming made in A & R is whether the investments were well spent. Using a mixed-method approach, AEFL transformation is evaluated quantitatively and qualitatively. Preliminary data indicate that something powerful is happening. Findings of the Wyoming case study support replication of Align & Redesign in other states and locations.

Examining the Relationship Between Learning, Continuing Legal Education, and the Improvement of the Practice of the Law

This is a case study about the Kansas Continuing Legal Education (CLE) Commission and its Education Initiative. As the regulator of CLE in the State of Kansas, the Commission ensures that quality course offerings are available for attorneys who are licensed within the state for use in meeting their annual mandatory continuing legal education (MCLE) requirements. The Commission's Education Initiative was focused on discovering current best practices in program development, delivery, and evaluation with the goal of improving the MCLE experience for Kansas attorneys and thus improving the practice of law. Findings from this effort point to innovative efforts currently underway, or aspired to, by providers to evaluate how MCLE disseminates new legal knowledge, increases attorney-to-attorney connections, encourages ethical behavior, and, ultimately, improves the practice of law.

Training's Impact on Time-to-Proficiency for New Bankers in a Financial Services Organization

The evaluation of training investments from a financial perspective is of interest to practitioners, however little support exists to guide the work. Specifically analyzing the return on investment (ROI) of training programs is discussed often, but practiced rarely. This chapter addresses training ROI through concept, theory, and examples. Concept and theory is explored via strategy, financial analysis, and training perspectives. A comprehensive financial services example explains how ROI performance measurement parameters are established and how related training costs are determined to support ROI calculation. The aim of the chapter is to arm training practitioners with the knowledge and framework needed to perform ROI analysis on investments in training.

Impetus for Culture Transformation

In this chapter, the authors describe how Walgreens distribution centers are conducting training evaluations to effectively integrate individuals with disabilities through the Transitional Work Group pre-hire training program. In particular, the aim of the pre-hire training program is to hire employees with disabilities to execute core business functions as an integral part of the organization. As other employers seek to become more inclusive in their hiring practices, there is a need to evaluate and identify individuals with disabilities who can demonstrate mastery of required technical tasks and soft skills, such as social skills, teamwork, and conflict resolution, to ensure job readiness. The authors conclude this chapter by assessing the program and providing recommendations to current and future evaluators of training programs.

The Impacts a Learner Response System Can Have in the Classroom

Organisational leaders are continually looking for ways to get the most out of employee learning. By maximizing learning transfer the organisation places itself in a very powerful position. The introduction of new technologies that help to enhance learning is coming on stream but typically organisations will ask “does it warrant the investment.” This case study demonstrates the role a learner response system can play in the classroom training of a global e-commerce organisation. Using an experimental control group design, this case study focused on learner achievement and learner experience with and without the introduction of a learner response system. Learner achievement was captured using pre- and post-tests, while the experience piece was captured through post-training surveys. Results for both sections were then further investigated using a focus group after the training. The findings from the case study allowed the organisation to conclude that a learner response system is a valuable learning instrument to aid and enhance the transfer of learning for their employees.

A British and Dutch Caribbean Overseas Territories Training Evaluation Case Study

This chapter will focus on the transfer of learning, the third level of the five levels Phillip (1997) model for the evaluation for training. From this impact evaluation, the authors will share the lessons learned from the evaluation of the training component of the multi-site project implemented to strengthen the response of British and Dutch Caribbean overseas territories (OCTs) to the HIV and AIDS epidemic. With a focus on the transfer of learning, this case study brings theory into a real-life application using the Theory of Change evaluation approach and Appreciative Inquiry to examine the impact of the workforce training activities, which were implemented to enhance the prevention efforts to reduce the spread of HIV. Lessons for current and future evaluators are also highlighted.

Raising the Bar

This chapter provides a case study describing the process of using theory in practice in implementing higher level evaluation of training initiatives within one unit of a global financial services organization. The organization has used Kirkpatrick's level one and two evaluations for many years to evaluate training. This case describes the first time a level three evaluation was used, covering why it was necessary, the process for developing, as well as the outcomes of the evaluation. This chapter also describes the influence of organizational culture in the decisions made in the implementation of the evaluation strategy.

Successful Evaluation by Design

The executive management team (EMT) of a Fortune 500 telecommunications corporation identified several key initiatives that were critical to enhancing its global competitive strategy. This transfer resulted in an initial 673% return on the training investment to the corporation, with implications that an anticipated annualized return of 808% can be realized, given continued support and stable external factors. The executive management team (EMT) of a Fortune 500 telecommunications corporation identified several key initiatives that were critical to enhancing its global competitive strategy. The corporate university (University) was engaged to assist in determining if the organization's country managers possessed the requisite knowledge and skills to achieve the objectives set forth in the strategic plan. A training-needs analysis revealed their lack of knowledge and skills in five important areas.