Evaluating Computer-Supported Learning Initiatives

The call for the integration of program evaluation into the development of computer-supported learning environments is ever increasing. Pushed not only by demands from policy makers and grant givers for more accountability within lean times, this trend is due also to the fact that outcomes of computer-supported learning environment projects often fall short of the expectations held by the project teams. The discrepancy between the targets set by the project staff and the outcomes achieved suggests there is a need for formative evaluation approaches (versus summative approaches) that facilitate the elicitation of information that can be used to improve a program while it is in its development stage (c.p., Worthen, Sanders & Fitzpatrick, 1997). While the call for formative evaluation as an integral part of projects that aim to develop complex socio-technical systems is widely accepted, we note a lack of theoretical frameworks that reflect the particularities of these kind of systems and the ways they evolve (c.p., Keil-Slawik, 1999). This is of crucial importance, as formative evaluation will only be an accepted and effective part of a project if it provides information useful for the project staff. Below we outline the obstacles evaluation faces with regard to projects that design computer-supported learning environments, and discuss two promising approaches that can be used in complimentary fashion.

Download Full-text

Evaluating Computer-Supported Learning Initiatives

Encyclopedia of Information Science and Technology, Second Edition ◽

10.4018/978-1-60566-026-4.ch230 ◽

2011 ◽

pp. 1454-1458

Author(s):

John B. Nash ◽

Christoph Richter ◽

Heidrun Allert

Keyword(s):

Learning Environments ◽

Formative Evaluation ◽

Development Stage ◽

Project Teams ◽

Crucial Importance ◽

Theoretical Frameworks ◽

Project Staff ◽

Policy Makers ◽

Effective Part ◽

Evaluation Approaches

The call for the integration of program evaluation into the development of computer-supported learning environments is ever increasing. Pushed not only by demands from policy makers and grant givers for more accountability within lean times, this trend is due also to the fact that outcomes of computer- supported learning environment projects often fall short of the expectations held by the project teams. The discrepancy between the targets set by the project staff and the outcomes achieved suggests there is a need for formative evaluation approaches (versus summative approaches) that facilitate the elicitation of information that can be used to improve a program while it is in its development stage (c.p., Worthen, Sanders & Fitzpatrick, 1997). While the call for formative evaluation as an integral part of projects that aim to develop complex socio-technical systems is widely accepted, we note a lack of theoretical frameworks that reflect the particularities of these kind of systems and the ways they evolve (c.p., Keil-Slawik, 1999). This is of crucial importance, as formative evaluation will only be an accepted and effective part of a project if it provides information useful for the project staff. Below we outline the obstacles evaluation faces with regard to projects that design computer-supported learning environments, and discuss two promising approaches that can be used in complimentary fashion.

Download Full-text

Improving Evaluations in Computer-Supported Learning Projects

Encyclopedia of Distance Learning, Second Edition ◽

10.4018/978-1-60566-198-8.ch162 ◽

2011 ◽

pp. 1142-1147

Author(s):

John B. Nash ◽

Christoph Richter ◽

Heidrun Allert

Keyword(s):

Learning Environments ◽

Formative Evaluation ◽

Development Stage ◽

Project Teams ◽

Crucial Importance ◽

Theoretical Frameworks ◽

Project Staff ◽

Effective Part ◽

Evaluation Approaches ◽

Learning Projects

The call for the integration of program evaluation into the development of computer-supported learning environments is ever-increasing. Pushed not only by demands from policy groups and grant makers who desire greater accountability in lean times, this trend is due also because outcomes of computer supported learning environment projects often fall short of the expectations held by the project teams. The discrepancy between the targets set by the project staff and the outcomes achieved suggests there is a need for formative evaluation approaches (vs. summative approaches) that derive information that can be used to improve a program while it is in its development stage (see Worthen, Sanders, & Fitzpatrick, 1997). And in spite of the known benefits of integrating evaluation into the project development process, we note a lack of theoretical frameworks that reflect the peculiarities of computer-supported learning projects and the ways they evolve (see Keil-Slawik, 1999). This is of crucial importance, as formative evaluation will only be an accepted and effective part of a project if it provides information useful for the project staff. The purpose of this chapter is to outline the obstacles to integrating evaluation in computer-supported learning projects and then discuss two promising approaches that can be used to address these challenges.

Download Full-text

Improving Evaluations in Computer-Supported Learning Projects

Encyclopedia of Distance Learning ◽

10.4018/978-1-59140-555-9.ch153 ◽

2011 ◽

pp. 1048-1053

Author(s):

John B. Nash ◽

Christoph Richter ◽

Heidrun Allert

Keyword(s):

Learning Environments ◽

Formative Evaluation ◽

Development Stage ◽

Project Teams ◽

Crucial Importance ◽

Theoretical Frameworks ◽

Project Staff ◽

Effective Part ◽

Evaluation Approaches ◽

Learning Projects

The call for the integration of program evaluation into the development of computer-supported learning environments is ever-increasing. Pushed not only by demands from policy groups and grant makers who desire greater accountability in lean times, this trend is due also because outcomes of computer supported learning environment projects often fall short of the expectations held by the project teams. The discrepancy between the targets set by the project staff and the outcomes achieved suggests there is a need for formative evaluation approaches (vs. summative approaches) that derive information that can be used to improve a program while it is in its development stage (see Worthen, Sanders, & Fitzpatrick, 1997). And in spite of the known benefits of integrating evaluation into the project development process, we note a lack of theoretical frameworks that reflect the peculiarities of computer-supported learning projects and the ways they evolve (see Keil-Slawik, 1999). This is of crucial importance, as formative evaluation will only be an accepted and effective part of a project if it provides information useful for the project staff. The purpose of this chapter is to outline the obstacles to integrating evaluation in computer-supported learning projects and then discuss two promising approaches that can be used to address these challenges.

Download Full-text

Information Systems Evaluation

Information Systems Research and Exploring Social Artifacts ◽

10.4018/978-1-4666-2491-7.ch017 ◽

2013 ◽

pp. 333-354 ◽

Cited By ~ 1

Author(s):

Si Chen ◽

Nor Mardziah Osman ◽

Guo Chao Alex Peng

Keyword(s):

Information Systems ◽

Formative Evaluation ◽

Current System ◽

Evaluation Research ◽

Evaluation Methodologies ◽

Is Implementation ◽

Is Research ◽

Is Evaluation ◽

Evaluation Approaches ◽

Goals And Objectives

Due to the prevalent use of Information Systems (IS) in modern organisations, evaluation research in this field is becoming more and more important. In light of this, a set of rigorous methodologies were developed and used by IS researchers and practitioners to evaluate the increasingly complex IS implementation used. Moreover, different types of IS and different focusing perspectives of the evaluation require the selection and use of different evaluation approaches and methodologies. This chapter aims to identify, explore, investigate, and discuss the various key methodologies that can be used in IS evaluation from different perspectives, namely in nature (e.g. summative vs. formative evaluation) and in strategy (e.g. goal-based, goal-free, and criteria-based evaluation). Six case studies are also presented and discussed in this chapter to illustrate how the different IS evaluation methodologies can be applied in practices. The chapter concludes that evaluation methodologies should be selected depending on the nature of the IS and the specific goals and objectives of the evaluation. Nonetheless, it is also proposed that formative criteria-based evaluation and summative criteria-based evaluation are currently among the more widely used in IS research. The authors suggest that the combined used of one or more of these approaches can be applied at different stages of the IS life cycle in order to generate more rigorous and reliable evaluation outcomes. Moreover, results and outcomes of IS evaluation research will not just be useful in practically guiding actions to improve the current system, but can also be used to generate new knowledge and theory to be adopted by future IS research.

Download Full-text

Enhancing Student-Driven Learning Strategies by Examining the Faculty Conceptions of Teaching

Student-Driven Learning Strategies for the 21st Century Classroom - Advances in Educational Technologies and Instructional Design ◽

10.4018/978-1-5225-1689-7.ch019 ◽

2017 ◽

pp. 300-317

Author(s):

Despina Varnava Marouchou

Keyword(s):

Professional Development ◽

Student Learning ◽

Learning Environments ◽

Learning Strategies ◽

Curriculum Design ◽

Teaching Practices ◽

Level Of Detail ◽

Critical Examination ◽

Policy Makers ◽

Conceptions Of Teaching

Despite the extensive interest demonstrated by academics and policy makers, and the vast amount of information written, during the past twenty years or so, in the area of student learning, very little of what has been written has been transformed into solid policies and practices, “the discourse advocates for learning, but seldom gets down to the level of detail” (Weimer 2002, p. xii). By making instructors aware of their own conceptions of teaching they will be informed of how such conceptions influence their teaching practices. Thus, a critical examination of the conceptions of teaching can initiate adjustments directed towards a reform-based perspective, leading to student driven learning strategies. In this chapter the findings of the conceptions of teaching identified by a group of lecturers in a small university are discussed, secondly, the possibility of a relation between the lecturers' conceptions of teaching and student-driven learning strategies are outlined and thirdly, the importance of taking these issues into consideration, when exploring learning environments, learning outcomes, curriculum design and the professional development of academics.

Download Full-text

Drawing on Design to Improve Evaluation of Computer Supported Collaborative Learning

Online Collaborative Learning ◽

10.4018/978-1-59140-174-2.ch013 ◽

2004 ◽

pp. 281-310

Author(s):

John B. Nash ◽

Christoph Richter ◽

Heidrun Allert

Keyword(s):

Social Context ◽

Collaborative Learning ◽

Human Computer Interaction ◽

Learning Environments ◽

Program Theory ◽

Computer Supported Collaborative Learning ◽

Theoretical Frameworks ◽

Design Learning ◽

Evaluative Thinking

This chapter addresses theoretical frameworks for the evaluation of computer-supported learning environments. It outlines the characteristics and obstacles this evaluation must face with regard to projects that design learning experiences, stressing the notion that human-computer interaction is imbedded in social context that is complex and dynamic. The authors examine how scenario-based design and program theory can contribute to the design and evaluation of computer-supported collaborative learning (CSCL) and present a case study in which both approaches are applied. Based on the revealed complementary frameworks, a compelling approach is drafted that combines both of them. Our goal is to make CSCL designers more aware of the benefits of evaluative thinking in their work and to introduce two tangible approaches to evaluation that, when implemented as a design step, can strengthen CSCL initiatives.

Download Full-text

Did We Become a Community? Multiple Methods for Identifying Community and Its Constituent Elements in Formal Online Learning Environments

Web-Based Education ◽

10.4018/978-1-61520-963-7.ch109 ◽

2010 ◽

pp. 1607-1625

Author(s):

Richard A. Schwier ◽

Ben Daniel

Keyword(s):

Learning Communities ◽

Learning Environments ◽

Sense Of Community ◽

Interaction Analysis ◽

Paired Comparison ◽

Classroom Community ◽

Bayesian Belief Network ◽

User Perceptions ◽

Evaluation Approaches ◽

Constituent Elements

To understand the nature of formal virtual learningcommunities in higher education, we are employing a variety of user-centred evaluation approaches to examinemethods for determining whether a community exists, andif it does, to isolate and understand interactions among itsconstituent elements, and ultimately to build a model offormal virtual learning communities. This chapter presents the methods we are employing to answer these seemingly simple questions, including user perceptions of community(Sense of Community Index, Classroom Community Scale), interaction analysis (density, reciprocity,) content analysis (transcript analysis, interviews, focus groups), paired-comparison analysis (Thurstone scaling) and communitymodeling techniques (Bayesian Belief Network analysis).

Download Full-text

Government Information Sharing

Electronic Governance and Cross-Boundary Collaboration ◽

10.4018/978-1-60960-753-1.ch002 ◽

2012 ◽

pp. 23-55 ◽

Cited By ~ 7

Author(s):

Elsa Estevez ◽

Pablo Fillottrani ◽

Tomasz Janowski ◽

Adegboyega Ojo

Keyword(s):

Information Sharing ◽

Government Information ◽

Theoretical Frameworks ◽

Policy Makers ◽

Holistic View ◽

Government Response ◽

Related Information ◽

Conceptual Clarity ◽

One Stop ◽

Abstraction Levels

Information sharing (IS) is a key capability required for one-stop and networked government, responding to a variety of intra-organizational, inter-organizational, or cross-national needs like sharing service-related information between parties involved in the delivery of seamless services, sharing information on available resources to enable whole-of-government response to emergencies, etc. Despite its importance, the IS capability is not common for governments due to various technical, organizational, cultural, and other barriers which are generally difficult to address by individual agencies. However, developing such capabilities is a challenging task which requires government-wide coordination, explicit policies and strategies, and concrete implementation frameworks. At the same time, reconciling existing theoretical frameworks with the IS practice can be difficult due to the differences in conceptions and abstraction levels. In order to address such difficulties, this chapter proposes a conceptual framework to guide the development of Government Information Sharing (GIS) policies, strategies, and implementations. By integrating theoretical frameworks and the GIS practice, the framework adopts a holistic view on the GIS problem, highlights the main areas for policy intervention, and provides policy makers and government managers with conceptual clarity on the GIS problem.

Download Full-text

Teleworking and a Green Computing Environment

Research Anthology on Digital Transformation, Organizational Change, and the Impact of Remote Work ◽

10.4018/978-1-7998-7297-9.ch018 ◽

2021 ◽

pp. 332-352

Author(s):

Iheanyi Chuku Egbuta ◽

Brychan Thomas ◽

Said Al-Hasan

Keyword(s):

Conceptual Model ◽

Driving Forces ◽

Green Computing ◽

Computing Environment ◽

Theoretical Frameworks ◽

Policy Makers ◽

Working Definition ◽

Worker Satisfaction ◽

Definition Of ◽

Business Sectors

This paper considers strategic green issues of teleworking in terms of the environment, transport, location, office space and resource use for modern organisations and business sectors and formulates a conceptual model of the processes involved. Teleworking technologies are variously implemented for green computing initiatives and many advantages include lower greenhouse gas emissions related to travel, greater worker satisfaction and, as a result of lower overhead office costs, increased profit margins. The paper initially investigates the appropriateness of a working definition of teleworking with regard to green computing and explores the benefits, and barriers, of teleworking in a green computing environment. Theoretical frameworks and models of teleworking are considered and a conceptual model of the contribution of teleworking to green computing is formulated. The application of the model is considered in terms of teleworking concepts, the organisational environment and driving forces. It is the intention of the paper to identify, and articulate, those teleworking concepts that will be useful to academicians, scientists, business entrepreneurs, practitioners, managers and policy makers, and to indicate future directions for research scholars and students with similar interests.

Download Full-text