A Brief Methodology for Researching and Evaluating Serious Games and Game-Based Learning

In this chapter, the authors present a methodology for researching and evaluating Serious Games (SG) and digital (or other forms of) Game-Based Learning (GBL). The methodology consists of the following elements: 1) frame-reflective analysis; 2) a methodology explicating the rationale behind a conceptual-research model; 3) research designs and data-gathering procedures; 4) validated research instruments and tools; 5) a body of knowledge that provides operationalised models and hypotheses; and 6) professional ethics. The methodology is intended to resolve the dilemma between the “generality” and “standardisation” required for comparative, theory-based research and the “specificity” and “flexibility” needed for evaluating specific cases.

The Quest for a Massively Multiplayer Online Game that Teaches Physics

The last 10 years have seen explosive growth in the fields of online gaming. The largest of these games are undoubtedly the Massively Multiplayer Online Games (MMOG), such as World of Warcraft or City of Heroes, which attract millions of users throughout the world every day. The last 20 years have also seen the growth of a new field of physics known as Physics Education Research (PER). This field consists of physicists dedicated to improving how we learn and teach the subject of physics. In this chapter, the author discusses his personal quest to combine PER with a MMOG and create an online virtual world dedicated to teaching Newtonian physics.

What is the “Learning” in Games-Based Learning?

This chapter explores the nature of “learning” in games-based learning and the cognitive and motivational processes that might underpin that learning by drawing on psychological theories and perspectives. Firstly, changing conceptions of learning over the last few decades are reviewed. This is described in relation to the changes in formal learning theories and connections made between learning theory and GBL. Secondly, the chapter reviews empirical research on the learning outcomes that have been identified for GBL, with specific focus on cognitive benefits, school attainment, collaborative working, and the motivational and engaging appeal of games. Finally, an overview of the dominant theoretical perspectives/findings mostly associated with GBL is presented in an attempt to broaden understanding of the potential for GBL in the classroom.

Non-Player Characters and Artificial Intelligence

Serious Games rely on interactive systems to provide an efficient communication medium between the tutor and the user. Designing and implementing such medium is a multi-disciplinary task that aims at an environment that engages the user in a learning activity. User engagement is significantly related to the users’ sense of immersion or his willingness to accept the reality proposed by a game environment. This is a very relevant research topic for Artificial Intelligence (AI), since it requires computational systems to generate believable behaviors that can promote the users’ willingness to enter and engage in the game environment. In order to do this, AI research has been relying on social sciences, in particular psychology and sociology models, to ground the creation of computational models for non-player characters that behave according to the users’ expectations. In this chapter, the authors present some of the most relevant NPC research contributions following this approach.

Executive Functions in Digital Games

Despite enhanced appreciation of the nature and scope of the cognitive advantages of playing games, our understanding of the actual mechanisms responsible for generating and maintaining these remains limited. In this chapter, the authors propose that viewing these changes from the information processing perspective of executive functions will help to elucidate the psychological infrastructure that underpins these gains. They apply Anderson’s model of executive functions to understanding how games support visual-perceptual processing and higher-level thinking and problem solving. As well as extending our appreciation of how digital games can support learning, research on executive functions highlights the implications of the limitations of our cognitive systems for game design.

Psychological Aspects of Serious Games

Over the past fifteen years there has been increasing interest in serious games as a new medium for learning, skill acquisition, and training. Developing and evaluating engaging and effective serious games presents an interdisciplinary challenge. Psychology is at the interface between hard science and social science and is uniquely placed to play an integrative role in advancing our understanding of the characteristics and impacts of serious games. As the diversity of the chapters in this book illustrates, psychologists have wide-ranging interests in serious games. The purpose of the current chapter is to introduce key concepts, constructs, theories, and research in psychology to examine areas where these are relevant to serious games and provide a context for subsequent chapters in the book.

Study Design and Data Gathering Guide for Serious Games’ Evaluation

The objective of this chapter is to provide an overview of the different methods that can be used to evaluate the learning outcomes of serious games. These include Randomised Control Trials (RCT), quasi-experimental designs, and surveys. Case studies of a selection of serious games developed for use in higher education are then presented along with evaluations of these games. The evaluations illustrate the different evaluation methods, along with an assessment of how well the evaluation method performed. Finally, the chapter discusses the lessons learned and compares the experiences with the evaluation methods and their transferability to other games.

Individual Differences in the Enjoyment and Effectiveness of Serious Games

Individuals differ in a myriad of ways and the promise of using a digital game format to teach or train new knowledge and skills is that they may be designed to allow each user to take their own path through the game and therefore create a more person-centered experience. The current chapter explores the research on some of the many individual differences that may be important to the design, use, and success of a serious game. These include factors that influence motivation to play and learn and learner characteristics such as age, gender, and ethnicity. Cognitive characteristics such as working memory and spatial skills can influence the play environment and may actually be improved by regular gaming. Finally, one area that has been much less studied is individual differences in teachers and trainers who often are charged with the implementation of the serious games.

Neurofeedback and Serious Games

Neuroscience as well as computer gaming have rapidly advanced in the last decades. Yet, the combination of both fields is still in its infancy. One example of an emerging alliance is neurofeedback, where participants are required to learn controlling their own brain activity. So far, this kind of training is mostly applied in therapeutic settings, for example improving symptoms in epilepsy, attention-deficit/hyperactivity disorder, or autism spectrum disorder. However, there are some promising approaches that used neurofeedback in everyday situations for healthy subjects. This may prove especially valuable for serious games that aim to improve learning capabilities and cognitive aspects of individual users. The following chapter introduces the basic concepts and standards of neurofeedback. The different non-invasive imaging techniques are introduced along with successful applications in neurofeedback. Finally, benefits and pitfalls for future combinations of neurofeedback and games are discussed: while the former may profit from realistic and motivating video scenarios, the latter is expected to be a tool for evaluating and monitoring the direct effects on the user’s brain.

A Case for Integration

There is growing interest in assessment of student learning within education, not least because assessment practice within some sectors (the UK higher education sector for example) is stagnant: many courses designed independently to the assessment method and assessed through a small number of traditional methods. Games-based learning has shown little deviation from this pattern – games themselves often removed from assessment of the skills they are designed to teach, and in the worst cases from the intended learning outcomes: gamification being a particularly formulaic example. This chapter makes the case for an integrated approach to assessment within learning games and the wider curriculum, drawing on elements within game design that provide natural opportunity for such integration. To demonstrate and evaluate such an approach, integrated assessment case studies (including a full study from the University of Leicester) are presented and discussed.