Assessment in the Modern Age

This chapter aims to address assessment in the modern age in terms of its importance, challenges and solutions by examining the views of 1,423 users at UK test centres following their recent experience of using two systems which employ computer-based assessment (CBA) and computer-assisted assessment (CAA). Generally speaking, based on the research, which informs the findings presented in this chapter, both systems face similar challenges but there are challenges which are specific to the CAA system. Similarly, both systems may require common solutions to improve user's future experience, but there are solutions which are more relevant to the CAA system. The chapter concludes with a discussion around the UK apprenticeship and a case study of a pilot apprenticeship programme in which CBA and CAA are also integrated.

Animalia

The study described in this chapter is based on a joint World ORT, Israeli Ministry of Education and Pearson initiative to provide an opportunity for international student collaboration on a series of complex science problems. Students from four schools in Israel, three in the United States and one in Mexico, participated in collaborative complex problem-solving on science topics selected by teachers at the participating schools. The intent was to expose students to the realities of collaborating with people under unfamiliar conditions (such as different cultures, languages, and time zones) in order to reach a shared goal, and to foster the value of this practice. The chapter presents the rationale for the project, describes the Animalia mini-course in detail, presents major findings and discusses implications for future curriculum development and further research.

Computer Agent Technologies in Collaborative Assessments

Often in our daily lives we learn and work in groups. In recognition of the importance of collaborative and problem solving skills, educators are realizing the need for effective and scalable learning and assessment solutions to promote the skillset in educational systems. In the settings of a comprehensive collaborative problem solving assessment, each student should be matched with various types of group members and must apply the skills in varied contexts and tasks. One solution to these assessment demands is to use computer-based (virtual) agents to serve as the collaborators in the interactions with students. The chapter presents the premises and challenges in the use of computer agents in the assessment of collaborative problem solving. Directions for future research are discussed in terms of their implications to large-scale assessment programs.

Rich-Media Interactive Simulations

High fidelity measures have proven to be powerful tools for measuring a broad range of competencies and their validity is well documented. However, their high-touch nature is often a deterrent to their use due to the cost and time required to develop and implement them. In addition, given the increased reliance on technology to screen and evaluate job candidates, organizations are continuing to search for more efficient ways to gather the information they need about one's capabilities. This chapter describes how innovative, interactive rich-media simulations that incorporate branching technology have been used in several real-world applications. The main focus is on describing the nature of these assessments and highlighting potential solutions to the unique measurement challenges associated with these types of assessments.

Simulations for Supporting and Assessing Science Literacy

Simulations have become core supports for learning in the digital age. For example, economists, mathematicians, and scientists employ simulations to model complex phenomena. Learners, too, are increasingly able to take advantage of simulations to understand complex systems. Simulations can display phenomena that are too large or small, fast or slow, or dangerous for direct classroom investigations. The affordances of simulations extend students' opportunities to engage in deep, extended problem solving. National and international studies are providing evidence that technologies are enriching curricula, tailoring learning environments, embedding assessment, and providing tools to connect students, teachers, and experts locally and globally. This chapter describes a portfolio of research and development that has examined and documented the roles that simulations can play in assessing and promoting learning, and has developed and validated sets of simulation-based assessments and instructional supplements designed for formative and summative assessment and customized instruction.

The Value of Metacognition and Reflectivity in Computer-Based Learning Environments

The purposes of this chapter are threefold: to explore the research on and relationships among metacognition, reflection, and self-regulated learning; to analyze students' experiences with metacognition, reflection, and self-regulated learning activities in computer-based learning (CBL) courses; and to provide strategies that can be used in a CBL environment to promote students' metacognition, reflection, and self-regulation. A review of underlying frameworks for and prior study findings in metacognition and reflection are presented. Case study findings are also described and form the basis for the suggested strategies. The value and implications of using such strategies are also offered. Finally, future research should address the teaching of metacognition and reflection in CBL environments with an emphasis on real world application.

Digital Competence

This chapter presents a proposal for a conceptual framework of digital competence, which is a civil right and need and is vital for appropriate, intelligent study and functioning in the real world, through means that technology and the internet offer the citizen. Digital competence in the 2010s is a multifaceted complex of a net of literacies that have been updated, reformulated and transformed under the influence of technology. The framework of the digital competency includes eight fields of digital literacies. At the top of the net is digital ethics literacy, outlines the moral core for proper use of technology; at the base are technological literacy and digital reading and writing literacy, comprising the foundation and interface for all the digital literacies, and in between are the digital literacies in these fields: information literacy, digital visual literacy, new media literacy, communication and collaboration literacy and social media literacy. These interconnected literacies compose a synergetic complex of the digital competence framework.

Analyzing Process Data from Technology-Rich Tasks

A key task emerging in item analysis is identification of what constitutes valid and reliable measurement information, and what data support proposed score interpretations. Measurement information takes on many forms with computerized tests. An enormous amount of data is gathered from technology-based items, tracing every click and movement of the mouse and time stamping actions taken, and the data recorded falls into two general categories: process and outcomes. Outcomes are traditional scored answers that students provides in response to prompts, but technology-based item types also provide information regarding the process that students used to answer items. The first consideration to the practical use of such data is the nature of the data generated when learners complete complex assessment tasks. The chapter we propose serves to discuss some possible methodological strategies that could be used to analyze data from such technology-rich testing tasks.

Automated Scoring of Multicomponent Tasks

Assessment of real-world skills increasingly requires efficient scoring of non-routine test items. This chapter addresses the scoring and psychometric treatment of a broad class of automatically-scorable complex assessment tasks allowing a definite set of responses orderable by quality. These multicomponent tasks are described and proposals are advanced on how to score them so that they support capturing gradations of performance quality. The resulting response evaluation functions are assessed empirically against alternatives using data from a pilot of technology-enhanced items (TEIs) administered to a sample of high school students in one U.S. state. Results support scoring frameworks leveraging the full potential of multicomponent tasks for providing evidence of partial knowledge, understanding, or skill.

Automated Scoring in Assessment Systems

Automated scoring of essays is founded upon the pioneer work of Dr. Ellis B. Page. His creation of Project Essay Grade (PEG) sparked the growth of a field that now includes universities and major corporations whose computer programs are capable of analyzing not only essays but short-answer responses to content-based questions. This chapter provides a brief history of automated scoring, describes in general terms how the programs work, outlines some of the current uses as well as challenges, and offers a glimpse of the future of automated scoring.