Questioning CRAAP

Librarians and instructors see college students struggle with evaluating information and wonder how to best teach source evaluation in a one-time course integrated library research session to ensure understanding and improve student performance. This research compared multiple sections of first-year students over two semesters taught two evaluation methods: the CRAAP method, and the six journalistic question words. Results indicate that students taught to evaluate information using the six question words produced better end-of-semester papers. Results of the pre-, post-, and end-of-semester quizzes were less conclusive, but do highlight some of the challenges first-year students face when determining credibility. Results have the potential to inform instructional practice.

New tools for assessing information literacy

There is a need for short and easily administered measures for assessing students’ levels of information literacy (IL), as currently existing measures are long and cumbersome. We have therefore created a suite of tools, the “Tromsø Information Literacy Suite” (TROILS), for IL assessment. This suite of tools is freely available on an open platform for others to both use, adapt, and supplement. In this presentation, we introduce three TROILS assessment tools: 1.a test to assess students’ knowledge of key aspects of IL 2.a source evaluation measure to assess students’ abilities to select and critically evaluate sources 3.a source use measure to assess students’ abilities to use sources correctly when writing Together, these tools measure what students know and do regarding key facets of IL. We will discuss the tools’ development and present results of our research with students at different levels higher education.The IL test was developed using procedures intended to ensure acceptable psychometric measurement properties. These included expert consultation for content validity, student think-aloud-protocols for readability, item selection based on a pilot sample, exploratory factor analysis, and measures of reliability and validity. The test was deployed during the fall semester of 2019. In addition to assessing students’ IL levels, test results were used to explore the dimensionality of the IL construct. Results indicate that IL is a heterogeneous construct, and we will discuss important implications of this find for how IL is measured. Results from the source evaluation and source use measures were compared with test results to see whether what the students actually do in their coursework correlates with what they know, based on the test. Results indicate weak to moderate, but statistically significant, correlations. All three measures will be used longitudinally to measure students’ progress over three years.

Knowing and doing

This study touches upon three major themes in the field of information literacy (IL): the assessment of IL, the association between IL knowledge and skills, and the dimensionality of the IL construct. Three quantitative measures were developed and tested with several samples of university students to assess knowledge and skills for core facets of IL. These measures are freely available, applicable across disciplines, and easy to administer. Results indicate they are likely to be reliable and support valid interpretations. By measuring both knowledge and practice, the tools indicated low to moderate correlations between what students know about IL, and what they actually do when evaluating and using sources in authentic, graded assignments. The study is unique in using actual coursework to compare knowing and doing regarding students’ evaluation and use of sources. It provides one of the most thorough documentations of the development and testing of IL assessment measures to date. Results also urge us to ask whether the source-focused components of IL – information seeking, source evaluation and source use – can be considered unidimensional constructs or sets of disparate and more loosely related components, and findings support their heterogeneity.