scholarly journals Norming a VALUE rubric to assess graduate information literacy skills

2016 ◽  
Vol 104 (3) ◽  
Author(s):  
David J. Turbow ◽  
Julie Evener, MLIS
Keyword(s):  
Case Report ◽  
Peer Review ◽  
Information Literacy ◽  
Review Essay ◽  
Health Sciences ◽  
Literacy Skills ◽  
Patient Case ◽  
Rater Reliability ◽  
Student Work ◽  
Information Literacy Skills

Objective: The study evaluated whether a modified version of the information literacy Valid Assessment of Learning in Undergraduate Education (VALUE) rubric would be useful for assessing the information literacy skills of graduate health sciences students.Methods: Through facilitated calibration workshops, an interdepartmental six-person team of librarians and faculty engaged in guided discussion about the meaning of the rubric criteria. They applied the rubric to score student work for a peer-review essay assignment in the ‘‘Information Literacy for Evidence-Based Practice’’ course. To determine inter-rater reliability, the raters participated in a follow-up exercise in which they independently applied the rubric to ten samples of work from a research project in the doctor of physical therapy program: the patient case report assignment.Results: For the peer-review essay, a high level of consistency in scoring was achieved for the second workshop, with statistically significant intra-class correlation coefficients above 0.8 for 3 criteria: ‘‘Determine the extent of evidence needed,’’ ‘‘Use evidence effectively to accomplish a specific purpose,’’ and ‘‘Access the needed evidence.’’ Participants concurred that the essay prompt and rubric criteria adequately discriminated the quality of student work for the peer-review essay assignment. When raters independently scored the patient case report assignment, inter-rater agreement was low and statistically insignificant for all rubric criteria (kappa¼[1]0.16, p.0.05–kappa¼0.12, p.0.05).Conclusions: While the peer-review essay assignment lent itself well to rubric calibration, scorers had a difficult time with the patient case report. Lack of familiarity among some raters with the specifics of the patient case report assignment and subject matter might have accounted for low inter-rater reliability. When norming, it is important to hold conversations about search strategies and expectations of performance. Overall, the authors found the rubric to be appropriate for assessing information literacy skills of graduate health sciences students.

scholarly journals Mapping the Association of College and Research Libraries information literacy framework and nursing professional standards onto an assessment rubric

2017 ◽  
Vol 105 (2) ◽  
Author(s):  
Gloria Willson, MLIS, MPH ◽  
Katelyn Angell, MLIS, MA
Keyword(s):  
Information Literacy ◽  
Information Needs ◽  
Literacy Skills ◽  
Professional Standards ◽  
Research Papers ◽  
Rater Reliability ◽  
Research Libraries ◽  
Undergraduate Nursing ◽  
Information Literacy Skills ◽  
Nursing Professional

Objective: The authors developed a rubric for assessing undergraduate nursing research papers for information literacy skills critical to their development as researchers and health professionals.Methods: We developed a rubric mapping six American Nurses Association professional standards onto six related concepts of the Association of College & Research Libraries (ACRL) Framework for Information Literacy for Higher Education. We used this rubric to evaluate fifty student research papers and assess inter-rater reliability.Results: Students tended to score highest on the “Information Has Value” dimension and lowest on the “Scholarship as Conversation” dimension. However, we found a discrepancy between the grading patterns of the two investigators, with inter-rater reliability being “fair” or “poor” for all six rubric dimensions.Conclusions: The development of a rubric that dually assesses information literacy skills and maps relevant disciplinary competencies holds potential. This study offers a template for a rubric inspired by the ACRL Framework and outside professional standards. However, the overall low inter-rater reliability demands further calibration of the rubric. Following additional norming, this rubric can be used to help students identify the key information literacy competencies that they need in order to succeed as college students and future nurses. These skills include developing an authoritative voice, determining the scope of their information needs, and understanding the ramifications of their information choices.

scholarly journals First Year University Students Arrive with Some Search Skills, But Struggle with Scholarly Sources

2014 ◽  
Vol 9 (2) ◽  
pp. 22
Author(s):  
Cari Merkley
Keyword(s):  
Information Literacy ◽  
Assessment Tool ◽  
Tertiary Education ◽  
Information Source ◽  
Health Sciences ◽  
Literacy Skills ◽  
First Year ◽  
First Year Students ◽  
Scholarly Literature ◽  
Information Literacy Skills

A Review of: Salisbury, F., & Karasmanis, S. (2011). Are they ready? Exploring student information literacy skills in the transition from secondary to tertiary education. Australian Academic & Research Libraries, 42(1), 43-58. Objective – To determine what existing information literacy skills first year students possess upon entering university. Design – Quantitative survey questionnaire. Setting – A research university in Australia. Subjects – 1,029 first year students in the health sciences. Methods – First year students enrolled in the health sciences were asked to complete a paper questionnaire in their first week of classes in 2009. The 20 question survey was distributed in student tutorial groups. The first 10 questions collected information on student demographics, expected library use, and existing information seeking behaviour. The remaining 10 questions tested students’ understanding of information literacy concepts. Data collected from the survey were analyzed using the statistical software SPSS. Main Results – Most of the students who responded to the questionnaire were between the ages of 16 and 21 (84.3%) with only 2.2% over the age of 40. Approximately 15% of respondents had completed some postsecondary university or vocational education prior to enrolling in their current program. The students ranked Google, a friend, and a book as the top three places they would go to find information on something they knew little about. Google was also the most popular choice for finding a scholarly article (35% of respondents), followed by the library catalogue (21%). A large proportion of students correctly answered questions relating to identifying appropriate search terms. For example, one third of the students selected the correct combination of search concepts for a provided topic, and 77% identified that the choice of search phrase could negatively impact search results. Students also demonstrated prior knowledge of the Boolean operator AND, with 38% correctly identifying its use in the related question. Most students were also able to identify key markers of a website’s credibility. Questions relating to ethical information use and scholarly literature proved more challenging. Almost half (45%) of the students said that they did not know the characteristics of a peer reviewed journal article. Twenty five percent of respondents indicated that citing an information source was only necessary in the case of direct quotes, with only 28% correctly identifying the need for citing both quotes and paraphrasing. Only 23% were able to select the example of a journal citation from the list presented. Conclusion – Students enter university with existing strengths in concept identification and basic search formulation, but require the most assistance with locating and identifying scholarly literature and how to cite it appropriately in their work. The findings will inform the development of an online information literacy assessment tool to assist incoming students in identifying areas where they may require additional support as they transition to university.

scholarly journals Wikipedia editing and information literacy: a case study

2017 ◽  
Vol 118 (1/2) ◽  
pp. 5-16 ◽  
Author(s):  
Lydia Dawe ◽  
Ainslie Robinson
Keyword(s):  
Action Research ◽  
Undergraduate Students ◽  
Information Literacy ◽  
Student Assessment ◽  
Literacy Skills ◽  
Student Work ◽  
Content Type ◽  
Self Confidence ◽  
Information Literacy Skills

Purpose This paper aims to evaluate the success of a Wikipedia editing assessment designed to improve the information literacy skills of a cohort of first-year undergraduate health sciences students. Design/methodology/approach In this action research case study (known hereafter as “the project” to differentiate this action research from the students’ own research), students researched, wrote and published Wikipedia articles on Australia-centric health topics. Students were given a pre- and post-test to assess levels of self-confidence in finding, evaluating and referencing information. Student work was also analysed in terms of article length and quantity and the type of information sources used. Findings Tests revealed that students’ self-confidence in their information literacy skills improved overall. Analysis of student work revealed that students wrote longer articles and incorporated more references than expected. References used were of appropriate quality relevant to the article despite minimal instructions. Originality/value There are few studies that investigate information literacy development through Wikipedia editing in Australian universities. This study shows that Wikipedia editing is an effective way to carry out student assessment prior to essay writing and an innovative platform to improve information literacy skills in undergraduate students.

scholarly journals Re-visioning library support for undergraduate educational programmes in an academic health sciences library

2019 ◽  
Vol 13 (2) ◽  
pp. 136
Author(s):  
Denise Smith
Keyword(s):  
Information Literacy ◽  
Information Science ◽  
Relevant Literature ◽  
Health Sciences ◽  
Literacy Skills ◽  
Academic Health ◽  
Ovid Medline ◽  
Information Literacy Skills ◽  
Comprehensive Search ◽  
New Education

McMaster University’s Health Sciences Library (HSL) began to transition to a new liaison service model in early 2018. One of its librarians sought to understand how an academic health sciences library can optimise its support for academic undergraduate programmes. This scoping review of the literature was pursued with the aim to submit an informed recommendation to HSL’s new Education and Lifelong Learning team, so the library could shift its approach to information literacy instruction in a manner that would optimise its outcomes for students and improve relationships with faculty staff. The author searched seven databases: Library, Information Science & Technology Abstracts (LISTA), ProQuest ERIC, OVID Embase, EBSCO CINAHL, OVID Medline, Web of Science and PapersFirst. She developed a robust and comprehensive search strategy that used a combination of subject headings and keywords to describe information literacy, metaliteracy, libraries and health sciences education. The author also hand-searched bibliographies of seminal publications to broaden her search for relevant literature. The findings in this review indicate that metaliteracy as a concept has not been intentionally implemented into information literacy training at academic health sciences libraries. The review finds that it is preferable to integrate information literacy skills directly into course or programme curricula and align those skills with the evidence-based practice skills undergraduates are already learning. Further, establishing a programme that builds on these skills gradually throughout the duration of the academic programme, rather than one-shot library instruction, is also preferred. To achieve success, libraries must build strong collaborative relationships with faculty staff. The author provides recommendations for practice that reflect the findings of this review. Other academic health libraries may benefit from this review by taking into consideration its findings and subsequent recommendations.

scholarly journals Beyond the Library: Using Multiple, Mixed Measures Simultaneously in a College-Wide Assessment of Information Literacy

2016 ◽  
Vol 77 (2) ◽  
pp. 236-262 ◽  
Author(s):  
Brandy Whitlock ◽  
Nassim Ebrahimi
Keyword(s):  
Information Literacy ◽  
Teaching And Learning ◽  
Literacy Skills ◽  
Assessment Tools ◽  
Student Work ◽  
Indirect Measures ◽  
Information Literacy Skills ◽  
Crucial Information ◽  
Effective Teaching And Learning ◽  
Graduating Students

To get the best sense of how graduating students demonstrate information literacy skills and how the institution can improve student learning, the Assessment in Action (AiA) project at Anne Arundel Community College (AACC) deployed a combination of indirect measures and authentic assessment of student work, utilizing assessment tools flexible enough to be deployed across the college. The results of AACC’s AiA project have provided college practitioners and stakeholders with evidence of the extent to which graduating students demonstrate crucial information literacy skills and with data that can inform decisions about how to foster more effective teaching and learning.

scholarly journals Developing and Applying an Information Literacy Rubric to Student Annotated Bibliographies

2013 ◽  
Vol 8 (3) ◽  
pp. 5 ◽  
Author(s):  
Erin E. Rinto
Keyword(s):  
Information Literacy ◽  
Literacy Instruction ◽  
Information Source ◽  
Literacy Skills ◽  
Information Resources ◽  
Student Work ◽  
Information Literacy Instruction ◽  
Evaluative Criteria ◽  
Information Literacy Skills ◽  
The University

Objective – This study demonstrates one method of developing and applying rubrics to student writing in order to gather evidence of how students utilize information literacy skills in the context of an authentic assessment activity. The process of creating a rubric, training scorers to use the rubric, collecting annotated bibliographies, applying the rubric to student work, and the results of the rubric assessment are described. Implications for information literacy instruction are also discussed. Methods – The focus of this study was the English 102 (ENG 102) course, a required research-based writing course that partners the instructors with the university librarians for information literacy instruction. The author developed an information literacy rubric to assess student evaluation of information resources in the ENG 102 annotated bibliography assignment and trained three other librarians how to apply the rubric to student work. The rubric assessed the extent to which students critically applied the evaluative criteria Currency, Relevance, Accuracy, Authority, and Purpose to the information sources in their annotations. At the end of the semester, the author collected up to three de-identified annotated bibliographies from each of the 58 sections of ENG 102. The rubric was applied to up to five annotations in each bibliography, resulting in a total examination of 773 annotations (some sections turned in fewer than 3 samples, and some bibliographies had fewer than 5 annotations). Results – The results of the study showed that students struggle with critically evaluating information resources, a finding that supports the existing information literacy assessment literature. The overwhelming majority of annotations consisted of summative information with little evidence that students used any evaluative criteria when they selected an information source. Of the five criteria examined, Relevance to the student’s research topic and Authority were the most commonly used methods of resource evaluation, while Currency, Accuracy, and Purpose were criteria least-often used. The low average scores on the rubric assessment indicate that students are not adequately learning how to apply this set of information literacy skills. Conclusions – The library instruction sessions for ENG 102 need to move beyond the skills of choosing and narrowing a topic, selecting keywords, and searching in a library database. Students also need more targeted instruction on higher-order skills, particularly how to critically evaluate and question the sources they find. The results of this assessment are being used to refocus the learning outcomes of ENG 102 library sessions so that instruction can better meet student needs. The results are also being used to make the case for further collaboration between ENG 102 and the university library.

Sign in / Sign up

Export Citation Format

Share Document