Digital library evaluation measures in academic settings: Perspectives from scholars and practitioners

2020 ◽  
pp. 096100062093550
Author(s):  
Iris Xie ◽  
Soohyung Joo ◽  
Krystyna K Matusiak

Evaluation criteria and appropriate measures are critical to the success of digital library evaluation. The key problem is a lack of specific measures, especially from a heuristic perspective, corresponding to diverse dimensions and criteria. This study explored a variety of measures for digital library evaluation focusing on their appropriateness. It also compared the similarities and differences in perceptions of the appropriateness of digital library evaluation measures of two groups. Sixty-one participants were recruited representing scholars and academic digital librarians. The participants were instructed to fill in an in-depth survey consisting of 174 measures associated with 10 evaluation dimensions and 82 criteria. The findings of this study highlight the most appropriate measures in each dimension and show significant agreement in identifying appropriate measures by digital library scholars and librarians. Differences in the rating of the measures were found between the two groups in the dimensions of “interface design,” “system and technology,” “effects on users,” “administration,” “user engagement,” and “context.” The significance of this study lies not only in the integration of the perspectives on measures from both researchers and practitioners, but also in providing an inclusive list of measures to guide practitioners to effectively evaluate digital libraries in academic settings.

Author(s):  
Soohyung Joo ◽  
Iris Xie

The intent of this chapter is to identify constructs and criteria for Digital Library (DL) evaluation based on document analysis. Eighty-five relevant articles and five websites were reviewed to generate the evaluation constructs and criteria. The findings consist of ten constructs, including collection, information organization, interface design, system performance, effects on users, user engagement, services, preservation, sustainability/administration, and context of use with associated criteria for each dimension. In addition, this chapter discusses challenges in DL evaluation research and practices.


Author(s):  
Kijpokin Kasemsap

This chapter gives an overview of digital library topics: digital libraries and information architecture; digital libraries and electronic learning (e-learning); digital libraries and the Semantic Web; digital library evaluation; digital libraries and service quality; and the significance of digital libraries in the digital age. The Internet and the World Wide Web provide the impetus and technological environment for the development and operation of digital libraries in the digital age. Digital libraries comprise digital collections, services, and infrastructure to educationally support the lifelong learning, research, and conservation of the recorded knowledge. Whereas traditional libraries are limited by storage space, digital libraries have the potential to effectively store much more information and documents, because digital information requires very little physical space to contain them. Encouraging digital libraries has the potential to improve academic library performance and gain educational goals in the digital age.


Author(s):  
Ali Shiri

This chapter introduces a new category of digital library user interfaces called metadata-enhanced visual interfaces. Drawing on the earlier information retrieval visual interfaces that have made use of thesauri, this chapter will review and analyze metadata-enhanced visual interfaces to digital libraries based on two key variables, namely metadata elements used for visualization purposes and visual metaphors incorporated into the user interfaces. The aim of this study is to inform the design of visual interfaces for digital libraries through bringing together issues that have roots in such communities as information retrieval, digital libraries, human-computer interaction, and metadata. The findings of this study provide design ideas and implications for digital library interface design in terms of the various metadata-based information search and retrieval features for visualization purposes.


2018 ◽  
Vol 36 (6) ◽  
pp. 1062-1081 ◽  
Author(s):  
Marta Kuzma ◽  
Albina Moscicka

Purpose This paper aims to present the authors’ attempts to evaluate the access to archival maps available in digital libraries. Its aim was to pinpoint the factors determining the effectiveness of access to old maps and to evaluate which Polish libraries provide resources in such a way as to give the users the best chance of finding the materials necessary for their research. Design/methodology/approach The presented research focussed on archival documents from academic libraries accessible from the Polish Digital Libraries Federation and available from Europeana Collections. The evaluation criteria were established along with features that determine the level of difficulty of access to data describing archival documents. The research took into account the way of recording the data about archival documents in metadata. The authors have also analysed the quantity of available resources and the consistency of metadata record. Findings The results of the research have demonstrated that one-third of the analysed libraries deserve a positive evaluation. The digital library of the Warsaw University and the Jagiellonian Digital Library received the best scores in terms of the description of archival documents. Considering the number of resources, the Jagiellonian Digital Library and the Digital Library of the University of Wrocław are positively distinguished. Originality/value The method of evaluating the access to archival maps has been developed. The criteria and features necessary for this evaluation and the way of their interpretation have also been defined. The future goals leading to the improvement of the access to the archival maps have been also presented.


2021 ◽  
Vol 5 (5) ◽  
pp. 731-746
Author(s):  
Dewa Gede Hendra Divayana ◽  
I Putu Wisna Ariawan ◽  
I Made Ardana ◽  
P. Wayan Arta Suyasa

One source of learning in universities is a digital library. In the era of industry 4.0, most universities have implemented digital libraries in supporting the learning process. However, the reality shows that digital library management is still ineffective. Therefore, the implementation of digital libraries needs to be evaluated for determining the digital library effectiveness used as learning resources in supporting the learning process in universities. Many evaluation tools are used to evaluate the effectiveness of digital libraries but have not provided accurate recommendation results to support decision-making. This research presents an innovation in the form of an evaluation tool that can be used to evaluate the digital library effectiveness in universities. That evaluation tool is called the Alkin-WP-based digital library evaluation software. This software is a desktop platform that contains aspects of measuring the digital library effectiveness by referring to the components of the Alkin evaluation model and the WP (Weighted Product) method. This research aimed to show the effectiveness level of the utilization of Alkin-WP-based digital library evaluation software. This research method was R & D (Research & Development) which refers to the ten development stages of the Borg and Gall model. In this research, development was focused only on a few stages, included: usage trials, final product revision, dissemination, and implementation. The subjects involved in assessing the implementation/utilization of the Alkin-WP-based digital library evaluation software were 35 people, in the usage trials were six people, in product revision were three people, and at the stage of dissemination were 15 people. The tools used to collect data were questionnaires and interview guidelines. The data analysis technique used was descriptive quantitative. The effectiveness level of utilizing the Alkin-WP-based digital library evaluation software was 88.34%. It showed that the evaluation software had effective. The impact of this research results on the scientific field of educational evaluation is being able to show the existence of a new evaluation tool based on educational evaluation and artificial intelligence. That evaluation tool can easier for library heads to make policies for revamping digital library services based on accurate recommendations. Doi: 10.28991/esj-2021-01308 Full Text: PDF


Author(s):  
Kennedy Arebamen Eiriemiokhale

This chapter examines copyright issues in a digital library environment. The author observes that the advent of digital technology has greatly accelerated the dissemination and distribution of information which greatly accelerated the dissemination and distribution of information with great speed and accuracy never seen before. The challenges in creating digital libraries include copyright management, development of appropriate technologies, issues related to storage, architectural and technical challenges, collection development challenges and the challenges of user engagement. The chapter recommends ways of managing orphan works and the roles of digital libraries in open access to knowledge. The author concludes that current developments point to the dominant role of information and communication technology in libraries for improved access to information.


2015 ◽  
Vol 116 (3/4) ◽  
pp. 122-135 ◽  
Author(s):  
Dan Albertson

Purpose – The purpose of this study is to synthesize prior user-centered research to develop and present a generalized framework for evaluating visual, i.e. both image and video digital libraries. The primary objectives include comprehensively examining the current state of visual digital library research to: develop a generalized framework applicable for designing user-centered evaluations of visual digital libraries; identify influential experimental factors warranting assessment evaluation as part of specific contexts; and provide examples of applied methods that have been used in research, demonstrating notable findings. Design/methodology/approach – The framework presented in the present study depicts a set of user-centered methodological considerations and examples, synthesized from a review of prior research that provides significant understanding of users and uses of visual information. Findings – Primary components for digital library evaluation, pertaining to user, interaction, system and domain and topic, and their implications for interactive research are presented. Methods, examples and discussion are presented for each primary evaluation component of the framework. Practical implications – Previously applied evaluations and their significance are described and presented as part of the developed framework, providing the importance of each component for practical application in future research and development of interactive visual digital libraries. Originality/value – Visual digital libraries warrant individual assessment, apart from other types of digital collections, as they offer users more ways to retrieve and interact with collection items. The present study complements prior digital library evaluation research by demonstrating the need for a separate framework due to variations influenced by visual information and reporting on evaluations from different perspectives.


2006 ◽  
Vol 1 (1) ◽  
pp. 86
Author(s):  
Susan Haigh

A review of Druin, Allison. “What Children Can Teach Us: Developing Digital Libraries for Children with Children.” The Library Quarterly 75.1 (January 2005): 20-41. Objective – Through use of an interdisciplinary research team that included children, the study aimed to demonstrate that including children in the design of a digital library for children would result in some new approaches that would improve the site’s usability for the target user group. Design – Case study. Setting – The research was conducted at University of Maryland over a four-year period and involved an interdisciplinary research team of adult researchers from information studies, computer science, education, art, and psychology as well as seven children aged 7-11. Subjects – Seven children participated in the design team over two years; 153 children were observed and interviewed in the design phase; and the resulting new approaches were validated post-launch by analysis of International Children’s Digital Library (ICDL) (http://www.icdlbooks.org) users and usage patterns from November 2002-November 2003 (over 90,000 unique users and 19,000 optional questionnaire respondents). Method – The study included seven children in the design team for a digital library of international children’s books, which resulted in new approaches to collection development, cataloguing, and the search interface. In the design phase, research methods involving the seven children included brainstorming techniques, “cooperative inquiry”, low-tech prototyping; and lab use studies. The team also undertook observation and interviews of 153 children engaged in searching and selecting books from public library catalogues. In validating the new approaches that resulted from the design research, the team employed web log analysis, a voluntary online survey, and working with children in local schools to understand their use of ICDL. Main results – The inclusion of children’s viewpoints in the design stage of the ICDL had an impact in three areas: collection development, metadata, and interface design. For collection development, the research showed that kids were interested in books about children from other cultures and other times in history; in animals, both real and make-believe; in books that are sensitive to other cultures; and in books that are in good condition. For metadata, the research showed that children do not distinguish ‘fiction’ and ‘non-fiction’; look for ‘scary stuff’ or ‘gross stuff’; are often seeking books that make them feel a certain way; care about the look of book covers and may recall books by jacket colours; and use free vocabulary like ‘princesses’ and ‘jokes’. For interface design, the children’s involvement led to more search options (utilizing the new categories of metadata that were created), and customization options such as ability to choose different forms and colour palettes for book readers (e.g. the comic book reader, the spiral book reader). Web log and survey data, as well as lab tests, showed that the innovations resulting from the children’s design input were used. Of the over 90,000 unique users who visited the site in its first year, “genre” and “color” were statistically the fourth and fifth most popular search categories. In lab tests, girls used “color” twice as often as boys, and older boys preferred “genre” while younger children did not pay attention to that category. Conclusions – A first conclusion is that children’s input is vital to creating an online library that meets children’s information needs, tendencies and preferences. Also, seven design principles emerged: 1. Children’s input is invaluable and they should be involved in the design of their libraries. 2. Digital collections for children should consider works both contemporary and historical, and in different languages and representative of different cultures. 3. A variety of search interfaces are needed and it is particularly important to express categories with visual icons. 4. Additional metadata can be needed to reflect children’s views of relevant search criteria. 5. Interfaces should be customizable, such as providing various formats of reader that could themselves be customized in colour. 6. Tools should be suitable for use from the home and for collaborative use, such as use by a parent with a child. 7. Innovation requiring high bandwidth must be balanced with a low bandwidth version to assure broadest possible use. Lastly, the researchers concluded that more research is merited to assess the broader impact of digital libraries on children as searchers and readers.


Author(s):  
Ali Shiri ◽  
Emily Villanueva

Digital library evaluation has become increasingly important in information science, yet there has been minimal evaluative work focusing on digital cultural heritage. This article reports on a comprehensive review of methodologies, frameworks and techniques used in the evaluation of cultural heritage digital libraries and archives. Empirical studies are examined using Saracevic’s digital library evaluation framework to identify models, frameworks, and methodologies in the literature and to categorize these past evaluative approaches. Through the classification of evaluative types and trends, we aim to develop a set of guidelines and recommendations for the future evaluation of cultural heritage digital libraries and archives. L'évaluation des bibliothèques numériques est devenue de plus en plus importante en sciences de l'information, mais très peu de travaux d'évaluation ont été axés sur le patrmoine culturel numérique jusqu'à présent. Cet article rend compte d'une revue complet des méthodologies, cadres de références et des techniques utilisés dans l'évaluation des bibliothèques et archives numériques du patrimoine culturel. Les études empiriques sont examinées à l'aide du cadre d'évaluation des bibliothèques numériques de Saracevic pour identifier les modèles, cadres et méthodologie dans la littérature et pour catégoriser ces anciennes approches évaluatives. À travers la classification des types d'évaluation et des tendances, nous visons à développer un ensemble de lignes directrices et de recommandation pour l'évaluation future des bibliothèques et archivers numériques du patrimoine culturel.


Sign in / Sign up

Export Citation Format

Share Document