Principles and practice in impact assessment for academic libraries

2018 ◽  
Vol 119 (1/2) ◽  
pp. 121-134 ◽  
Author(s):  
Christine Urquhart
Keyword(s):  
Performance Evaluation ◽  
Impact Assessment ◽  
Academic Libraries ◽  
Data Analytics ◽  
Open Data ◽  
Library Services ◽  
Content Type ◽  
Qualitative And Quantitative ◽  
Library Assessment ◽  
And Performance

Purpose This paper aims to examine the principles that underpin library assessment, methods used for impact and performance evaluation and how academic libraries should use the findings, and it discusses how value frameworks help. Design/methodology/approach This is a literature review covering aspects of value (value propositions, value co-creation), value frameworks (including the 2015 ACRL framework, Holbrook typology with worked example), data analytics and collaborative projects including LibQUAL+ initiatives and the use of balanced scorecard principles (including a values scorecard). Findings The use of data analytics in library assessment requires collaboration among library services to develop reliable data sets. Scorecards help ongoing impact and performance evaluation. Queries that arise may require a framework, or logic model, to formulate suitable questions and assemble evidence (qualitative and quantitative) to answer new questions about the value of library services. The perceived value framework of Holbrook’s typology, the values scorecard and the ACRL framework all support the deeper level of inquiry required. Research limitations/implications Includes examples of possible application of frameworks. Practical implications A value framework might help data analytic approaches in combining qualitative and quantitative data. Social implications Impact assessment may require assessing how value is co-created with library users in use of e-resources and open data. Originality/value The study contrasts the varying approaches to impact evaluation and library assessment in academic libraries, and it examines more in-depth value frameworks.

Reflections on the value and impact of library and information services

2016 ◽  
Vol 17 (1) ◽  
pp. 5-28 ◽  
Author(s):  
Christine Urquhart ◽  
Jenny Turner
Keyword(s):  
Impact Assessment ◽  
Impact Evaluation ◽  
Library Services ◽  
Content Type ◽  
Quantitative Evidence ◽  
Qualitative And Quantitative ◽  
Library Assessment ◽  
Effectiveness Analysis ◽  
Economic Analyses ◽  
The Impact

Purpose – The purpose of this paper is to critically review methods of impact assessment and economic analyses. Design/methodology/approach – Reviews the literature on impact assessment, and examines some case studies of impact assessment in health libraries. Relates satisfaction and dissatisfaction assessment to impact assessment. Identifies recent impact assessment approaches in public and academic libraries, and explores different uses of terminology for impact and outcome. Reviews literature on economic analysis of value, including cost analyses, cost-effectiveness analysis and return on investment (ROI) studies. Findings – Terms for library assessment (outcome, output, impact, value and benefit) vary among different sectors. Impact evaluation should examine initial assumptions of value carefully, and examine possible gaps in the line of argument linking inputs of library services to presumed impacts (positive and negative). More attention to sampling and response rates is required, to avoid problems such as the “warm glow” effect. Qualitative and quantitative evidence should be carefully integrated to tell the impact story. Economic impact can be estimated, but methodology depends on the service aims and sector. Originality/value – Demonstrates that some of the requirements for traditional impact assessment also affect economic analyses and ROI studies.

Estimation of the knowledge management model for performance measurement in university libraries

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Muhammad Rafi ◽  
Zheng Jian Ming ◽  
Khurshid Ahmad
Keyword(s):  
Knowledge Management ◽  
Performance Evaluation ◽  
Academic Libraries ◽  
Structural Equation ◽  
Academic Library ◽  
Academic Community ◽  
Assessment Model ◽  
Management Model ◽  
Library Services ◽  
Content Type

PurposeThe study aims to expand the literature on evaluating the performance of professionals and academic libraries, rationalizing management and providing reliable services to the academic community. The performance assessment model covers the four components (management competence, professional experience, financial add/projects and library services) in the context of the knowledge management model.Design/methodology/approachBased on quantitative data, the study defines a set of assumptions for testing the four components of performance evaluation within a knowledge management framework to develop appropriate and robust models for improving employee performance and library services. The structural equation model has been applied to sample data from 339 administrative librarians at 190 universities in Pakistan.FindingsStatistical evidence confirms that the applicability of the proposed performance-based model enhances management competence, makes accurate decisions, develops professional skills and strengthens human resource organization and knowledge management techniques in developing the efficiency of academic libraries.Practical implicationsIn the long term, academic leaders and policymakers value investment in the professional development of top library management as they participate in the decision-making process. Organizing training for service employees, supporting innovative research projects and providing library technology infrastructures ultimately improve academic performance and research when integrated into the knowledge management model.Originality/valueSo far, comprehensive literature on performance and knowledge management has been published separately. However, based on the key data collected by senior library administrators using the structured research questionnaire, the comprehensive performance evaluation research based on a knowledge management model is innovative to improve academic library services and close the literature gap.

European academic libraries Key Performance Indicators (KPI)

2019 ◽  
Vol 20 (3) ◽  
pp. 143-158
Author(s):  
Cecile Swiatek
Keyword(s):  
Comparative Study ◽  
Academic Libraries ◽  
Performance Indicators ◽  
Research Performance ◽  
Key Performance Indicators ◽  
Strong Increase ◽  
Library Services ◽  
Content Type ◽  
And Performance ◽  
Teaching Learning

Purpose The French academic libraries directors association (ADBU) launched and updated a European comparative study on Key Performance Indicators (KPI) in Academic Libraries over the 2013–2017 period. The purpose of this paper is to briefly present the ADBU initiative; its main focus is how to use these figures to ground decision making process, notably so as to better assess the contribution of libraries in student success or research performance. Design/methodology/approach As universities and academic libraries are facing research evolutions and teaching/learning new models, a strong increase in students’ number is observed everywhere in Europe. This trend will most probably continue in the coming years and the pressure on academic libraries will grow. That is why the French ADBU took the initiative to launch a European comparative study on key and performance indicators in Academic Libraries over the 2013–2016 period. Data were updated in 2019, covering 2017 figures and including additional countries. Findings Thanks to the European partners and the help of a consultant firm, the first observations were updated with the 2017 figures and the country scope grew broader. Libraries buildings and equipment, space and seats per student, opening days and hours, frequentation rate, library services, users training, staff per student/documentary budgets/total expenditure, staff training, documentation loans/consultation/downloads: these KPI are gathered and can be compared in both relative and absolute value. Originality/value The ADBU initiative is unique, results are widely shared.

The development of performance measures through an activity based benchmarking project across an international network of academic libraries

2014 ◽  
Vol 15 (1/2) ◽  
pp. 58-66 ◽  
Author(s):  
Simon Hart ◽  
Howard Amos
Keyword(s):  
Performance Measures ◽  
Academic Libraries ◽  
Teaching And Learning ◽  
Assessment Practices ◽  
Maturity Model ◽  
International Network ◽  
Content Type ◽  
Capability Maturity ◽  
Library Assessment ◽  
University Life

Purpose – The purpose of this paper is to outline the findings from the initial stages of an activity-based benchmarking project developed across an international network of academic libraries. Through working on a shared response to the question: “if we enable and support the academic endeavour how do we measure our effectiveness?” the network of libraries is using the formal mechanism of benchmarking as a means of continuous improvement. Design/methodology/approach – Actual improvements as a result of benchmarking are known to arise from considering and looking at processes, tools and techniques rather than from simply comparing and reviewing measurements of activity. The establishment of the Matariki Network of Universities provided the opportunity to begin work on international benchmarking amongst the member libraries. The project is a collaborative exercise involving comparisons across the libraries on a selected number of services and activities that are similar across the institutions and are representative of overall library service provision. In this first stage of the project the focus is on support for teaching and learning, specifically activities and programmes that support the transition of first-year students into university life. To simplify the process the libraries have shared details in relation to specific cohorts of students. In order to achieve this, participating libraries used an online collaborative workspace to respond to a series of questions. These responses were analysed to identify common themes, highlight exemplars and select further topics for discussion. Findings – Acknowledging the challenge of international collaboration, processes and mechanisms were developed. It was important to establish a shared language with a set of agreed terms and definitions. Similarly, taking time for each partner to contribute to the project has been valuable. Consideration of each of the libraries responses to the survey questions reveals a diverse number of activities and practices that represent a strong commitment to the needs of students. Drawing on institutional strategic direction and policy, each library allocates substantial resourcing to these activities and practices. The exercise within the benchmarking project produced a valuable set of data for each library to review and learn from. In terms of managing the project, findings are consistent with those reported in the administration of other benchmarking projects. The libraries are in the early stages of developing a series of common international performance measures. It is evident that learning more detail about assessment processes used across each library is necessary to advance the project aims. Further work with the project partners on measuring the effectiveness of their activities will allow the testing of the application of a maturity model for quality improvement of library assessment practices. Research limitations/implications – The project provides the opportunity to develop a series of performance measures that can be verified across an international network of libraries. Sharing information on activities and practices that impact upon the wider institution provides a means to review and improve library assessment practices. Originality/value – This paper outlines the first benchmarking activity in the development of a review of performance measures through an activity-based benchmarking project. The activity included an international network of academic libraries. This work will lead to benchmarking effectiveness measures and the development of a library assessment capability maturity model. This will offer a development path, and a better understanding of progress, to demonstrate value and provide evidence of successful outcomes.

Assessment committees: good practices from ARL libraries

2016 ◽  
Vol 17 (3) ◽  
pp. 224-240 ◽  
Author(s):  
Michelle H. Brannen ◽  
Sojourna J. Cunningham ◽  
Regina Mays
Keyword(s):  
Academic Libraries ◽  
Academic Library ◽  
Content Type ◽  
Good Practices ◽  
Research Libraries ◽  
Association Of Research Libraries ◽  
Library Assessment ◽  
Current State ◽  
Survey Results ◽  
Assessment Activities

Purpose Assessment activities in academic libraries continue to grow as libraries explore assessment endeavors. Ranging from basic stats gathering and reporting to surveys, focus groups, and usability studies and beyond. Many practitioners are finding it necessary to create new processes and programs, with little guidance. The purpose of this paper is to paint a broad picture of assessment activities in Association of Research Libraries (ARL) university libraries with the goal of creating a resource for libraries developing or improving their assessment programs. Design/methodology/approach A survey was developed that asked questions about assessment personnel, activities, mission, and website. A total of 113 surveys were sent to academic library members of ARL. Survey results were analyzed to compile a list of recommended good practices for assessment and working with assessment committees in academic libraries. Findings The investigators had a response rate of 43 percent. The open-ended nature of the survey questions allowed for the respondents to specifically narrow down the problems and opportunities inherent in library assessment committees. Originality/value This study takes the temperature of the current state of assessment programs in ARL libraries, demonstrating the growth of assessment programs. It begins to document the practices of these libraries, particularly in regards to the sometimes informal and hard to track use of committees and other in-house collaborations, as a first step toward developing best practices for the field. The results illuminate productive areas for further study, including investigating how to measure a culture of assessment and maximizing impact of assessment information presented on assessment websites.

A validation of the performance management scale

2018 ◽  
Vol 35 (6) ◽  
pp. 1253-1267 ◽  
Author(s):  
Khahan Na-nan ◽  
Kanokporn Chaiprasit ◽  
Peerapong Pukkeeree
Keyword(s):  
Factor Analysis ◽  
Performance Evaluation ◽  
Performance Management ◽  
Goodness Of Fit ◽  
Mean Square ◽  
Evaluation Performance ◽  
Content Type ◽  
Performance Review ◽  
And Performance ◽  
Fit Index

Purpose The purpose of this paper is to develop a performance management (PM) scale questionnaire that encompasses the pre-requisite, performance planning, performance evaluation, performance review, and performance application dimensions of PM. Design/methodology/approach In the study, the 33 questionnaire questions were first validated using exploratory factor analysis (EFA) and then by confirmatory factor analysis (CFA) along the three performance dimensions. The research sample consists of 330 entrepreneurs. The factor analysis results confirm the validity of the questionnaire as a reliable entrepreneur PM evaluation tool, as evidenced by the composite reliability of 0.845 and the average variance extracted of 0.532. Findings All constructs revealed the acceptable internal consistency reliability. A good model fit was found for the measurement model using several fit index like χ2=449.983, degree of freedom=415, p-value (p)=0.114, goodness of fit index=0.927, adjusted goodness of fit index=0.901, root mean square error of approximation=0.016, and root of mean square residuals=0.032. Research limitations/implications The PM model was examined using EFA and CFA only. A sample with only SMEs entrepreneurs and large sample size and sample area can be used in future research. Practical implications This research paper is an endeavor to explore only the reliability and validity of the PM model. Thus all the five dimension, namely “pre-requisite” “performance planning,” “performance evaluation,” “performance review,” and “performance application” proved out of be reliable and validated when it will be tested in case of SMEs’ high-growth sectors and high-impact sectors. Originality/value The main contribution of this research is that all factors have a good fit and acceptable reliability value; each factor can be measured individually depending on the nature of the research.

Impact of employee pro-organizational unethical behavior on performance evaluation rated by supervisor: a moderated mediation model of supervisor bottom-line mentality

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Xiaohui Zhan ◽  
Yun Liu
Keyword(s):  
Performance Evaluation ◽  
Organizational Behavior ◽  
Hierarchical Linear Modeling ◽  
Mediating Effect ◽  
Mediation Effect ◽  
Linear Modeling ◽  
Bottom Line ◽  
Content Type ◽  
And Performance ◽  
The Relationship

Purpose The topic of employees’ unethical pro-organizational behavior (UPB) has attracted more and more interest in both practice and academic fields. However, previous studies have mainly investigated the antecedents of UPB and little research has discussed the outcome variables. This study aims to provide a comprehensive understanding of the effect mechanism of UPB on employee performance evaluation rated by a supervisor through a leader-member exchange (LMX) and the moderating role of supervisor bottom-line mentality (BLM). Design/methodology/approach This study used a sample consisting of 304 employees and 96 supervisors in several manufacturing firms in China. The authors used hierarchical linear modeling to test the hypotheses, as this was cross-level research. In addition, this paper also uses Mplus7.4 to test the moderating effects of supervisor BLM on the indirect effects between the UPB and performance evaluation by a moderated path analysis. Findings The results confirm that UPB is positively related to performance evaluation rated by supervisors. Additionally, the mediating effect of LMX in the relationship between UPB and performance evaluation is successfully demonstrated. Furthermore, supervisor BLM cross-levelly moderates the relationship between UPB and LMX, as well as moderates the mediation effect of LMX on the correlation between UPB and performance evaluation. Research limitations/implications The primary contribution of this research is building a cross-level model for the effect of UPB on followers’ performance evaluation scored by the supervisor and thereby extending the nomological networks of both UPB and performance evaluation literature. Another contribution the study makes to the literature is that it provides a new perspective to understand how UPB relates to followers’ performance evaluation. Originality/value This is the first study about how and when UPB predicts followers’ performance evaluation rated by the supervisor.

Readiness of academic libraries in South Africa to research, teaching and learning support in the Fourth Industrial Revolution

2020 ◽  
Vol 41 (6/7) ◽  
pp. 355-368 ◽  
Author(s):  
Dennis N. Ocholla ◽  
Lyudmila Ocholla
Keyword(s):  
South Africa ◽  
Conceptual Framework ◽  
Information Literacy ◽  
Academic Libraries ◽  
Industrial Revolution ◽  
Academic Library ◽  
Theory And Practice ◽  
Library Services ◽  
Content Type ◽  
Fourth Industrial Revolution

PurposeIn this paper, we refer to the World Economic Forum in Davos, Switzerland, in 2016, where the concept of the Fourth Industrial Revolution (4IR) was coined by Klaus Schwab, with the reference that it would be building on “the Third, the digital revolution” and would be “characterized by a fusion of technologies that is blurring the lines between the physical, digital, and biological spheres”. While acknowledging that the 4IR will impact on everything, everywhere, including research and libraries, we conceptualize 4IR, and we compare current academic library services/trends in South Africa with 4IR requirements, through the analysis of 26 public university library websites.Design/methodology/approachBesides conceptualization of 4IR, a content analysis of websites of 26 public universities’ libraries in South Africa was achieved followed up with verification of the data by respective libraries through a preliminary research report circulated to them by email. 23 areas were identified as the trends in academic libraries, which included free Wi-Fi in the libraries; 24/7 study areas and access to library resources on and off campus; research commons; makerspace; borrowing ICTs (e.g. laptops); e-resources; e-catalogues; research data services (RDS; RDM, IR); open scholarship; information literacy and reference/bibliographic tools, library as a publisher, among others. Data obtained were captured in Excel and analyzed by the research questions.FindingsThe 4IR concept does not occur often in literature, in relation to academic libraries, but it is implied. The findings show that the libraries are responding well to the revolution through their services, with remarkable innovation and creativity on display. There was a 64% presence of the analyzed trends/services in the libraries, with emerging trends/services such as library as a publisher (4%), robotics/AI (4%), makerspace (8%), RDS (27%), borrowing of ICTs/devices (19%) and user experience (19%) scoring low, while information literacy and digital scholarship (e.g. IR) (88%), e-catalogue and e-resources (92%), group study area (85%) and off campus access (77%) scoring above 75%. The scatter of the trends/services among the university libraries is noted for knowledge sharing of best practice.Research limitations/implicationsIn order to improve accordance with trends, academic libraries have to be better resourced, accessed and used, as well as improve web visibility. The study expects library services to be responsive, resourced and accessible anytime and anywhere, and it provides a conceptual framework and a benchmark for further research and exploration in the country, region and perhaps elsewhere.Practical implicationsThe study can be used for benchmarking current and future academic library services in Africa. The conceptual framework provides an agenda for theoretical discussions and deliberations.Social implicationsThe trends, framework and 4IR representations in the study can inform theory and practice in LIS, particularly in Africa.Originality/valueLinking 4IR to current and future library services provides a tool for academic libraries services benchmarking and development and provides a conceptual framework for theoretical and practical debates and implementation. The study is quite current and appropriate for the ongoing discussions of 4IR implications to academic libraries.

Uncertain risk: assessing open data signals

2020 ◽  
Vol 14 (4) ◽  
pp. 623-637
Author(s):  
Anne L. Washington
Keyword(s):  
Information Asymmetry ◽  
Data Analytics ◽  
Design Methodology ◽  
Open Data ◽  
Digital Government ◽  
Content Type ◽  
Information Signaling ◽  
Future Value ◽  
Data Signal

Purpose Open data resources contain few signals for assessing their suitability for data analytics. The purpose of this paper is to characterize the uncertainty experienced by open data consumers with a framework based on economic theory. Design/methodology/approach Drawing on information asymmetry theory about market exchanges, this paper investigates the practical challenges faced by data consumers seeking to reuse open data. An inductive qualitative analysis of over 2,900 questions asked between 2013 and 2018 on an internet forum identified how a community of 15,000 open data consumers expressed uncertainty about data sources. Findings Open data consumers asked direct questions that expressed uncertainty about the availability, interoperability and interpretation of data resources. Questions focused on future value and some requests were devoted to seeking data that matched known sources. The study proposes a data signal framework that explains uncertainty about open data within the context of control and visibility. Originality/value The proposed framework bridges digital government practice to information signaling theory. The empirical evidence substantiates market aspects of open data portals. This paper provided a needed case study of how data consumers experience uncertainty. The study integrates established theories about risk to improve the reuse of open data.

Societal institutions and control system characteristics: empirical evidence from the UAE

2015 ◽  
Vol 6 (2) ◽  
pp. 268-291 ◽  
Author(s):  
Mayada Abd El-Aziz Youssef ◽  
Essam Moustafa
Keyword(s):  
Performance Evaluation ◽  
Empirical Evidence ◽  
Business Environment ◽  
Management Control ◽  
Survey Method ◽  
Content Type ◽  
The Government ◽  
Business Entities ◽  
And Performance ◽  
And Control

Purpose – This paper aims to explore the existence of two sets of factors societal institutions and management control systems’ (MCS) characteristics in the UAE business entities. Subsequently, this paper empirically examines the bilateral and the multivariate associations between the two sets. The societal institutions include six factors categorised in three main groups: cultural conventions, state structures and policies and skill development and control. The MCS characteristics consist of four factors which are: reliance on formal rules, control over the behaviour of employees, involvement of subordinates in target setting and performance evaluation and scope of information used in performance evaluation. Design/methodology/approach – Whitley’s model (1999) is adopted in the UAE business environment and the analyses are performed at the organisational level. Qualified accounting officials and managers are surveyed. The Kruskal-Wallis test, Spearman’s partial correlation and multiple regression are used for data analyses. Findings – Findings reveal the characteristics of the UAE societal institutions and the MCS in UAE organisations. They also reveal significant associations among four of the societal institution factors and most of the MCS characteristics. The results highlight the role played by the government structures and policies group in influencing the MCS characteristics in the UAE organisations. However, these results do not entirely agree with Whitley’s model. Research limitations/implications – The results of this study are restricted by the typical constraints associated with the survey method. The obtained results have implications for researchers and managers in facilitating the understanding of the relations among the various societal institutions and the MCS characteristics. Originality/value – This research, to the best of the authors’ knowledge, provides significant new empirical evidence into the relation between societal institutions and MCS characteristics in a non-Western economy.

Sign in / Sign up

Export Citation Format

Share Document