Information Systems Evaluation

2013 ◽  
pp. 333-354 ◽  
Author(s):  
Si Chen ◽  
Nor Mardziah Osman ◽  
Guo Chao Alex Peng
Keyword(s):  
Information Systems ◽  
Formative Evaluation ◽  
Current System ◽  
Evaluation Research ◽  
Evaluation Methodologies ◽  
Is Implementation ◽  
Is Research ◽  
Is Evaluation ◽  
Evaluation Approaches ◽  
Goals And Objectives

Due to the prevalent use of Information Systems (IS) in modern organisations, evaluation research in this field is becoming more and more important. In light of this, a set of rigorous methodologies were developed and used by IS researchers and practitioners to evaluate the increasingly complex IS implementation used. Moreover, different types of IS and different focusing perspectives of the evaluation require the selection and use of different evaluation approaches and methodologies. This chapter aims to identify, explore, investigate, and discuss the various key methodologies that can be used in IS evaluation from different perspectives, namely in nature (e.g. summative vs. formative evaluation) and in strategy (e.g. goal-based, goal-free, and criteria-based evaluation). Six case studies are also presented and discussed in this chapter to illustrate how the different IS evaluation methodologies can be applied in practices. The chapter concludes that evaluation methodologies should be selected depending on the nature of the IS and the specific goals and objectives of the evaluation. Nonetheless, it is also proposed that formative criteria-based evaluation and summative criteria-based evaluation are currently among the more widely used in IS research. The authors suggest that the combined used of one or more of these approaches can be applied at different stages of the IS life cycle in order to generate more rigorous and reliable evaluation outcomes. Moreover, results and outcomes of IS evaluation research will not just be useful in practically guiding actions to improve the current system, but can also be used to generate new knowledge and theory to be adopted by future IS research.

Critical Realism

2010 ◽  
pp. 57-76 ◽  
Author(s):  
Sven A. Carlsson
Keyword(s):  
Critical Realism ◽  
Design Science ◽  
Evaluation Research ◽  
Structuration Theory ◽  
Science Research ◽  
Is Research ◽  
Is Theory ◽  
Is Evaluation ◽  
Is Design ◽  
Structural Theories

Different strands of non-positivistic research approaches and theories, for example, constructivism, grounded theory, and structuration theory, have gained popularity in the information systems (IS) field. Although, they are managing to overcome some problems with positivism and structural theories they are not completely without problems. This chapter puts critical realism forward as an alternative philosophical underpinning for IS research. Critical realism starts from an ontology that identifies structures and mechanisms, through which events and discourses are generated, as being fundamental to the constitution of our natural and social reality. The chapter presents critical realism and how it can be used in IS research. Examples of how critical realism have been used and can be used in research aiming at generating new IS theory, IS evaluation research, and IS design science research are provided.

L'évaluation des Ressources Alternatives: À La Recherche De Modèles Alternatifs En Évaluation

1985 ◽  
Vol 4 (2) ◽  
pp. 57-71
Author(s):  
Céline Mercier
Keyword(s):  
Mentally Ill ◽  
Financial Support ◽  
Formative Evaluation ◽  
Social Research ◽  
Evaluation Research ◽  
Service Delivery System ◽  
Service Network ◽  
Systemic Model ◽  
Evaluation Approaches ◽  
General Service

Following the current literature, alternative resources for the mentally ill have already demonstrated their efficiency. However, the question remains of their role and specificity within the general service network in the field of mental health, especially when looking at their development and financial support. Social research, especially evaluation research, can contribute an answer to these questions concerning the organization of the service delivery system. This paper focuses on various evaluation approaches (systemic model, course of implementation and formative evaluation) of alternative resources and expectancies towards research.

scholarly journals An empirically sound telemedicine taxonomy – applying the CAFE methodology

2021 ◽  
Author(s):  
Lorenz Harst ◽  
Lena Otto ◽  
Patrick Timpel ◽  
Peggy Richter ◽  
Hendrikje Lantzsch ◽  
...  
Keyword(s):  
Content Analysis ◽  
Information Systems ◽  
Statistical Analysis ◽  
Quantitative Data ◽  
Evaluation Research ◽  
Is Research ◽  
Coding Scheme ◽  
Research Fields ◽  
Quantitative Content ◽  
Structure Research

Abstract Aim Because the field of information systems (IS) research is vast and diverse, structuring it is a necessary precondition for any further analysis of artefacts. To structure research fields, taxonomies are a useful tool. Approaches aiming to develop sound taxonomies exist, but they do not focus on empirical development. We aimed to close this gap by providing the CAFE methodology, which is based on quantitative content analysis. Subject and methods Existing taxonomies are used to build a coding scheme, which is then validated on an IS project database. After describing the methodology, it is applied to develop a telemedicine taxonomy. Results The CAFE methodology consists of four steps, including applicable methods. It helps in producing quantitative data for statistical analysis to empirically ground any newly developed taxonomy. By applying the methodology, a taxonomy for telemedicine is presented, including, e.g. application types, settings or the technology involved in telemedicine initiatives. Conclusion Taxonomies can serve in identifying both components and outcomes to analyse. As such, our empirically sound methodology for deriving those is a contribution not only to evaluation research but also to the development of future successful telemedicine or other digital applications.

scholarly journals The Delphi Method in Information Systems Research (2004-2017)

2019 ◽  
Author(s):  
Ayman Alarabiat ◽  
Isabel Ramos
Keyword(s):  
Information System ◽  
Information Systems ◽  
Delphi Method ◽  
Is Research ◽  
Systems Research ◽  
Information Systems Research ◽  
Innovative Techniques ◽  
Delphi Studies ◽  
Procedural Recommendations ◽  
Do So

After almost 30 years of being used in the information system (IS) discipline, only a few studies have focused on how IS scholars apply the method's guidelines to design Delphi studies. Thus, this paper focuses on the use of the Delphi method in IS research. To do so, articles published between 2004 and 2017 in the Senior IS Scholars' collection of journals of the Association of Information Systems (AIS), describing Delphi studies, were analised. Based on analysis of sixteen (16) retrieved IS studies, we concluded that IS researchers have applied the method’s most important phases and the procedural recommendations to promote rigor were considered in the majority of the analised studies. Nonetheless, IS researchers still need to include detailed information about (1) the steps taken to ensure the validity of the achieved results, (2) better describe the process of selecting and recruiting the experts, and (3) experiment with innovative techniques to keep participants involved in the Delphi process.

Social Drivers for ICT Diffusion among Agrarian Business Enterprises in Nigeria

2013 ◽  
pp. 91-104
Author(s):  
Bartholomew Aleke ◽  
Udechukwu Ojiako ◽  
David Wainwright
Keyword(s):  
Developing Countries ◽  
Social Network ◽  
Information Systems ◽  
Diffusion Of Innovation ◽  
Social Network Theory ◽  
Conceptual Basis ◽  
Diffusion Of Innovation Theory ◽  
Is Research ◽  
Business Enterprises ◽  
Successful Innovation

In this paper, the authors examine the process of diffusion of innovation ICT within agrarian business enterprises operating in developing countries. There is substantial research in the area of Diffusion of Innovation Theory (DoI) and its application to Information Systems (IS) research within organisations. However, in recognition of the conceptual limitations of DoI, researchers have called for the incorporation of aspects of Social Network Theory (SNT) into DoI frameworks. The findings of this research suggest that an understanding of the conceptual basis of innovation is a major driver of successful innovation adoption.

Extending IMPLEMENT Framework for Enterprise Information Systems Implementation to Information System Innovation

2017 ◽  
pp. 137-177 ◽  
Author(s):  
Aparna Raman ◽  
D. P. Goyal
Keyword(s):  
Information System ◽  
Information Systems ◽  
Innovation Process ◽  
System Innovation ◽  
System Implementation ◽  
Enterprise Information ◽  
Enterprise Information Systems ◽  
Is Implementation ◽  
Systems Implementation ◽  
Information Systems Implementation

Enterprise Information systems implementation is one of the most challenging parts of IT strategy for an organization, since implementation brings in efficiency in the system and justifies the investments made. Therefore, it becomes increasingly important to study the perspectives of implementation to understand the current dynamics. The purpose of this paper is twofold, first is to explore the type of literature that exists in information system implementation and secondly to determine the research methodologies incorporated therein for the information system field's implementation in specific. The basic content analysis is done to review the articles on information system implementation. A total of 47 articles were selected from peer reviewed journals and conferences. The study was conducted to assess the methodology used, the strategies followed along with the issues and challenges faced in the implementation. It presents an arena of the studies done in information system implementation in past 20 years (typically 1993 to 2013). The IMPLEMENT framework has been proposed to synthesize the literature finding for smooth functioning of IS implementation process. The factors influencing the adoption of information system innovation are described. The comprehensive framework for information innovation process is developed. This framework is then mapped to IMPLEMENT framework. This study would encourage the practitioners in the information systems domain to improve upon their organizational capability and incorporate other best practices.

Information Systems for Organizational Effectiveness Model

2012 ◽  
pp. 39-61
Author(s):  
Govindan Marthandan ◽  
Chun Meng Tang
Keyword(s):  
Information Systems ◽  
Organizational Effectiveness ◽  
Evaluation Model ◽  
Business Value ◽  
Business Managers ◽  
Vast Number ◽  
Organisational Effectiveness ◽  
Effectiveness Model ◽  
Evaluation Models ◽  
Is Evaluation

Despite the proposal of various Information Systems (IS) evaluation models and approaches, IS evaluation has never been straightforward. There are issues and challenges in proving the business value of IS. Adding to the difficulty, a vast number of measures have been employed conveniently for evaluation purposes without going through a rigorous validation process. Recognising the complexity for IS researchers, IS specialists, and business managers to agree on a common model for the evaluation of IS business value, this chapter presents an empirically validated IS evaluation model, the IS for organisational effectiveness (ISOE) model, for planning, designing, implementing, and appraising IS. There also emerges a new theory, the Information System business value (ISBV) theory, from the ISOE model to establish that IS business value is multifaceted and are observable in the form of improvements in organisational effectiveness.

Using Technology in Providing Effective Training

2012 ◽  
pp. 138-153
Author(s):  
Ernest W. Brewer ◽  
Stephen D. Stockton ◽  
Tammi M. Basile
Keyword(s):  
Formative Evaluation ◽  
Training Programs ◽  
Clear Understanding ◽  
Training Methods ◽  
Implementation Phase ◽  
Evaluation Phase ◽  
Effective Training ◽  
Individualized Training ◽  
Learning With Technology ◽  
Goals And Objectives

The key to an organization’s abilities to provide effective training in the 21st century is by infusing technology into their training programs. Before technology infusion can occur, there must be a clear understanding of the dynamics of learning with technology and of the instructional culture of the organization. Developers of effective training programs infuse technology based on four phases—development, integration, implementation, and evaluation. The development phase involves the analysis of the organization, the creation of training goals and objectives, and the design of the training program. The integration phase determines the organization’s technology levels and allows developers and trainers to expedite training methods suitable for particular needs. The implementation phase consists of the actual delivery of the training. The evaluation phase is ongoing through formative evaluation, and it ends with a summative evaluation. Future trends of technology in training show promising models for effective individualized training in virtual environments.

Information Systems Research Relevance

2011 ◽  
pp. 2004-2009 ◽  
Author(s):  
Shirish C. Srivastava ◽  
Thompson S.H. Teo
Keyword(s):  
Information Technology ◽  
Information Systems ◽  
Empirical Study ◽  
Information Management ◽  
Management Information Systems ◽  
Is Research ◽  
Systems Research ◽  
Information Systems Research ◽  
Key Issues ◽  
It Executives

Though there have been extended deliberations for making information systems (IS) research more relevant1 and useful for IS executives, to our knowledge, there has been no empirical study which examines the extent of relevance in the current IS research. In this chapter, we analyze the topical relevance of 388 published academic articles in the three top IS journals: MIS Quarterly (MISQ), Information Systems Research (ISR), and Journal of Management Information Systems (JMIS), for a 5 year period from 2000-2004. We do this by examining their fit with the key issues for information technology (IT) executives identified by the latest Society for Information Management (SIM) survey. Based on our results, we make recommendations for making IS research more meaningful for practitioners.

Sign in / Sign up

Export Citation Format

Share Document