scholarly journals How Reliable and Valid are the Evaluations of Digital Competence in Higher Education: A Systematic Mapping Study

SAGE Open ◽  
2022 ◽  
Vol 12 (1) ◽  
pp. 215824402110684
Author(s):  
Rafael Saltos-Rivas ◽  
Pavel Novoa-Hernández ◽  
Rocío Serrano Rodríguez
Keyword(s):  
Ad Hoc ◽  
Reliability And Validity ◽  
Reporting Quality ◽  
Systematic Mapping Study ◽  
Validity And Reliability ◽  
Digital Competence ◽  
Systematic Mapping ◽  
Specific Dimension ◽  
Engineering And Technology ◽  
Quality Assessments

Evaluating digital competencies has become a topic of growing interest in recent years. Although several reviews and studies have summarized the main elements of progress and shortcomings in this area, some issues are yet to be explored. Very little information is available about the ways of ensuring the validity and reliability of the instrument used. This study addresses this issue through systematic mapping covering the period from January 2015 to July 2020. Based on 88 primary studies, we conclude that a growing number of studies have emerged over the years; most of them are based on European university students in social science programs; the quality of the journals where they were published is low; and the instruments used are mostly questionnaires and ad-hoc surveys that measure the various dimensions of digital competence based on participants’ perceptions. Out of the eight possible types of quality assessment, studies commonly report only four (one in the case of reliability and three in the case of validity). More than 50% of the studies do not provide evidence of having assessed the reliability and validity at the same time. Although participant discipline resulted in a significantly association with the practice of reporting reliability and validity assessments, no specific dimension explained such association. A non-parametric multivariate analysis reveals, among other interesting patterns, that the practice of not reporting quality assessments is more present in studies where participants belong to the fields of Engineering and Technology, Health Sciences, and Humanities.

scholarly journals On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0257344
Author(s):  
Rafael Saltos-Rivas ◽  
Pavel Novoa-Hernández ◽  
Rocío Serrano Rodríguez
Keyword(s):  
Higher Education ◽  
Reliability And Validity ◽  
Systematic Mapping Study ◽  
Mapping Study ◽  
Digital Competence ◽  
Systematic Mapping ◽  
Reporting Practices ◽  
Report Quality ◽  
Quality Assessments

In this study, we report on a Systematic Mapping Study (SMS) on how the quality of the quantitative instruments used to measure digital competencies in higher education is assured. 73 primary studies were selected from the published literature in the last 10 years in order to 1) characterize the literature, 2) evaluate the reporting practice of quality assessments, and 3) analyze which variables explain such reporting practices. The results indicate that most of the studies focused on medium to large samples of European university students, who attended social science programs. Ad hoc, self-reported questionnaires measuring various digital competence areas were the most commonly used method for data collection. The studies were mostly published in low tier journals. 36% of the studies did not report any quality assessment, while less than 50% covered both groups of reliability and validity assessments at the same time. In general, the studies had a moderate to high depth of evidence on the assessments performed. We found that studies in which several areas of digital competence were measured were more likely to report quality assessments. In addition, we estimate that the probability of finding studies with acceptable or good reporting practices increases over time.

scholarly journals Digital Competence Assessment Methods in Higher Education: A Systematic Literature Review

2021 ◽  
Vol 11 (8) ◽  
pp. 402
Author(s):  
Linda Helene Sillat ◽  
Kairit Tammets ◽  
Mart Laanpere
Keyword(s):  
Higher Education ◽  
Literature Review ◽  
Systematic Literature Review ◽  
Reliability And Validity ◽  
Assessment Tools ◽  
Competence Assessment ◽  
Validity And Reliability ◽  
Digital Competence ◽  
Need To Evaluate ◽  
Assessment In Higher Education

The rapid increase in recent years in the number of different digital competency frameworks, models, and strategies has prompted an increasing popularity for making the argument in favor of the need to evaluate and assess digital competence. To support the process of digital competence assessment, it is consequently necessary to understand the different approaches and methods. This paper carries out a systematic literature review and includes an analysis of the existing proposals and conceptions of digital competence assessment processes and methods in higher education, with the aim of better understanding the field of research. The review follows three objectives: (i) describe the characteristics of digital competence assessment processes and methods in higher education; (ii) provide an overview of current trends; and, finally, (iii) identify challenges and issues in digital competence assessment in higher education with a focus on the reliability and validity of the proposed methods. On the basis of the findings, and as a result of the COVID-19 pandemic, digital competence assessment in higher education requires more attention, with a specific focus on instrument validity and reliability. Furthermore, it will be of great importance to further investigate the use of assessment tools to support systematic digital competence assessment processes. The analysis includes possible opportunities and ideas for future lines of work in digital competence evaluation in higher education.

scholarly journals Design, Validation and Implementation of a Questionnaire to Assess Teenagers’ Digital Competence in the Area of Communication in Digital Environments

2021 ◽  
Vol 13 (12) ◽  
pp. 6733
Author(s):  
Ana Iglesias-Rodríguez ◽  
Azucena Hernández-Martín ◽  
Yolanda Martín-González ◽  
Patricia Herráez-Corredera
Keyword(s):  
Citizen Participation ◽  
Ad Hoc ◽  
Identity Management ◽  
New Technologies ◽  
Compulsory Education ◽  
Reliability And Validity ◽  
Psychometric Validation ◽  
Digital Competence ◽  
Education Students ◽  
Design Validation

This article describes the process of design, validation, and implementation (N = 609) of a questionnaire drawn up ad hoc to assess the digital competence of compulsory education students (ages 11 to 13) in the area of communication. The test measures students’ knowledge, skills, and attitudes in the six competences that make up the area of communication, as established in the Framework for the Development and Knowledge of Digital Competence in Europe (DigComp): interacting through new technologies, sharing of information and content, online citizen participation, collaboration through digital technologies, netiquette, and digital identity management. The purposes of the study are to design and validate an instrument to assess compulsory education students’ digital competences in the area of communication based on their knowledge, skills, and attitudes and to analyse such instrument’s psychometric characteristics with special emphasis on its reliability and validity. The method used consisted of the implementation of various psychometric validation techniques and the analysis of the results based on statistical descriptions. Items show adequate discrimination and difficulty indices. Validity was guaranteed through expert judgement and factorial analysis of the test. The conclusion stresses the pressing need for education centres to provide students with adequate educational-communicative training.

Using Principal Component Scores to Enhance the Validity and Reliability of Big Five Personality Measures

2017 ◽  
Vol 38 (2) ◽  
pp. 83-93
Author(s):  
Jeffrey M. Cucina ◽  
Nicholas L. Vasilopoulos ◽  
Arwen H. DeCostanza
Keyword(s):  
Big Five ◽  
Discriminant Validity ◽  
Reliability And Validity ◽  
Principal Component ◽  
Big Five Personality ◽  
Validity And Reliability ◽  
Scoring Method ◽  
Five Factors ◽  
Big Five Factors ◽  
Component Scores

Abstract. Varimax rotated principal component scores (VRPCS) have previously been offered as a possible solution to the non-orthogonality of scores for the Big Five factors. However, few researchers have examined the reliability and validity of VRPCS. To address this gap, we use a lab study and a field study to investigate whether using VRPCS increase orthogonality, reliability, and criterion-related validity. Compared to the traditional unit-weighting scoring method, the use of VRPCS enhanced the reliability and discriminant validity of the Big Five factors, although there was little improvement in criterion-related validity. Results are discussed in terms of the benefit of using VRPCS instead of traditional unit-weighted sum scores.

From ICT skills to Digital competence for academic needs in Trakia University – Stara Zagora

2018 ◽  
pp. 132-137 ◽  
Author(s):  
Zlatoeli Ducheva ◽  
Veselina Nedeva
Keyword(s):  
Higher Education ◽  
Labour Market ◽  
Employment Status ◽  
Basic Skill ◽  
Research Model ◽  
Academic Needs ◽  
Digital Competence ◽  
Ict Skills ◽  
Engineering And Technology ◽  
Digital Competencies

From the beginning of the 21st century, digital competencies are perceived as a "requirement and right," as a "life/basic skill". The purpose of this article is to justify the creation of a blitz-survey, designed and conducted to determine the level of digital competence of students. The completed research will try to answer the question of how training in Faculty of Engineering and Technology develops the digital competence of students - future engineers, which factors influence the development and attitudes to improve this type of competence. The spectrum of components in the digital competencies is defined when developing the conceptual model of the study. The research model also reflects European documents in this area, the needs, and requirements of the labour market related to the training of engineers and the new approaches and paradigms in higher education. The questions were provisionally divided into seven sections, which also have connecting links. At this stage, the study was carried out with 280 students. The end goal of the scientific research is to suggest strategies for adapting the training of the students to the European requirements and the needs of the labour market in order to improve their employment status, their adaptiveness, and their professional development.

The Challenge of Evaluating Virtual Communities of Practice: A Systematic Mapping Study

10.28945/4505 ◽  
2020 ◽  
Vol 15 ◽  
pp. 039-064
Author(s):  
Rogerio Ferreira da Silva ◽  
Itana Maria de Souza Gimenes ◽  
José Carlos Maldonado
Keyword(s):  
Communities Of Practice ◽  
Virtual Communities ◽  
Evaluation Process ◽  
Evaluation Methods ◽  
Systematic Mapping Study ◽  
Research Gaps ◽  
Systematic Mapping ◽  
Research Questions ◽  
Virtual Communities Of Practice ◽  
Selection Of

Aim/Purpose: This paper presents a study of Virtual Communities of Practice (VCoP) evaluation methods that aims to identify their current status and impact on knowledge sharing. The purposes of the study are as follows: (i) to identify trends and research gaps in VCoP evaluation methods; and, (ii) to assist researchers to position new research activities in this domain. Background: VCoP have become a popular knowledge sharing mechanism for both individuals and organizations. Their evaluation process is complex; however, it is recognized as an essential means to provide evidences of community effectiveness. Moreover, VCoP have introduced additional features to face to face Communities of Practice (CoP) that need to be taken into account in evaluation processes, such as geographical dispersion. The fact that VCoP rely on Information and Communication Technologies (ICT) to execute their practices as well as storing artifacts virtually makes more consistent data analysis possible; thus, the evaluation process can apply automatic data gathering and analysis. Methodology: A systematic mapping study, based on five research questions, was carried out in order to analyze existing studies about VCoP evaluation methods and frameworks. The mapping included searching five research databases resulting in the selection of 1,417 papers over which a formal analysis process was applied. This process led to the preliminary selection of 39 primary studies for complete reading. After reading them, we select 28 relevant primary studies from which data was extracted and synthesized to answer the proposed research questions. Contribution: The authors of the primary studies analyzed along this systematic mapping propose a set of methods and strategies for evaluating VCoP, such as frameworks, processes and maturity models. Our main contribution is the identification of some research gaps present in the body of studies, in order to stimulate projects that can improve VCoP evaluation methods and support its important role in social learning. Findings: The systematic mapping led to the conclusion that most of the approaches for VCoP evaluation do not consider the combination of data structured and unstructured metrics. In addition, there is a lack of guidelines to support community operators’ actions based on evaluation metrics.

scholarly journals Test Smell Detection Tools: A Systematic Mapping Study

2021 ◽  
Author(s):  
Wajdi Aljedaani ◽  
Anthony Peruma ◽  
Ahmed Aljohani ◽  
Mazen Alotaibi ◽  
Mohamed Wiem Mkaouer ◽  
...  
Keyword(s):  
Systematic Mapping Study ◽  
Mapping Study ◽  
Systematic Mapping

scholarly journals Global and Latin American female participation in evidence-based software engineering: a systematic mapping study

2021 ◽  
Vol 27 (1) ◽  
Author(s):  
Katia Romero Felizardo ◽  
Amanda Möhring Ramos ◽  
Claudia de O. Melo ◽  
Érica Ferreira de Souza ◽  
Nandamudi L. Vijaykumar ◽  
...  
Keyword(s):  
Software Engineering ◽  
Latin American ◽  
Gender Issue ◽  
Research Area ◽  
Systematic Mapping Study ◽  
Evidence Based ◽  
Mapping Study ◽  
Systematic Mapping ◽  
Participation Of Women ◽  
New Generation

Abstract Context While the digital economy requires a new generation of technology for scientists and practitioners, the software engineering (SE) field faces a gender crisis. SE research is a global enterprise that requires the participation of both genders for the advancement of science and evidence-based practice. However, women across the world tend to be significantly underrepresented in such research, receiving less funding and less participation, frequently, than men as authors in research publications. Data about this phenomenon is still sparse and incomplete; particularly in evidence-based software engineering (EBSE), there are no studies that analyze the participation of women in this research area. Objective The objective of this work is to present the results of a systematic mapping study (SM) conducted to collect and evaluate evidence on female researchers who have contributed to the area of EBSE. Method Our SM was performed by manually searching studies in the major conferences and journals of EBSE. We identified 981 studies and 183 were authored/co-authored by women and, therefore, included. Results Contributions from women in secondary studies have globally increased over the years, but it is still concentrated in European countries. Additionally, collaboration among research groups is still fragile, based on a few women as a bridge. Latin American researchers contribute a great deal to the field, despite they do not collaborate as much within their region. Conclusions The findings from this study are expected to be aggregated to the existing knowledge with respect to women’s contribution to the EBSE area. We expect that our results bring up a reflection on the gender issue and motivate actions and policies to attract female researchers to this area.

Sign in / Sign up

Export Citation Format

Share Document