scholarly journals Web-based and mixed-mode cognitive large-scale assessments in higher education: An evaluation of selection bias, measurement bias, and prediction bias

Author(s):  
Sabine Zinn ◽  
Uta Landrock ◽  
Timo Gnambs

Abstract Educational large-scale studies typically adopt highly standardized settings to collect cognitive data on large samples of respondents. Increasing costs alongside dwindling response rates in these studies necessitate exploring alternative assessment strategies such as unsupervised web-based testing. Before respective assessment modes can be implemented on a broad scale, their impact on cognitive measurements needs to be quantified. Therefore, an experimental study on N = 17,473 university students from the German National Educational Panel Study has been conducted. Respondents were randomly assigned to a supervised paper-based, a supervised computerized, and an unsupervised web-based mode to work on a test of scientific literacy. Mode-specific effects on selection bias, measurement bias, and predictive bias were examined. The results showed a higher response rate in web-based testing as compared to the supervised modes, without introducing a pronounced mode-specific selection bias. Analyses of differential test functioning showed systematically larger test scores in paper-based testing, particularly among low to medium ability respondents. Prediction bias for web-based testing was observed for one out of four criteria on study-related success factors. Overall, the results indicate that unsupervised web-based testing is not strictly equivalent to other assessment modes. However, the respective bias introduced by web-based testing was generally small. Thus, unsupervised web-based assessments seem to be a feasible option in cognitive large-scale studies in higher education.

2012 ◽  
Vol 5 (3) ◽  
pp. 1-24 ◽  
Author(s):  
Suraya Miskon ◽  
Wasana Bandara ◽  
Guy G. Gable ◽  
Erwin Fielt

Shared services is a prominent organizational arrangement for organizations, in particular for support functions. The success (or failure) of shared services is a critical concern as the move to shared services can entail large scale investment and involve fundamental organizational change. The Higher Education (HE) sector is particularly well poised to benefit from shared services as there is a need to improve organizational performance and strong potential from sharing. Through a multiple case study of shared services experiences in HE, this study identifies ten important antecedents of shared services success: (1) Understanding of shared services; (2) Organizational environment; (3) Top management support; (4) IT environment; (5) Governance; (6) Process centric view; (7) Implementation strategy; (8) Project management; (9) Change management; and (10) Communication. The study then develops a preliminary model of shared services success that addresses the interdependencies between the success factors. As the first empirical success model for shared services, it provides valuable guidance to practice and future research.


Author(s):  
Wafi Al-Karaghouli ◽  
Ahmed Al Azri ◽  
Zahran Al Salti

Transformational e-government projects and large-scale Enterprise Information System (EIS) implementation projects have one thing in common: they both overrun their time and budget due to unclear vision and unrealistic expectations. The aim of this chapter is to report on a success story of implementing e-government in the Higher Education Admission Centre (HEAC) that is beneficial in providing an insight to both categories of projects. The case is unlike many other case studies that look at project failures; it is concerned with exploring and discussing the key critical factors that facilitate the success of the projects of both categories (Brady & Maylor, 2010). The research is a qualitative approach, and the investigation uses a single case study, with data collected by means of semi-structured interviews and organisational documents from the Ministry of Higher Education in Oman. The research findings suggest that there are three paradigms with a set of factors that impact the success of projects, namely organisational paradigm, technology paradigm, and end-user paradigm.


Seminar.net ◽  
2011 ◽  
Vol 7 (2) ◽  
Author(s):  
Yngve Nordkvelle

Yngve Nordkvelle, editorIn this issue we have collected four papers from Norwegian authors. The papers have been guest edited by associate professor Hugo Nordseth of North-Trøndelag University College. The authors represent the new generation of teachers in higher education who follow up on what new technologies have to offer for teaching and learning in higher education. Gard B. Jenset presents a study which investigates attitudes among student teachers toward using electronic resources in teaching. Two groups of student teachers, were asked to assess their skills and attitudes. Jenset introduced them to an example of how open-source, Web-based data and software could be used in teaching English culture and history. The students apporved of the experiment, but Jenset concludes that teacher education contributes little to improve their skills with technology. Cecilie Asting and Anne Swanberg discuss how to manage feedback and responses to students in large scale classes. Well planned teaching and learning activities can invite students to a variety of feedback activities. The experiements described gives insights into have such initiatives can be improved and succeed.Sven Åke Bjørke discuss the need to be aware of classifications of knowledge when curricula and learning activities are designed. The author argues for the entire sector to jump directly to "state of the art" of e-learning. To pursue this, he makes a quest for investment in teacher training in higher education, rather than focusing on technology. Hege Emma Rimmereide, Barbara Blair and Jon Hoem describe a project involving the use of wiki. Wiki Storyline is a web-based Storyline using wikis, and which demands an interdisciplinary approach to second language teaching. The Storyline creates motivation for written and oral communication. The authors have used Wiki Storyline in two in-service courses and the study presents a comparative analysis of  technical solutions as well as pedagogical potentials explored in the two courses. They also employed Etherpad for collaborative writing and as an arena for reflection.Dr. Mamtuz Ahmad discusses a vital problem of many emerging economies, namely the issue of literacy for all. As he states, literacy is an instrument of stability and a vehicle for development. Pakistan is a country which faces the problem of illiteracy. It is a grim picture he draws, and he analyses the steps that has been taken by the government to meet the real needs.Literacy is an instrument of stability within and among countries and thus may prove an indispensable means of effective participation in the societies and (the) economies of today’s world. Eradication of illiteracy from the world is an important agenda of UNESCO, and one of the six goals of Dakar Framework of Action on Education for All. Illiteracy is also a major problem in Pakistan. The picture of illiteracy in Pakistan is grim, and although successive governments have announced various programmes to promote literacy the situation is still poor because of various political, social, economic and cultural obstacles. To sum up, it can be said that literacy is a skill necessary to acquire or transmit (information) to others.  It is a means not an end in itself. Keeping in view the gravity of the situation of literacy and basic education in the country, Pakistan has completed/implemented a number of actions/activities for broad-based consultations with principal actors of EFA. Furthermore, the Government of Pakistan has accomplished the preparation of provincial and national plans of action and resource mobilization for EFA planning. This paper therefore examines the efforts to decrease illiteracy in Pakistan, a signatory of the worldwide EFA movement.


Diagnostica ◽  
2020 ◽  
Vol 66 (3) ◽  
pp. 147-157
Author(s):  
Martin Senkbeil ◽  
Jan Marten Ihme

Zusammenfassung. ICT Literacy legt eine performanzbasierte Erfassung mit simulierten und interaktiven Testaufgaben nahe. Der vorliegende Beitrag untersucht, ob mit Multiple-Choice (MC)-Aufgaben ein vergleichbares Konstrukt wie mit Simulationsaufgaben erfasst wird. Hierfür wurden die Testergebnisse zweier Instrumente aus aktuellen Large-Scale-Studien gegenübergestellt, die an N = 2 075 Jugendlichen erhoben wurden: der auf MC-Aufgaben basierende ICT Literacy-Test für Klasse 9 des Nationalen Bildungspanels (National Educational Panel Study, NEPS) und der simulationsbasierte Kompetenztest der internationalen Schulleistungsstudie ICILS 2013 (International Computer and Information Literacy Study). Die Analysen unterstützen die Gültigkeit der Konstruktinterpretation des MC-basierten Tests in NEPS. Im Sinne der konvergenten Evidenz korrelieren die MC-Aufgaben substanziell mit den computer- und simulationsbasierten Aufgaben in ICILS 2013 (.68 ≤  r ≤ .90). Weiterhin ergeben sich positive und für beide Tests vergleichbar hohe Korrelationen mit ICT-bezogenen Schülermerkmalen (z. B. Selbstwirksamkeit). Weiterführende Analysen zum Zusammenhang mit allgemeinen kognitiven Fähigkeiten zeigen zudem, dass ICT Literacy und kognitive Grundfähigkeiten distinkte Faktoren repräsentieren.


2013 ◽  
Author(s):  
Laura S. Hamilton ◽  
Stephen P. Klein ◽  
William Lorie

2020 ◽  
Vol 59 (04) ◽  
pp. 294-299 ◽  
Author(s):  
Lutz S. Freudenberg ◽  
Ulf Dittmer ◽  
Ken Herrmann

Abstract Introduction Preparations of health systems to accommodate large number of severely ill COVID-19 patients in March/April 2020 has a significant impact on nuclear medicine departments. Materials and Methods A web-based questionnaire was designed to differentiate the impact of the pandemic on inpatient and outpatient nuclear medicine operations and on public versus private health systems, respectively. Questions were addressing the following issues: impact on nuclear medicine diagnostics and therapy, use of recommendations, personal protective equipment, and organizational adaptations. The survey was available for 6 days and closed on April 20, 2020. Results 113 complete responses were recorded. Nearly all participants (97 %) report a decline of nuclear medicine diagnostic procedures. The mean reduction in the last three weeks for PET/CT, scintigraphies of bone, myocardium, lung thyroid, sentinel lymph-node are –14.4 %, –47.2 %, –47.5 %, –40.7 %, –58.4 %, and –25.2 % respectively. Furthermore, 76 % of the participants report a reduction in therapies especially for benign thyroid disease (-41.8 %) and radiosynoviorthesis (–53.8 %) while tumor therapies remained mainly stable. 48 % of the participants report a shortage of personal protective equipment. Conclusions Nuclear medicine services are notably reduced 3 weeks after the SARS-CoV-2 pandemic reached Germany, Austria and Switzerland on a large scale. We must be aware that the current crisis will also have a significant economic impact on the healthcare system. As the survey cannot adapt to daily dynamic changes in priorities, it serves as a first snapshot requiring follow-up studies and comparisons with other countries and regions.


2020 ◽  
Vol 1 (2) ◽  
pp. 18-33
Author(s):  
Zarina Che Imbi ◽  
Tse-Kian Neo ◽  
Mai Neo

In the era of digital learning, multimedia-based classroom has been commonly used in higher education including Malaysian higher education institutions. A case study has been performed to evaluate web-based learning using Level 1 to 3 of Kirkpatrick's model in a multi-disciplinary course at Multimedia University, Malaysia. In this study, mixed method research was employed in which triangulation was performed from multiple sources of data collection to give deeper understanding. Students perceived that learning with multimedia was enjoyable. They were also motivated in learning and engaged through the use of web module as multimedia was perceived to motivate them and make learning fun. Students showed significant improvements in their knowledge based on the pre-test and post-test results on learning evaluation. Students were perceived to transfer the learning from web-based learning into the learning outcome. The systematic evaluation can provide the feedback that educators and institution as a whole need to improve the learning environment and programme quality. This study contributes to the research field by adding another perspective in evaluations of web-based learning. It also provides empirical evidence on student perspectives, learning and behaviour in a private university. It demonstrated that the Kirkpatrick's model is useful as an evaluation tool to be used in higher education.


10.31355/42 ◽  
2019 ◽  
Vol 3 ◽  
pp. 065-077
Author(s):  
Agyei Fosu

NOTE: THIS ARTICLE WAS PUBLISHED WITH THE INFORMING SCIENCE INSTITUTE. Aim/Purpose...................................................................................................................................................................................................... The purpose of this study is to expand the knowledge base on factors likely to impede implementation and adoption of web-based learning management systems to blend with traditional methods of lecturing in universities to cater for the next generation of learners in Africa and Eastern Cape Province South Africa in particular. Background........................................................................................................................................................................................................ The shift from the industrial economies to 21st century digital and knowledge-based economies, fueled by rapid Information and Communication Technologies (ICTs) such as Internet, YouTube, Chartrooms, Skype, Social media networks and its introduction to the educational system not only resulted in a new teaching approach globally but also paved way to usher in new generation of learners (anytime, anywhere learners) in the higher education system. Despite the fact that universities and other institutions of higher education in developed countries and some Africa countries have since recognized that the 21st century global digital and knowledge-based economies evolution has ushered in the next generation of learners, and as a result have taken the necessary steps to blend the traditional method of lecturing in higher education with web-based learning management systems in order to accommodate these learners. However, in Africa not much research have been done on the readiness of higher education institutions in terms of blending web-based learning management systems with the traditional method of lecturing to cater for the next generation of learners. Methodology....................................................................................................................................................................................................... Quantitative and two non-probability sampling methods, namely, quota and purposive sampling was used to investigate the technological skills of selected lecturers from universities within Buffalo City Metropolitan as one of the core component to check the readiness of their faculty for the next generation of learners. Contribution........................................................................................................................................................................................................ This research will add to the growing knowledge about the blending of web-based learning management with the traditional style of lecturing in higher education in the 21st century digital economies. Findings.............................................................................................................................................................................................................. The results indicated that the participating lecturers need to be trained and sup-ported in the skills of using of the ICTs and computer programs applicable to enhance web-based learning in teaching and learning environment in higher education in order to cater for the next generation of learners associated with the 21st century digital economies. Recommendations for Practitioners................................................................................................................................................................. Much as there is a need for increased in investment in infrastructure within higher education institutions to support teaching and learning, continuous sup-port and training for academics to be technologically literate and also be abreast on rapidly evolving field of ICTs is paramount as it can expedite the teaching and learning process in higher education. Recommendation for Researchers................................................................................................................................................................... There is the need to explore in depth the other two components suggested by Mishra and Koehler (2007) which can serve as barriers for successfully integration of technology into teaching and learning by locus of knowledge. Impact on Society............................................................................................................................................................................................... The research will assist stakeholders, policy makers and agencies tasked with transforming institutions of higher learning to identify the barriers likely to hinder transformation efforts and address them accordingly. Future Research................................................................................................................................................................................................. Conducting research on technological skills of students are critical in this context.


Sign in / Sign up

Export Citation Format

Share Document