Information and Computer Technologies for Improving International Assessment

Large Scale ◽

International Student ◽

Student Assessment ◽

Multiple Test ◽

Computer Based ◽

Paper And Pencil ◽

Automatic Item Generation ◽

Computer And Information Literacy

International assessments such as the trends in international math and science study (TIMSS), the program for international student assessment (PISA), and the international computer and information literacy study (ICILS) have traditionally relied on paper and pencil administration. These assessments are rapidly transforming into or have been developed as computer-based tests due to advances in information and communication technologies of the past decade. These computer-based assessments will eventually make traditional paper and pencil assessments obsolete. Specifically, international and other large-scale assessments can benefit from the use of automatic item generation (AIG) and/or computer adaptive testing (CAT) to enhance and strengthen test security and validity, as well as reduce costs over the course of multiple test administrations, encourage student engagement, and efficiently measure students' abilities.

Understanding and detecting data fabrication in large-scale assessments

Quality Assurance in Education ◽

10.1108/qae-07-2017-0038 ◽

2018 ◽

Vol 26 (2) ◽

pp. 196-212 ◽

Cited By ~ 5

Author(s):

Kentaro Yamamoto ◽

Mary Louise Lennon

Keyword(s):

Large Scale ◽

International Student ◽

Student Assessment ◽

Response Patterns ◽

Content Type ◽

Large Scale Assessment ◽

Data Fabrication ◽

Computer Based ◽

Types Of Information

Purpose Fabricated data jeopardize the reliability of large-scale population surveys and reduce the comparability of such efforts by destroying the linkage between data and measurement constructs. Such data result in the loss of comparability across participating countries and, in the case of cyclical surveys, between past and present surveys. This paper aims to describe how data fabrication can be understood in the context of the complex processes involved in the collection, handling, submission and analysis of large-scale assessment data. The actors involved in those processes, and their possible motivations for data fabrication, are also elaborated. Design/methodology/approach Computer-based assessments produce new types of information that enable us to detect the possibility of data fabrication, and therefore the need for further investigation and analysis. The paper presents three examples that illustrate how data fabrication was identified and documented in the Programme for the International Assessment of Adult Competencies (PIAAC) and the Programme for International Student Assessment (PISA) and discusses the resulting remediation efforts. Findings For two countries that participated in the first round of PIAAC, the data showed a subset of interviewers who handled many more cases than others. In Case 1, the average proficiency for respondents in those interviewers’ caseloads was much higher than expected and included many duplicate response patterns. In Case 2, anomalous response patterns were identified. Case 3 presents findings based on data analyses for one PISA country, where results for human-coded responses were shown to be highly inflated compared to past results. Originality/value This paper shows how new sources of data, such as timing information collected in computer-based assessments, can be combined with other traditional sources to detect fabrication.

Gender and Information and Communication Technologies interest: results from PISA 2018

10.36253/978-88-5518-461-8.07 ◽

2021 ◽

pp. 31-36

Author(s):

Mariangela Zenga

Keyword(s):

Gender Difference ◽

International Student ◽

Student Assessment ◽

Key Driver ◽

Using Data

ICT engagement has become an important key driver regarding the competence for the adults, but even more for young people. As it was pointed out in 2015, ICT interest (a component of ICT engagement) is defined as “content-specific motivational disposition” and describes ”individuals’ long-term preference for dealing with topics, tasks, or activities related to ICT” (Goldhammer et al., 2017). Using data from 2018 Programme for International Student assessment (PISA), this work will show the gender difference in ICT interest for 15-years old students of 23 OECD countries. Moreover a three-level multilevel model will explain the effects of the characteristics of the student, of the school in which the student learns and of the country in which the student livis on the ICT interest.

Are There Test Administrator Effects in Large-Scale Educational Assessments?

Methodology ◽

10.1027/1614-2241.3.4.149 ◽

2007 ◽

Vol 3 (4) ◽

pp. 149-159 ◽

Cited By ~ 9

Author(s):

Oliver Lüdtke ◽

Alexander Robitzsch ◽

Ulrich Trautwein ◽

Frauke Kreuter ◽

Jan Marten Ihme

Keyword(s):

Large Scale ◽

International Student ◽

Student Assessment ◽

Student Groups ◽

Mathematics Scores ◽

Sample Attrition ◽

Educational Assessments ◽

Test Administrator ◽

Practical Implications

Abstract. In large-scale educational assessments such as the Third International Mathematics and Sciences Study (TIMSS) or the Program for International Student Assessment (PISA), sizeable numbers of test administrators (TAs) are needed to conduct the assessment sessions in the participating schools. TA training sessions are run and administration manuals are compiled with the aim of ensuring standardized, comparable, assessment situations in all student groups. To date, however, there has been no empirical investigation of the effectiveness of these standardizing efforts. In the present article, we probe for systematic TA effects on mathematics achievement and sample attrition in a student achievement study. Multilevel analyses for cross-classified data using Markov Chain Monte Carlo (MCMC) procedures were performed to separate the variance that can be attributed to differences between schools from the variance associated with TAs. After controlling for school effects, only a very small, nonsignificant proportion of the variance in mathematics scores and response behavior was attributable to the TAs (< 1%). We discuss practical implications of these findings for the deployment of TAs in educational assessments.

To the comparative analysis of the digitalization process of state administration in Russia and other countries

Russian Economic Journal ◽

10.33983/0130-9757-2020-5-75-117 ◽

2020 ◽

pp. 75-117

Author(s):

A.N. Shvetsov

Keyword(s):

Public Administration ◽

Large Scale ◽

Developed Countries ◽

Electronic Government ◽

Digital Government ◽

The Developed Countries ◽

Comprehensive Programs

The article compares the processes of dissemination of modern information and communication technologies in government bodies in Russia and abroad. It is stated that Russia began the transition to «electronic government» later than the developed countries, in which this process was launched within the framework of large-scale and comprehensive programs for reforming public administration in the 1980s and 1990s. However, to date, there is an alignment in the pace and content of digitalization tasks. At a new stage in this process, the concept of «electronic government» under the influence of such newest phenomena of the emerging information society as methods of analysis of «big data», «artificial intelligence», «Internet of things», «blockchain» is being transformed into the category of «digital government». Achievements and prospects of public administration digitalization are considered on the example of countries with the highest ratings — Denmark, Australia, Republic of Korea, Great Britain, USA and Russia.

Early tracking and different types of inequalities in achievement: difference-in-differences evidence from 20 years of large-scale assessments

Educational Assessment Evaluation and Accountability ◽

10.1007/s11092-020-09346-4 ◽

2021 ◽

Vol 33 (1) ◽

pp. 139-167

Author(s):

Andrés Strello ◽

Rolf Strietholt ◽

Isa Steinmann ◽

Charlotte Siepmann

Keyword(s):

Large Scale ◽

International Student ◽

Student Assessment ◽

Difference In Differences ◽

Seminal Paper ◽

Units Of Analysis ◽

Social Achievement ◽

Mathematics And Science ◽

Combined Data

AbstractResearch to date on the effects of between-school tracking on inequalities in achievement and on performance has been inconclusive. A possible explanation is that different studies used different data, focused on different domains, and employed different measures of inequality. To address this issue, we used all accumulated data collected in the three largest international assessments—PISA (Programme for International Student Assessment), PIRLS (Progress in International Reading Literacy Study), and TIMSS (Trends in International Mathematics and Science Study)—in the past 20 years in 75 countries and regions. Following the seminal paper by Hanushek and Wößmann (2006), we combined data from a total of 21 cycles of primary and secondary school assessments to estimate difference-in-differences models for different outcome measures. We synthesized the effects using a meta-analytical approach and found strong evidence that tracking increased social achievement gaps, that it had smaller but still significant effects on dispersion inequalities, and that it had rather weak effects on educational inadequacies. In contrast, we did not find evidence that tracking increased performance levels. Besides these substantive findings, our study illustrated that the effect estimates varied considerably across the datasets used because the low number of countries as the units of analysis was a natural limitation. This finding casts doubt on the reproducibility of findings based on single international datasets and suggests that researchers should use different data sources to replicate analyses.

Neotecnicismo - a Retomada do Tecnicismo em Novas Bases

Revista de Ensino Educação e Ciências Humanas ◽

10.17921/2447-8733.2018v19n1p10-16 ◽

2018 ◽

Vol 19 (1) ◽

pp. 10

Author(s):

Andréa Villela Mafra da Silva

Keyword(s):

Large Scale ◽

Quality Criteria ◽

Educational Process ◽

Educational Policies ◽

Present Moment ◽

Total Quality ◽

Use Of Technology ◽

Technical Rationality

Trata-se de um artigo de revisão, que busca estudar o contexto neotecnicista mais recente, apresentado agora sob a forma de um processo de ensino e aprendizagem centrado nos resultados, em que se propõe a mesma racionalidade técnica dos anos setenta, para assim garantir a efciência e a produtividade na educação. Fazendo um resgate da história da educação brasileira, já na década de setenta, a confguração do discurso pedagógico estava atrelada aos princípios de racionalidade, efciência e produtividade. A pedagogia tecnicista estruturava o processo educativo em uma perspectiva operacional. Reportando-se ao momento atual, ao se examinar as políticas educacionais se encontra a ideia do neotecnicismo atrelada às avaliações de larga escala, com base nos conceitos de efciência, de produtividade e de qualidade total. Nesse contexto, a estratégia parece ser a incorporação das tecnologias na educação no primado da dimensão técnica. Em outras palavras, o neotecnicismo pedagógico se faz presente nas atuais políticas educacionais, que enfatizam o critério da qualidade com base na utilização das tecnologias como estratégia de adequação da educação escolar à sociedade da informação. A pesquisa é bibliográfca com aportes em artigos científcos e documentos ofciais. O estudo conclui que o paradigma neotecnicista poderá trazer novas formas de racionalização do sistema educativo, especialmente, através de concepções educacionais ancoradas no discurso da qualidade total na educação.Palavras-chave: Políticas Educacionais. Neotecnicismo. Tecnologias de Informação e Comunicação.AbstractThis is a review article that seeks to study the latest neotechnical context, now presented in the form of a teaching and centered learning results, which proposes the same technical rationality of the seventies in order to ensure the efciency and productivity in education. Making a rescue of the history of Brazilian education, already in the seventies, the confguration of the pedagogical discourse was associated with the principles of rationality, efciency and productivity. The technicist pedagogy structured the educational process in an operational perspective. Referring to the present moment, as the educational policies are examined is the idea of linked neotechnicism to the large-scale assessments based on the concepts of efciency, productivity and overall quality. In this context, the strategy seems to be the incorporation of technology in education in the primacy of the technical dimension. In other words, the pedagogical neotechnicism is present in the current educational policies that emphasize the quality criteria based on the use of technology as a strategy of adaptation of education to the information society. The research is literature with contributions in scientifc papers and ofcial documents . The study concludes that the neotechnical paradigm can bring new ways of streamlining the education system, especially through educational conceptions anchored in total quality discourse in education.Keywords: Educational Policies. Neotechnicism . Information and Communication Technologies.

Introduction: Ethics of Information Technology in Health Care

The International Review of Information Ethics ◽

10.29173/irie188 ◽

2006 ◽

Vol 5 ◽

pp. 2-5

Author(s):

Georg Marckmann ◽

Kenneth W Goodman

Keyword(s):

Health Care ◽

Information Technology ◽

Information And Communication Technology ◽

Communication Technology ◽

Social Issues ◽

Theme Issue ◽

Computer Based

Computer-based information and communication technologies continue to transform the delivery of health care and the conception and scientific understanding of the human body and the diseases that afflict it. While information technology has the potential to improve the quality and efficiency of patient care, it also raises important ethical and social issues. This IRIE theme issue seeks to provide a forum to identify, analyse and discuss the ethical and social issues raised by various applications of information and communication technology in medicine and health care. The contributions give a flavour of the extraordinarily broad landscape shaped by the intersection of medicine, computing and ethics. In fact, their diversity suggests that much more work is needed to clarify issues and approaches, and to provide practical tools for clinicians.

Technology adoption in Malaysian schools: An analysis of national ICT in education policy initiatives

Digital Education Review ◽

10.1344/der.2020.37.172-194 ◽

2020 ◽

pp. 172-194

Author(s):

Azlin Zaiti Zainal ◽

Siti Zaidah Zainuddin

Keyword(s):

Education Policy ◽

Large Scale ◽

Evaluation Studies ◽

Virtual Learning Environment ◽

Ict In Education ◽

Educational Innovations ◽

Teachers And Students ◽

Information And Communication

Educational change, particularly change involving the adoption of educational innovations, is a complex process. In Malaysia, the significant role of information and communication technologies (ICTs) in promoting digital education is reflected in the national ICT in education policy. Nevertheless, studies in the Malaysian educational context have shown that the policy implementation resulted in varying degrees of success. Through a discourse analysis approach, this paper aims to examine the evaluation studies on large scale initiatives introduced to digitize the Malaysian education system, from the Smart Schools programme to the incorporation of the Frog Virtual Learning Environment in classroom. The rationale behind these top-down initiatives and how they affected the stakeholders at the micro level, namely, teachers and students, are reviewed and analysed. The analysis informs our understanding of the factors that contribute to the successful and unsuccessful implementation of these initiatives and guide the planning of future policies.

Handbook of Research on Enhancing Teacher Education with Advanced Instructional Technologies - Advances in Higher Education and Professional Development ◽

Potentials of Selected Information and Communication Technologies in Adult Education Programmes in Nigeria

10.4018/978-1-4666-8162-0.ch003 ◽

2015 ◽

pp. 34-56 ◽

Cited By ~ 1

Author(s):

Melvins Enwuvesi Hanachor ◽

Rex Aduvo Needom

Keyword(s):

Adult Education ◽

Staff Development ◽

Course Design ◽

E Learning ◽

Computer Based ◽

The Subject ◽

Development Course

This chapter evaluated the potentials of selected information and communication technologies in adult education programmes in Nigeria. Infrastructure and funding are among the important issues, but scepticism about the pedagogic value of information and communication technologies and staff development are probably the most challenging. Institutions are grappling with bringing use and funding of e-learning and other computer-based instructional strategies into the mainstream of their organizations, and are beginning to contemplate restructuring to take account of information and communication technologies, in terms of staffing, staff development, course design and student support. Even though studies have captured the imperatives of information and communication technologies in Nigeria's educational system, little is still known about the subject matter in relation to adult education systems in the country. Consequently, this chapter explores and provides the much-needed insight on the subject and the issues that the process raises in the context of adult education in Nigeria.

Handbook of Research on E-Assessment in Higher Education - Advances in Higher Education and Professional Development ◽

E-Assessment and Multiple-Choice Questions

10.4018/978-1-5225-5936-8.ch001 ◽

2019 ◽

pp. 1-27

Author(s):

José Azevedo ◽

Ema Patrícia Oliveira ◽

Patrícia Damas Beites

Keyword(s):

Multiple Choice ◽

Summative Assessment ◽

Assessment Process ◽

Continuous Assessment ◽

Multiple Choice Questions ◽

Mathematics Tests ◽

Computer Based

The use of information and communication technologies (ICT) in the assessment process is becoming an asset, giving rise to the so-called computer-based assessment or e-assessment. Nowadays, its use is becoming more usual in higher education institutions. Closed formats for questions, namely multiple choice, are the most commonly used. This chapter presents a literature review of the main aspects related to this topic, including the main modalities of assessment (summative assessment and continuous assessment). Issues related to multiple choice questions (MCQ) are discussed with more detail, referring to the various formats of MCQ, its advantages and limitations, with a particular focus on its use in mathematics tests. Also, some guidelines for the quality assurance of MCQ with quality are included.