Moving beyond Country Rankings in International Assessments: The Case of PISA

2015 ◽  
Vol 117 (1) ◽  
pp. 1-10
Author(s):  
Nancy Perry ◽  
Kadriye Ercikan

The Programme for International Student Assessment (PISA) was designed by the Organisation for Economic Cooperation and Development (OECD) to evaluate the quality, equity, and efficiency of school systems around the world. Specifically, the PISA has assessed 15-year-old students’ reading, mathematics, and science literacy on a 3-year cycle, since 2000. Also, the PISA collects information about how those outcomes are related to key demographic, social, economic, and educational variables. However, the preponderance of reports involving PISA data focus on achievement variables and cross-national comparisons of achievement variables. Challenges in evaluating achievement of students from different cultural and educational settings and data concerning students’ approaches to learning, motivation for learning, and opportunities for learning are rarely reported. A main goal of this themed issue of Teachers College Record (TCR) is to move the conversation about PISA data beyond achievement to also include factors that affect achievement (e.g., SES, home environment, strategy use). Also we asked authors to consider how international assessment data can be used for improving learning and education and what appropriate versus inappropriate inferences can be made from the data. In this introduction, we synthesize the six articles in this issue and themes that cut across them. Also we examine challenges associated with using data from international assessments, like the PISA, to inform education policy and practice within and across countries. We conclude with recommendations for collecting and using data from international assessments to inform research, policy, and teaching and learning.

2015 ◽  
Vol 117 (1) ◽  
pp. 1-10 ◽  
Author(s):  
Nancy Perry ◽  
Kadriye Ercikan

The Programme for International Student Assessment (PISA) was designed by the Organisation for Economic Cooperation and Development (OECD) to evaluate the quality, equity, and efficiency of school systems around the world. Specifically, the PISA has assessed 15-year-old students’ reading, mathematics, and science literacy on a 3-year cycle, since 2000. Also, the PISA collects information about how those outcomes are related to key demographic, social, economic, and educational variables. However, the preponderance of reports involving PISA data focus on achievement variables and cross-national comparisons of achievement variables. Challenges in evaluating achievement of students from different cultural and educational settings and data concerning students’ approaches to learning, motivation for learning, and opportunities for learning are rarely reported. A main goal of this themed issue of Teachers College Record (TCR) is to move the conversation about PISA data beyond achievement to also include factors that affect achievement (e.g., SES, home environment, strategy use). Also we asked authors to consider how international assessment data can be used for improving learning and education and what appropriate versus inappropriate inferences can be made from the data. In this introduction, we synthesize the six articles in this issue and themes that cut across them. Also we examine challenges associated with using data from international assessments, like the PISA, to inform education policy and practice within and across countries. We conclude with recommendations for collecting and using data from international assessments to inform research, policy, and teaching and learning.


2021 ◽  
Author(s):  
Kylie Hillman ◽  
Sue Thomson

Australia was one of nine countries and economies to participate in the 2018 TALIS-PISA link study, together with Cuidad Autónoma de Buenos Aires (Argentina), Colombia, the Czech Republic, Denmark, Georgia, Malta, Turkey and Viet Nam. This study involved coordinating the samples of schools that participated in the Program of International Student Assessment (PISA, a study of the performance of 15-year-old students) and the Teaching and Learning International Survey (TALIS, a study that surveys teachers and principals in lower secondary schools) in 2018. A sample of teachers from schools that were selected to participate in PISA were invited to respond to the TALIS survey. TALIS data provides information regarding the background, beliefs and practices of lower secondary teachers and principals, and PISA data delivers insights into the background characteristics and cognitive and non-cognitive skills of 15-year-old students. Linking these data offers an internationally comparable dataset combining information on key education stakeholders. This report presents results of analyses of the relationships between teacher and school factors and student outcomes, such as performance on the PISA assessment, expectations for further study and experiences of school life. Results for Australia are presented alongside those of the average (mean) across all countries and economies that participated in the TALIS-PISA link study for comparison, but the focus remains on what relationships were significant among Australian students.


2021 ◽  
Author(s):  
Kylie Hillman ◽  
Sue Thomson

Australia was one of nine countries and economies to participate in the 2018 TALIS-PISA link study, together with Cuidad Autónoma de Buenos Aires (Argentina), Colombia, the Czech Republic, Denmark, Georgia, Malta, Turkey and Viet Nam. This study involved coordinating the samples of schools that participated in the Program of International Student Assessment (PISA, a study of the performance of 15-year-old students) and the Teaching and Learning International Survey (TALIS, a study that surveys teachers and principals in lower secondary schools) in 2018. A sample of teachers from schools that were selected to participate in PISA were invited to respond to the TALIS survey. TALIS data provides information regarding the background, beliefs and practices of lower secondary teachers and principals, and PISA data delivers insights into the background characteristics and cognitive and non-cognitive skills of 15-year-old students. Linking these data offers an internationally comparable dataset combining information on key education stakeholders. This report presents results of analyses of the relationships between teacher and school factors and student outcomes, such as performance on the PISA assessment, expectations for further study and experiences of school life. Results for Australia are presented alongside those of the average (mean) across all countries and economies that participated in the TALIS-PISA link study for comparison, but the focus remains on what relationships were significant among Australian students.


2021 ◽  
Vol 12 ◽  
Author(s):  
Bing Jia ◽  
Zhemin Zhu ◽  
Hang Gao

The cognitive diagnosis model is an emerging evaluation theory. The mastery of fine-grained knowledge points of students can be obtained via the cognitive diagnostic model (CDM), which can subsequently describe the learning trajectory. The latter is a description of the learning progress of students in a specific area, through which teaching and learning can be linked. This research is based on nine statistical items in the Program for International Student Assessment (PISA) 2012 and an analysis of the response data of 30,092 students from 14 countries from four attributes based on CDM. Then, it obtains the learning trajectory of students in statistical knowledge. The study found that Bulgaria, Costa Rica, Peru, Mexico, and Serbia have the same learning trajectories. The learning trajectories of almost 14 countries are as follows: (1) uncertainty, (2) data handling, (3) statistical chart, and (4) average.


Author(s):  
Björn Högberg ◽  
Solveig Petersen ◽  
Mattias Strandh ◽  
Klara Johansson

AbstractStudents’ sense of belonging at school has declined across the world in recent decades, and more so in Sweden than in almost any other high-income country. However, we do not know the characteristics or causes of these worldwide trends. Using data on Swedish students aged 15–16 years from the Programme for International Student Assessment (PISA) between 2000 and 2018, we show that the decline in school belonging in Sweden was driven by a disproportionately large decline at the bottom part of the distribution, and was greatest for foreign-born students, students from disadvantaged social backgrounds, and for low-achieving students. The decline cannot be accounted for by changes in student demographics or observable characteristics related to the school environment. The decline did, however, coincide with a major education reform, characterized by an increased use of summative evaluation, and an overall stronger performance-orientation.


2018 ◽  
Vol 26 (2) ◽  
pp. 213-226 ◽  
Author(s):  
Jörg Blasius

Purpose Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task by fabricating entire interviews via copy and paste. Design/methodology/approach Using data from the principal questionnaires in the Programme for International Student Assessment (PISA) 2012 and the Programme for the International Assessment of Adult Competencies (PIAAC) data, the author applies statistical methods to search for fraudulent methods used by interviewers and employees at survey research organizations. Findings The author provides empirical evidence for potential fraud performed by interviewers and employees of survey research organizations in several countries that participated in PISA 2012 and PIAAC. Practical implications The proposed methods can be used as early as the initial phase of fieldwork to flag potentially problematic interviewer behavior such as copying responses. Originality/value The proposed methodology may help to improve data quality in survey research by detecting fabricated data.


Author(s):  
S. Marshall Perry ◽  
Karen M. Sealy ◽  
Héctor X. Ramírez-Pérez ◽  
Thomas C. DeNicola ◽  
Yair Cohen

Connections between principal leadership activities, school context, and student achievement are examined within this paper. Data for this quantitative study are from the 2013 Teaching and Learning International Survey (TALIS) and the 2012 Programme for International Student Assessment (PISA). The eight countries of examination participated in both the TALIS and PISA and the researchers merged datasets, yielding a study sample of 1,301 schools. This paper supports a context-specific view of instructional leadership. When looking across countries, the researchers found different practices were more strongly associated with the academic achievement of students, and suggest that school leaders have a meaningful overall relationship with academic achievement, both directly and indirectly. This study therefore supports prior research about the direct and indirect effects of instructional leadership. Further study, which accounts for differences in family academic resources and school-level opportunities to learn, will better illuminate the connection between instructional leadership practices and academic achievement.


2019 ◽  
Vol 44 (6) ◽  
pp. 752-781
Author(s):  
Michael O. Martin ◽  
Ina V.S. Mullis

International large-scale assessments of student achievement such as International Association for the Evaluation of Educational Achievement’s Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study and Organization for Economic Cooperation and Development’s Program for International Student Assessment that have come to prominence over the past 25 years owe a great deal in methodological terms to pioneering work by National Assessment of Educational Progress (NAEP). Using TIMSS as an example, this article describes how a number of core techniques, such as matrix sampling, student population sampling, item response theory scaling with population modeling, and resampling methods for variance estimation, have been adapted and implemented in an international context and are fundamental to the international assessment effort. In addition to the methodological contributions of NAEP, this article illustrates how the large-scale international assessments go beyond measuring student achievement by representing important aspects of community, home, school, and classroom contexts in ways that can be used to address issues of importance to researchers and policymakers.


2019 ◽  
Vol 20 (1) ◽  
pp. 45-65
Author(s):  
Sam P.E. Hopp

The Programme for International Student Assessment (PISA) scores are a leading international measure of achievement. This study reviews German 2015 PISA data and imputes scores on income and time in nation to provide comparisons between native, immigrant and refugee students. This quantitative study uses cultural capital to explain the association of independent variables to PISA scores for students, revealing an unexpected negative linear relationship between those variables. The results and significance of this study may assist those involved in policy for refugee populations and inform the strategies of test protocols and measures in a new global student paradigm.


Author(s):  
S. Marshall Perry ◽  
Karen M. Sealy ◽  
Héctor X. Ramírez-Pérez ◽  
Thomas C. DeNicola ◽  
Yair Cohen

Connections between principal leadership activities, school context, and student achievement are examined within this paper. Data for this quantitative study are from the 2013 Teaching and Learning International Survey (TALIS) and the 2012 Programme for International Student Assessment (PISA). The eight countries of examination participated in both the TALIS and PISA and the researchers merged datasets, yielding a study sample of 1,301 schools. This paper supports a context-specific view of instructional leadership. When looking across countries, the researchers found different practices were more strongly associated with the academic achievement of students, and suggest that school leaders have a meaningful overall relationship with academic achievement, both directly and indirectly. This study therefore supports prior research about the direct and indirect effects of instructional leadership. Further study, which accounts for differences in family academic resources and school-level opportunities to learn, will better illuminate the connection between instructional leadership practices and academic achievement.


Sign in / Sign up

Export Citation Format

Share Document