Fabrication of interview data

2018 ◽  
Vol 26 (2) ◽  
pp. 213-226 ◽  
Author(s):  
Jörg Blasius
Keyword(s):  
Survey Research ◽  
International Student ◽  
Student Assessment ◽  
Content Type ◽  
International Student Assessment ◽  
Pisa 2012 ◽  
Research Organizations ◽  
Using Data ◽  
Copy And Paste ◽  
Practical Implications

Purpose Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task by fabricating entire interviews via copy and paste. Design/methodology/approach Using data from the principal questionnaires in the Programme for International Student Assessment (PISA) 2012 and the Programme for the International Assessment of Adult Competencies (PIAAC) data, the author applies statistical methods to search for fraudulent methods used by interviewers and employees at survey research organizations. Findings The author provides empirical evidence for potential fraud performed by interviewers and employees of survey research organizations in several countries that participated in PISA 2012 and PIAAC. Practical implications The proposed methods can be used as early as the initial phase of fieldwork to flag potentially problematic interviewer behavior such as copying responses. Originality/value The proposed methodology may help to improve data quality in survey research by detecting fabricated data.

Are There Test Administrator Effects in Large-Scale Educational Assessments?

Methodology ◽  
2007 ◽  
Vol 3 (4) ◽  
pp. 149-159 ◽  
Author(s):  
Oliver Lüdtke ◽  
Alexander Robitzsch ◽  
Ulrich Trautwein ◽  
Frauke Kreuter ◽  
Jan Marten Ihme
Keyword(s):  
Large Scale ◽  
International Student ◽  
Student Assessment ◽  
Student Groups ◽  
Mathematics Scores ◽  
International Student Assessment ◽  
Sample Attrition ◽  
Educational Assessments ◽  
Test Administrator ◽  
Practical Implications

Abstract. In large-scale educational assessments such as the Third International Mathematics and Sciences Study (TIMSS) or the Program for International Student Assessment (PISA), sizeable numbers of test administrators (TAs) are needed to conduct the assessment sessions in the participating schools. TA training sessions are run and administration manuals are compiled with the aim of ensuring standardized, comparable, assessment situations in all student groups. To date, however, there has been no empirical investigation of the effectiveness of these standardizing efforts. In the present article, we probe for systematic TA effects on mathematics achievement and sample attrition in a student achievement study. Multilevel analyses for cross-classified data using Markov Chain Monte Carlo (MCMC) procedures were performed to separate the variance that can be attributed to differences between schools from the variance associated with TAs. After controlling for school effects, only a very small, nonsignificant proportion of the variance in mathematics scores and response behavior was attributable to the TAs (< 1%). We discuss practical implications of these findings for the deployment of TAs in educational assessments.

scholarly journals Determinants of Declining School Belonging 2000–2018: The Case of Sweden

2021 ◽  
Author(s):  
Björn Högberg ◽  
Solveig Petersen ◽  
Mattias Strandh ◽  
Klara Johansson
Keyword(s):  
Education Reform ◽  
School Environment ◽  
International Student ◽  
Student Assessment ◽  
School Belonging ◽  
Foreign Born ◽  
Bottom Part ◽  
Performance Orientation ◽  
International Student Assessment ◽  
Using Data

AbstractStudents’ sense of belonging at school has declined across the world in recent decades, and more so in Sweden than in almost any other high-income country. However, we do not know the characteristics or causes of these worldwide trends. Using data on Swedish students aged 15–16 years from the Programme for International Student Assessment (PISA) between 2000 and 2018, we show that the decline in school belonging in Sweden was driven by a disproportionately large decline at the bottom part of the distribution, and was greatest for foreign-born students, students from disadvantaged social backgrounds, and for low-achieving students. The decline cannot be accounted for by changes in student demographics or observable characteristics related to the school environment. The decline did, however, coincide with a major education reform, characterized by an increased use of summative evaluation, and an overall stronger performance-orientation.

Leading schools with migrant children in Shanghai

2017 ◽  
Vol 31 (5) ◽  
pp. 564-579
Author(s):  
Haiyan Qian ◽  
Allan David Walker
Keyword(s):  
International Student ◽  
Student Assessment ◽  
School Context ◽  
Migrant Children ◽  
Leadership Strategies ◽  
Work Lives ◽  
Content Type ◽  
International Student Assessment ◽  
Multiple Challenges ◽  
Leading Schools

Purpose The purpose of this paper is threefold: to sketch the current policy context that frames the education of migrant children in Shanghai; to explore the work lives of school leaders in the privately owned but government-supported schools; and to understand the socio-cultural and educational factors that shape the leadership practices in these schools. Design/methodology/approach This paper drew from publicly accessible policy papers and interview data with four principals leading migrant children’s schools in Shanghai. Findings Migrant children’s schools have received increasing policy recognition and attention. Principals of these schools have strived to adopt various leadership strategies to enhance the quality of education as received by migrant children. However, due to the institutional barriers such as hukou, multiple challenges continue to face migrant children and leaders leading migrant schools. Originality/value This is one of the first few papers that collected data from principals leading migrant children’s schools. The paper contributes to further understandings about leadership in high-needs school context and about education quality and equity in relation to programme for international student assessment success in Shanghai.

Mexico education figures highlight need for reform

2016 ◽  
Keyword(s):  
International Student ◽  
Student Assessment ◽  
The State ◽  
School Systems ◽  
Content Type ◽  
Knowledge And Skills ◽  
International Student Assessment ◽  
Poor State ◽  
The Poor ◽  
Equity And Efficiency

Subject The state of Mexican education. Significance The OECD released its Programme for International Student Assessment (PISA) 2015 report on December 6. The triennial study evaluates the quality, equity and efficiency of school systems, and the latest assesses to what extent 15-year-old pupils have the necessary knowledge and skills to participate in modern societies. Impacts The report will trigger mutual accusations of responsibility for the poor state of education between supporters and opponents of reform. The effectiveness of reform will see uneven implementation and outcomes in different states. Consequently, Mexico will perform poorly in the next PISA test, to be published in 2018.

scholarly journals Examining school variables affecting PISA 2012 math achievement in Turkey and Shanghai-China

2017 ◽  
Vol 6 (4) ◽  
pp. 167-174
Author(s):  
Mustafa Kale
Keyword(s):  
International Student ◽  
Math Achievement ◽  
Student Assessment ◽  
International Student Assessment ◽  
Pisa 2012 ◽  
Math Scores ◽  
Model Research ◽  
School Variables ◽  
Research Population ◽  
The Difference

The main purpose of the research is to examine school variables that have effect on Programme for International Student Assessment (PISA) 2012 math achievement in Turkey and Shanghai-China. The research was designed in casual comparison model. Research population was constituted by student in age group of 15 in Turkey and Shanghai-China in 2012.The sample consists of 4848 students and 170 schools in Turkey and 5177 students and 155 schools in  Shanghai-China that participated in PISA 2012. Two-leveled Hierarchical Linear Modelling (HLM) was used to analyze data because the data collected in PISA 2012 had a hierarchical data structure. As a result of analysis, variability in math scores, 63% in Turkey and  47% in Shanghai-China, was found due to the difference between the mean math scores of schools. It was determined that  MACTIV, SCMATEDU and TCMORALE in Turkey and MACTIV, in Shanghai-China statistically affect on math achievement. Keywords: PISA, school administration, school variables, HLM

Educational standards resist improvement in Peru

2016 ◽  
Keyword(s):  
International Student ◽  
Educational Achievement ◽  
Student Assessment ◽  
Educational Standards ◽  
Content Type ◽  
Fiscal Constraints ◽  
International Student Assessment ◽  
University Courses ◽  
Low Wages

Subject Efforts to improve educational standards. Significance Of the 65 countries covered by the Programme for International Student Assessment (PISA) study in 2012, Peru came last for educational achievement. Although President Ollanta Humala's administration has received praise for the priority it has given to the educational sector, Peru will need to expend more effort in improving standards if it is ever to be admitted to the OECD. Impacts Teachers' low wages will continue to represent an obstacle to educational improvements. Fiscal constraints will make it more difficult for the next government, which takes office in July, to devote more money to education. There is still a mismatch between existing university courses and the need to boost technological expertise.

School discipline, investment, competitiveness and mediating educational performance

2017 ◽  
Vol 31 (3) ◽  
pp. 293-319 ◽  
Author(s):  
Hana Krskova ◽  
Chris Baumann
Keyword(s):  
International Student ◽  
School Discipline ◽  
Student Assessment ◽  
Partial Least Square ◽  
Least Square ◽  
Educational Performance ◽  
Start Time ◽  
Content Type ◽  
International Student Assessment ◽  
Education Investment

Purpose The purpose of this paper is to combine seemingly unrelated factors to explain global competitiveness. The study argues that school discipline and education investment affect competitiveness with the association being mediated by educational performance. Crucially, diachronic effects of discipline on performance are tested to demonstrate effects over time. Design/methodology/approach Partial least square (PLS) modelling is used to analyse the Organization for Economic Co-operation and Development’s Programme for International Student Assessment (PISA) data. The study further draws from World Bank data on Government Expenditure and World Economic Forum data on competitiveness. Five PISA dimensions of school discipline (students listening well, noise levels, teacher waiting time, students working well, class start time) are hypothesised to affect academic performance in reading, math and science, and to ultimately impact competitiveness. Findings Findings confirm the relative importance of school discipline (88 per cent) in comparison to education investment (12 per cent) on educational performance, with both variables also being found to be significantly associated with competitiveness directly. Originality/value This study demonstrates the time effects of discipline, more specifically that discipline dimensions (students listen well in 2003 and students work well in 2009) are associated with competitiveness in 2012. Implications for school policy and further research are discussed.

Athens braces for battle on school reform

2021 ◽  
Keyword(s):  
School Reform ◽  
International Student ◽  
Student Assessment ◽  
Private Investment ◽  
Political Crisis ◽  
State Budget ◽  
Content Type ◽  
International Student Assessment ◽  
Major Reform ◽  
The Government

Significance Prime Minister Kyriakos Mitsotakis and Education Minister Niki Kerameus announced the reform on June 28. It includes hiring 11,700 elementary and high-school teachers on tenured job contracts. Since the performance of Greece’s school system ranks among the lowest on Programme for International Student Assessment (PISA) indicators, a major reform has been long overdue. Impacts Given the high number of teachers to be hired, the government may have underestimated the financial cost of school reform. The state budget will simultaneously require funds for the management of the ongoing COVID-19 pandemic. The reform may open a new battle front for the government this autumn, uniting anti-government unions and opposition parties. A general election is not in sight, but a political crisis could upset the stability being sought to attract much-needed private investment.

Understanding and detecting data fabrication in large-scale assessments

2018 ◽  
Vol 26 (2) ◽  
pp. 196-212 ◽  
Author(s):  
Kentaro Yamamoto ◽  
Mary Louise Lennon
Keyword(s):  
Large Scale ◽  
International Student ◽  
Student Assessment ◽  
Response Patterns ◽  
Content Type ◽  
Large Scale Assessment ◽  
International Student Assessment ◽  
Data Fabrication ◽  
Computer Based ◽  
Types Of Information

Purpose Fabricated data jeopardize the reliability of large-scale population surveys and reduce the comparability of such efforts by destroying the linkage between data and measurement constructs. Such data result in the loss of comparability across participating countries and, in the case of cyclical surveys, between past and present surveys. This paper aims to describe how data fabrication can be understood in the context of the complex processes involved in the collection, handling, submission and analysis of large-scale assessment data. The actors involved in those processes, and their possible motivations for data fabrication, are also elaborated. Design/methodology/approach Computer-based assessments produce new types of information that enable us to detect the possibility of data fabrication, and therefore the need for further investigation and analysis. The paper presents three examples that illustrate how data fabrication was identified and documented in the Programme for the International Assessment of Adult Competencies (PIAAC) and the Programme for International Student Assessment (PISA) and discusses the resulting remediation efforts. Findings For two countries that participated in the first round of PIAAC, the data showed a subset of interviewers who handled many more cases than others. In Case 1, the average proficiency for respondents in those interviewers’ caseloads was much higher than expected and included many duplicate response patterns. In Case 2, anomalous response patterns were identified. Case 3 presents findings based on data analyses for one PISA country, where results for human-coded responses were shown to be highly inflated compared to past results. Originality/value This paper shows how new sources of data, such as timing information collected in computer-based assessments, can be combined with other traditional sources to detect fabrication.

Sign in / Sign up

Export Citation Format

Share Document