The Dimensionality of Reading Self-Concept: Examining Its Stability Using Local Structural Equation Models

Assessment ◽  
2022 ◽  
pp. 107319112110696
Author(s):  
Geetanjali Basarkod ◽  
Herbert W. Marsh ◽  
Baljinder K. Sahdra ◽  
Philip D. Parker ◽  
Jiesi Guo ◽  
...  
Keyword(s):  
Reading Achievement ◽  
Structural Equation ◽  
Large Scale ◽  
Structural Equation Models ◽  
Student Assessment ◽  
Cognitive Testing ◽  
Self Concept ◽  
International Student Assessment ◽  
High Competence ◽  
Achievement Levels

For results from large-scale surveys to inform policy and practice appropriately, all participants must interpret and respond to items similarly. While organizers of surveys assessing student outcomes often ensure this for achievement measures, doing so for psychological questionnaires is also critical. We demonstrate this by examining the dimensionality of reading self-concept—a crucial psychological construct for several outcomes—across reading achievement levels. We use Programme for International Student Assessment 2018 data ( N = 529,966) and local structural equation models (LSEMs) to do so. Results reveal that reading self-concept dimensions (assessed through reading competence and difficulty) vary across reading achievement levels. Students with low reading achievement show differentiated responses to the two item sets (high competence–high difficulty). In contrast, students with high reading achievement have reconciled responses (high competence–low difficulty). Our results highlight the value of LSEMs in examining factor structure generalizability of constructs in large-scale surveys and call for greater cognitive testing during item development.

scholarly journals The Mechanism Underlying the Association of School Climate and Reading Achievement: The Mediating Role of Intrinsic Motivation and Reading Self-Concept in the Greek Context

2020 ◽  
Vol 7 ◽  
pp. 59-73
Author(s):  
Ioannis Katsantonis
Keyword(s):  
Academic Achievement ◽  
School Climate ◽  
Reading Achievement ◽  
Intrinsic Motivation ◽  
Structural Equation ◽  
Student Assessment ◽  
Self Concept ◽  
Equation Modeling ◽  
International Student Assessment ◽  
Underlying Mechanisms

The association between school climate and students’ achievement is currently well-documented in international literature. However, the relevant studies, that contemplate the underlying mechanisms of this association, are sparse. Therefore, the present study’s purpose is to confirm the mediating effects of intrinsic motivation and reading self-concept in the association of school climate and reading achievement. Further, due to the amassing evidence of gender-related individual differences in academic achievement, this study also examines whether the underlying mechanisms toward academic achievement are varying as a function of gender. The data of N=6,403 Greek adolescent students were extracted from the Program of International Student Assessment (PISA) for further analyses. Structural equation modeling (SEM) was conducted to examine both mediating and moderating effects. The results indicate that there is a positive indirect effect of school climate on reading achievement via intrinsic motivation and self-concept. Additionally, structural equivalence via multigroup SEM (MGSEM) showed that gender does not moderate the structural regressive relations; that is, the regression coefficients did not vary as a function of gender. These findings are discussed within the framework of improving educational practices.

scholarly journals Science Self-Concept – More Than the Sum of its Parts? A Comparison of Different Assessment and Modeling Approaches

2019 ◽  
Author(s):  
Ulrich Schroeders ◽  
Malte Jansen
Keyword(s):  
Structural Equation ◽  
Large Scale ◽  
Structural Equation Models ◽  
Factor Model ◽  
Self Concept ◽  
General Science ◽  
Theoretical Status ◽  
Subject Specific ◽  
Mean Structure ◽  
The Subject

Academic self-concept is understood as a multidimensional, hierarchical construct. Multidimensionality refers to the subject-specific differentiation of academic self-concepts, whereas hierarchy refers to the aggregation of more specific facets of self-concepts into more general ones. Previous research demonstrated that students distinguish between their self-concepts in biology, chemistry, and physics if taught as separate school subjects, as is done in Germany. However, large-scale international educational studies, such as PISA, often use a monolithic science self-concept measure. It is yet unclear whether an aggregate of subject-specific self-concepts is equivalent to a directly measured science self-concept. We assessed the subject-specific and and a general science self-concept of 1,232 German grade 10 students. A higher-order factor model and a bifactor model demonstrated a very high correlation between the “inferred” and the explicitly assessed general science self-concept. Despite the high empirical overlap, we argue for a more nuanced view of the science self-concept, because statistical unity is not to be confused with causal unity. Moreover, from a methodological perspective, we used multi-group confirmatory factor analysis to examine the mean structure and local weighted structural equation models to study measurement invariance across science ability. Implications for the theoretical status of self-concept as a hierarchical construct are discussed.

Predictors of teachers' appreciation of evidence-based practice and educational research findings

2021 ◽  
pp. 1-12 ◽  
Author(s):  
Eva Thomm ◽  
Christine Sälzer ◽  
Manfred Prenzel ◽  
Johannes Bauer
Keyword(s):  
Educational Research ◽  
Research Methods ◽  
Structural Equation ◽  
International Student ◽  
Structural Equation Models ◽  
Evidence Based Practice ◽  
Student Assessment ◽  
Evidence Based ◽  
International Student Assessment ◽  
Research Findings

Abstract. Teachers' reception of educational research is considered important for improving teaching and student learning. Yet, it is a challenging task requiring teachers to have access to scientific sources, the skill and time to find and exhaust such resources, and the capacity to interpret retrieved information. If such essential conditions are not met, teachers have hardly any chance to engage in research reception and, consequently, may question the value and relevance of research findings to their practice. Prior research has suggested that teachers are indeed critical of educational research findings and rarely refer to them. Based on data from the field trial ( N = 674) and main study ( N = 2,549) of a national extension study of the Programme for International Student Assessment (PISA) 2012 in Germany, this study explored the role of (a) teachers' access to scientific sources, (b) perceived lack of skill and time to search for research findings, and (c) their familiarity with research methods/statistics as potential predictors of their appreciation of evidence-based practice, and perceived irrelevance of educational research findings. Structural equation models demonstrated that perceived lack of skill and time to find research findings, in particular, substantially affected participants' irrelevance perceptions. The more participants assessed their sourcing skill and time to be too constrained to engage in research reception, the more they judged research findings to be irrelevant to their practice. Though source access and familiarity with research methods/statistics indicated only small or even no effects, they strongly correlated with participants' perceived lack of sourcing skill and time. Better source access and greater familiarity were associated with less concern about one's skill and time resources to search for relevant research findings. These findings potentially underline the relevance of strengthening both teachers' access to scientific sources and individual capacities to understanding research contents.

Three Paradoxical Effects on Academic Self-Concept Across Countries, Schools, and Students

2019 ◽  
Vol 24 (3) ◽  
pp. 231-242 ◽  
Author(s):  
Herbert W. Marsh ◽  
Philip D. Parker ◽  
Reinhard Pekrun
Keyword(s):  
International Student ◽  
Student Assessment ◽  
Cross Cultural ◽  
Frame Of Reference ◽  
School Level ◽  
Self Concept ◽  
Positive Effects ◽  
International Student Assessment ◽  
Country Level ◽  
Starting School

Abstract. We simultaneously resolve three paradoxes in academic self-concept research with a single unifying meta-theoretical model based on frame-of-reference effects across 68 countries, 18,292 schools, and 485,490 15-year-old students. Paradoxically, but consistent with predictions, effects on math self-concepts were negative for: • being from countries where country-average achievement was high; explaining the paradoxical cross-cultural self-concept effect; • attending schools where school-average achievement was high; demonstrating big-fish-little-pond-effects (BFLPE) that generalized over 68 countries, Organisation for Economic Co-operation and Development (OECD)/non-OECD countries, high/low achieving schools, and high/low achieving students; • year-in-school relative to age; unifying different research literatures for associated negative effects for starting school at a younger age and acceleration/skipping grades, and positive effects for starting school at an older age (“academic red shirting”) and, paradoxically, even for repeating a grade. Contextual effects matter, resulting in significant and meaningful effects on self-beliefs, not only at the student (year in school) and local school level (BFLPE), but remarkably even at the macro-contextual country-level. Finally, we juxtapose cross-cultural generalizability based on Programme for International Student Assessment (PISA) data used here with generalizability based on meta-analyses, arguing that although the two approaches are similar in many ways, the generalizability shown here is stronger in terms of support for the universality of the frame-of-reference effects.

Are There Test Administrator Effects in Large-Scale Educational Assessments?

Methodology ◽  
2007 ◽  
Vol 3 (4) ◽  
pp. 149-159 ◽  
Author(s):  
Oliver Lüdtke ◽  
Alexander Robitzsch ◽  
Ulrich Trautwein ◽  
Frauke Kreuter ◽  
Jan Marten Ihme
Keyword(s):  
Large Scale ◽  
International Student ◽  
Student Assessment ◽  
Student Groups ◽  
Mathematics Scores ◽  
International Student Assessment ◽  
Sample Attrition ◽  
Educational Assessments ◽  
Test Administrator ◽  
Practical Implications

Abstract. In large-scale educational assessments such as the Third International Mathematics and Sciences Study (TIMSS) or the Program for International Student Assessment (PISA), sizeable numbers of test administrators (TAs) are needed to conduct the assessment sessions in the participating schools. TA training sessions are run and administration manuals are compiled with the aim of ensuring standardized, comparable, assessment situations in all student groups. To date, however, there has been no empirical investigation of the effectiveness of these standardizing efforts. In the present article, we probe for systematic TA effects on mathematics achievement and sample attrition in a student achievement study. Multilevel analyses for cross-classified data using Markov Chain Monte Carlo (MCMC) procedures were performed to separate the variance that can be attributed to differences between schools from the variance associated with TAs. After controlling for school effects, only a very small, nonsignificant proportion of the variance in mathematics scores and response behavior was attributable to the TAs (< 1%). We discuss practical implications of these findings for the deployment of TAs in educational assessments.

scholarly journals Early tracking and different types of inequalities in achievement: difference-in-differences evidence from 20 years of large-scale assessments

2021 ◽  
Vol 33 (1) ◽  
pp. 139-167
Author(s):  
Andrés Strello ◽  
Rolf Strietholt ◽  
Isa Steinmann ◽  
Charlotte Siepmann
Keyword(s):  
Large Scale ◽  
International Student ◽  
Student Assessment ◽  
Difference In Differences ◽  
Seminal Paper ◽  
International Student Assessment ◽  
Units Of Analysis ◽  
Social Achievement ◽  
Mathematics And Science ◽  
Combined Data

AbstractResearch to date on the effects of between-school tracking on inequalities in achievement and on performance has been inconclusive. A possible explanation is that different studies used different data, focused on different domains, and employed different measures of inequality. To address this issue, we used all accumulated data collected in the three largest international assessments—PISA (Programme for International Student Assessment), PIRLS (Progress in International Reading Literacy Study), and TIMSS (Trends in International Mathematics and Science Study)—in the past 20 years in 75 countries and regions. Following the seminal paper by Hanushek and Wößmann (2006), we combined data from a total of 21 cycles of primary and secondary school assessments to estimate difference-in-differences models for different outcome measures. We synthesized the effects using a meta-analytical approach and found strong evidence that tracking increased social achievement gaps, that it had smaller but still significant effects on dispersion inequalities, and that it had rather weak effects on educational inadequacies. In contrast, we did not find evidence that tracking increased performance levels. Besides these substantive findings, our study illustrated that the effect estimates varied considerably across the datasets used because the low number of countries as the units of analysis was a natural limitation. This finding casts doubt on the reproducibility of findings based on single international datasets and suggests that researchers should use different data sources to replicate analyses.

Structural equation modelling with complex sampling designs and non-random attrition: A tutorial using lavaan and Mplus

2021 ◽  
Author(s):  
Aja Louise Murray ◽  
Anastasia Ushakova ◽  
Helen Wright ◽  
Tom Booth ◽  
Peter Lynn
Keyword(s):  
Structural Equation ◽  
Large Scale ◽  
Structural Equation Models ◽  
Cluster Sampling ◽  
Complex Sampling ◽  
Latent Trajectory ◽  
Adequate Sampling ◽  
Complex Sampling Designs ◽  
Selection Probabilities ◽  
Pseudo Maximum Likelihood

Complex sampling designs involving features such as stratification, cluster sampling, and unequal selection probabilities are often used in large-scale longitudinal surveys to improve cost-effectiveness and ensure adequate sampling of small or under-represented groups. However, complex sampling designs create challenges when there is a need to account for non-random attrition; a near inevitability in social science longitudinal studies. In this article we discuss these challenges and demonstrate the application of weighting approaches to simultaneously account for non-random attrition and complex design in a large UK-population representative survey. Using an auto-regressive latent trajectory model with structured residuals (ALT-SR) to model the relations between relationship satisfaction and mental health in the Understanding Society study as an example, we provide guidance on implementation of this approach in both R and Mplus is provided. Two standard error estimation approaches are illustrated: pseudo-maximum likelihood robust estimation and Bootstrap resampling. A comparison of unadjusted and design-adjusted results also highlights that ignoring the complex survey designs when fitting structural equation models can result in misleading conclusions.

TIMSS 2015: Illustrating Advancements in Large-Scale International Assessments

2019 ◽  
Vol 44 (6) ◽  
pp. 752-781
Author(s):  
Michael O. Martin ◽  
Ina V.S. Mullis
Keyword(s):  
Student Achievement ◽  
Large Scale ◽  
International Student ◽  
Variance Estimation ◽  
International Association ◽  
Student Assessment ◽  
Population Sampling ◽  
International Student Assessment ◽  
International Assessments ◽  
Matrix Sampling

International large-scale assessments of student achievement such as International Association for the Evaluation of Educational Achievement’s Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study and Organization for Economic Cooperation and Development’s Program for International Student Assessment that have come to prominence over the past 25 years owe a great deal in methodological terms to pioneering work by National Assessment of Educational Progress (NAEP). Using TIMSS as an example, this article describes how a number of core techniques, such as matrix sampling, student population sampling, item response theory scaling with population modeling, and resampling methods for variance estimation, have been adapted and implemented in an international context and are fundamental to the international assessment effort. In addition to the methodological contributions of NAEP, this article illustrates how the large-scale international assessments go beyond measuring student achievement by representing important aspects of community, home, school, and classroom contexts in ways that can be used to address issues of importance to researchers and policymakers.

scholarly journals A Practical Guide for Analyzing Large-Scale Assessment Data Using Mplus: A Case Demonstration Using the Program for International Assessment of Adult Competencies Data

2020 ◽  
pp. 107699862097855
Author(s):  
Takashi Yamashita ◽  
Thomas J. Smith ◽  
Phyllis A. Cummins
Keyword(s):  
Structural Equation ◽  
Large Scale ◽  
Structural Equation Models ◽  
Preparation Process ◽  
Data Preparation ◽  
Assessment Data ◽  
Statistical Software ◽  
Large Scale Assessment ◽  
Scale Assessment ◽  
International Assessment

In order to promote the use of increasingly available large-scale assessment data in education and expand the scope of analytic capabilities among applied researchers, this study provides step-by-step guidance, and practical examples of syntax and data analysis using Mplus. Concise overview and key unique aspects of large-scale assessment data from the 2012/2014 Program for International Assessment of Adult Competencies (PIAAC) are described. Using commonly-used statistical software including SAS and R, a simple macro program and syntax are developed to streamline the data preparation process. Then, two examples of structural equation models are demonstrated using Mplus. The suggested data preparation and analytic approaches can be immediately applicable to existing large-scale assessment data.

Sign in / Sign up

Export Citation Format

Share Document