scholarly journals Why ability point estimates can be pointless: a primer on using skill measures from large-scale assessments in secondary analyses

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Clemens M. Lechner ◽  
Nivedita Bhaktha ◽  
Katharina Groskurth ◽  
Matthias Bluemke
Keyword(s):  
Measurement Error ◽  
Statistical Models ◽  
Test Scores ◽  
Large Scale ◽  
Equation Modeling ◽  
Model Parameters ◽  
Advantages And Disadvantages ◽  
Point Estimates ◽  
Secondary Analyses ◽  
Large Scale Assessments

AbstractMeasures of cognitive or socio-emotional skills from large-scale assessments surveys (LSAS) are often based on advanced statistical models and scoring techniques unfamiliar to applied researchers. Consequently, applied researchers working with data from LSAS may be uncertain about the assumptions and computational details of these statistical models and scoring techniques and about how to best incorporate the resulting skill measures in secondary analyses. The present paper is intended as a primer for applied researchers. After a brief introduction to the key properties of skill assessments, we give an overview over the three principal methods with which secondary analysts can incorporate skill measures from LSAS in their analyses: (1) as test scores (i.e., point estimates of individual ability), (2) through structural equation modeling (SEM), and (3) in the form of plausible values (PVs). We discuss the advantages and disadvantages of each method based on three criteria: fallibility (i.e., control for measurement error and unbiasedness), usability (i.e., ease of use in secondary analyses), and immutability (i.e., consistency of test scores, PVs, or measurement model parameters across different analyses and analysts). We show that although none of the methods are optimal under all criteria, methods that result in a single point estimate of each respondent’s ability (i.e., all types of “test scores”) are rarely optimal for research purposes. Instead, approaches that avoid or correct for measurement error—especially PV methodology—stand out as the method of choice. We conclude with practical recommendations for secondary analysts and data-producing organizations.

Developments in Psychometric Population Models for Technology-Based Large-Scale Assessments: An Overview of Challenges and Opportunities

2019 ◽  
Vol 44 (6) ◽  
pp. 671-705 ◽  
Author(s):  
Matthias von Davier ◽  
Lale Khorramdel ◽  
Qiwei He ◽  
Hyo Jeong Shin ◽  
Haiwen Chen
Keyword(s):  
Test Scores ◽  
Large Scale ◽  
Population Models ◽  
Data Types ◽  
Challenges And Opportunities ◽  
Computer Based ◽  
Complex Test ◽  
Large Scale Assessments ◽  
Item Types ◽  
Latent Regression

International large-scale assessments (ILSAs) transitioned from paper-based assessments to computer-based assessments (CBAs) facilitating the use of new item types and more effective data collection tools. This allows implementation of more complex test designs and to collect process and response time (RT) data. These new data types can be used to improve data quality and the accuracy of test scores obtained through latent regression (population) models. However, the move to a CBA also poses challenges for comparability and trend measurement, one of the major goals in ISLAs. We provide an overview of current methods used in ILSAs to examine and assure the comparability of data across different assessment modes and methods that improve the accuracy of test scores by making use of new data types provided by a CBA.

scholarly journals To Score or Not to Score? A Simulation Study on the Performance of Test Scores, Plausible Values, and SEM, in Regression With Socio-Emotional Skill or Personality Scales as Predictors

2021 ◽  
Vol 12 ◽  
Author(s):  
Nivedita Bhaktha ◽  
Clemens M. Lechner
Keyword(s):  
Measurement Error ◽  
Simulation Study ◽  
Test Scores ◽  
Regression Coefficient ◽  
Maximum Likelihood Estimates ◽  
Regression Coefficients ◽  
Equation Modeling ◽  
Emotional Skills ◽  
Plausible Values ◽  
Different Types

This article addresses a fundamental question in the study of socio-emotional skills, personality traits, and related constructs: “To score or not to score?” When researchers use test scores or scale scores (i.e., fallible point estimates of a skill or trait) as predictors in multiple regression, measurement error in these scores tends to attenuate regression coefficients for the skill and inflate those of the covariates. Unlike for cognitive assessments, it is not fully established how severe this bias can be in socio-emotional skill assessments, that is, how well test scores recover the true regression coefficients — compared with methods designed to account for measurement error: structural equation modeling (SEM) and plausible values (PV). The different types of scores considered in this study are standardized mean scores (SMS), regression factor scores (RFS), empirical Bayes modal (EBM) score, weighted maximum likelihood estimates (WLE), and expected a posteriori (EAP) estimates. We present a simulation study in which we compared these approaches under conditions typical of socio-emotional skill and personality assessments. We examined the performance of five types of test scores, PV, and SEM with regard to two outcomes: (1) percent bias in regression coefficient of the skill in predicting an outcome; and (2) percent bias in the regression coefficient of a covariate. We varied the number of items, factor loadings/item discriminations, sample size, and relative strength of the relationship of the skill with the outcome. Results revealed that whereas different types of test scores were highly correlated with each other, the ensuing bias in regression coefficients varied considerably. The magnitude of bias was highest for WLE with short scales of low reliability. Bias when using SMS or WLE test scores was sometimes large enough to lead to erroneous research conclusions with potentially adverse implications for policy and practice (up to 55% for the regression coefficient of the skill and 20% for that of the covariate). EAP, EBM, and RFS performed better, producing only small bias in some conditions. Additional analyses showed that the performance of test scores also depended on whether standardized or unstandardized scores were used. Only PV and SEM performed well in all scenarios and emerged as the clearly superior options. We recommend that researchers use SEM, and preferably PV, in studies on the (incremental) predictive power of socio-emotional skills.

Implantable neural interfaces

2018 ◽  
Author(s):  
Stefano Vassanelli
Keyword(s):  
Brain Function ◽  
Large Scale ◽  
Ionic Channels ◽  
Patch Clamp Technique ◽  
Advantages And Disadvantages ◽  
Optogenetic Stimulation ◽  
Physical Interfaces ◽  
The Brain

Establishing direct communication with the brain through physical interfaces is a fundamental strategy to investigate brain function. Starting with the patch-clamp technique in the seventies, neuroscience has moved from detailed characterization of ionic channels to the analysis of single neurons and, more recently, microcircuits in brain neuronal networks. Development of new biohybrid probes with electrodes for recording and stimulating neurons in the living animal is a natural consequence of this trend. The recent introduction of optogenetic stimulation and advanced high-resolution large-scale electrical recording approaches demonstrates this need. Brain implants for real-time neurophysiology are also opening new avenues for neuroprosthetics to restore brain function after injury or in neurological disorders. This chapter provides an overview on existing and emergent neurophysiology technologies with particular focus on those intended to interface neuronal microcircuits in vivo. Chemical, electrical, and optogenetic-based interfaces are presented, with an analysis of advantages and disadvantages of the different technical approaches.

Age and gender differences in socially aversive (“dark”) personality traits

2021 ◽  
pp. 089020702098843
Author(s):  
Johanna Hartung ◽  
Martina Bader ◽  
Morten Moshagen ◽  
Oliver Wilhelm
Keyword(s):  
Personality Traits ◽  
Structural Equation ◽  
Age Groups ◽  
Dark Triad ◽  
Equation Modeling ◽  
Model Parameters ◽  
Age And Gender ◽  
Measurement Models ◽  
And Gender ◽  
Latent Means

The strong overlap of personality traits discussed under the label of “dark personality” (e.g., psychopathy, spitefulness, moral disengagement) endorses a common framework for socially aversive traits over and beyond the dark triad. Despite the rapidly growing research on socially aversive traits, there is a lack of studies addressing age-associated differences in these traits. In the present study ( N = 12,501), we investigated the structure of the D Factor of Personality across age and gender using local structural equation modeling, thereby expressing the model parameters as a quasi-continuous, nonparametric function of age. Specifically, we evaluated loadings, reliabilities, factor (co-)variances, and means across 35 locally weighted age groups (from 20 to 54 years), separately for females and males. Results indicated that measurement models were highly stable, thereby supporting the conceptualization of the D factor independent of age and gender. Men exhibited uniformly higher latent means than females and all latent means decreased with increasing age. Overall, D and its themes were invariant across age and gender. Therefore, future studies can meaningfully pursue causes of mean differences across age and between genders.

scholarly journals Cyberstalking Victimization Model Using Criminological Theory: A Systematic Literature Review, Taxonomies, Applications, Tools, and Validations

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1670
Author(s):  
Waheeb Abu-Ulbeh ◽  
Maryam Altalhi ◽  
Laith Abualigah ◽  
Abdulwahab Ali Almazroi ◽  
Putra Sumari ◽  
...  
Keyword(s):  
Data Analysis ◽  
Structural Equation ◽  
Large Scale ◽  
Review Paper ◽  
Essential Element ◽  
Routine Activities ◽  
Criminological Theory ◽  
Equation Modeling ◽  
Future Research ◽  
Proposed Model

Cyberstalking is a growing anti-social problem being transformed on a large scale and in various forms. Cyberstalking detection has become increasingly popular in recent years and has technically been investigated by many researchers. However, cyberstalking victimization, an essential part of cyberstalking, has empirically received less attention from the paper community. This paper attempts to address this gap and develop a model to understand and estimate the prevalence of cyberstalking victimization. The model of this paper is produced using routine activities and lifestyle exposure theories and includes eight hypotheses. The data of this paper is collected from the 757 respondents in Jordanian universities. This review paper utilizes a quantitative approach and uses structural equation modeling for data analysis. The results revealed a modest prevalence range is more dependent on the cyberstalking type. The results also indicated that proximity to motivated offenders, suitable targets, and digital guardians significantly influences cyberstalking victimization. The outcome from moderation hypothesis testing demonstrated that age and residence have a significant effect on cyberstalking victimization. The proposed model is an essential element for assessing cyberstalking victimization among societies, which provides a valuable understanding of the prevalence of cyberstalking victimization. This can assist the researchers and practitioners for future research in the context of cyberstalking victimization.

Correlated personality change across time and age

2021 ◽  
pp. 089020702110140
Author(s):  
Gabriel Olaru ◽  
Mathias Allemand
Keyword(s):  
Structural Equation ◽  
Rank Order ◽  
Personality Development ◽  
The United States ◽  
Change Score ◽  
Personality Change ◽  
Equation Modeling ◽  
Model Parameters ◽  
Cross Sectional ◽  
Adult Lifespan

The goal of this study was to examine differential and correlated change in personality across the adult lifespan. Studying differential and correlated change can help understand whether intraindividual trait change trajectories deviate from the norm and how these trajectories are coupled with each other. We used data from two large longitudinal panel studies from the United States that covered a total age range of 20 to 95 years on the first measurement occasion. We used correlated factor models and bivariate latent change score models to examine the rank-order stability and correlations between change across three measurement waves covering 18 years ( N = 3250) and four measurement waves covering 12 years ( N = 4145). We examined the moderation effects of continuous age on these model parameters using local structural equation modeling. The results suggest that the test–retest correlations decrease with increasing time between measurements but are unaffected by participants’ age. We found that change processes in Extraversion, Openness, Agreeableness, and Conscientiousness were strongly related, particularly in late adulthood. Correlated change patterns were highly stable across time intervals and similar to the initial cross-sectional Big Five correlations. We discuss potential mechanisms and implications for personality development research.

Explanation Plus Prediction—The Logical Focus of Project Management Research

2021 ◽  
pp. 875697282199994
Author(s):  
Joseph F. Hair ◽  
Marko Sarstedt
Keyword(s):  
Project Management ◽  
Statistical Models ◽  
Predictive Power ◽  
Mean Absolute Error ◽  
Explanatory Power ◽  
Absolute Error ◽  
Model Parameters ◽  
Mean Square ◽  
Management Research ◽  
The Mean

Most project management research focuses almost exclusively on explanatory analyses. Evaluation of the explanatory power of statistical models is generally based on F-type statistics and the R 2 metric, followed by an assessment of the model parameters (e.g., beta coefficients) in terms of their significance, size, and direction. However, these measures are not indicative of a model’s predictive power, which is central for deriving managerial recommendations. We recommend that project management researchers routinely use additional metrics, such as the mean absolute error or the root mean square error, to accurately quantify their statistical models’ predictive power.

Green Practices Implementation as Prerequisite to Sustain Firm Competitive Advantages

2015 ◽  
Vol 6 (4) ◽  
pp. 34-53
Author(s):  
Kardison Lumbanbatu ◽  
Vincent Didiek Wiet Aryanto
Keyword(s):  
Product Innovation ◽  
Structural Equation ◽  
Large Scale ◽  
Management Practices ◽  
Corporate Image ◽  
Equation Modeling ◽  
Competitive Advantages ◽  
Green Management ◽  
Marketing Performance

Encompassing firms to apply green policy in a holistic management practices are strongly required in order to maintain competitive advantages and experience long-term marketing performance. This current empirical research is aimed to fill the lack of empirical findings and empirical studies on firm's innovative concept. Green-based product innovation, green management practices and green corporate image are presented as the antecedents and postulated as the sources of sustaining firm competitive advantages. A questionnaire-based survey was deployed to collect data from Large Scale Enterprises in Indonesia with Top Management, Operational and Marketing Managers served as respondents. 500 questionnaires were mailed and 388 were valid for further analysis. Data was analyzed by using Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM) via AMOS statistical software. Statistical findings demonstrated that green-based product innovation, green management practices and green corporate image significantly has a positive affect to sustain firm competitive advantages which is led to enhance long term marketing performance. However, green-based product innovation plays insignificant direct relationship on long term marketing performance. This study discusses some managerial implications for enterprises and recommendations on a basis of green implementation.

Sign in / Sign up

Export Citation Format

Share Document