scholarly journals To Score or Not to Score? A Simulation Study on the Performance of Test Scores, Plausible Values, and SEM, in Regression With Socio-Emotional Skill or Personality Scales as Predictors

2021 ◽  
Vol 12 ◽  
Author(s):  
Nivedita Bhaktha ◽  
Clemens M. Lechner
Keyword(s):  
Measurement Error ◽  
Simulation Study ◽  
Test Scores ◽  
Regression Coefficient ◽  
Maximum Likelihood Estimates ◽  
Regression Coefficients ◽  
Equation Modeling ◽  
Emotional Skills ◽  
Plausible Values ◽  
Different Types

This article addresses a fundamental question in the study of socio-emotional skills, personality traits, and related constructs: “To score or not to score?” When researchers use test scores or scale scores (i.e., fallible point estimates of a skill or trait) as predictors in multiple regression, measurement error in these scores tends to attenuate regression coefficients for the skill and inflate those of the covariates. Unlike for cognitive assessments, it is not fully established how severe this bias can be in socio-emotional skill assessments, that is, how well test scores recover the true regression coefficients — compared with methods designed to account for measurement error: structural equation modeling (SEM) and plausible values (PV). The different types of scores considered in this study are standardized mean scores (SMS), regression factor scores (RFS), empirical Bayes modal (EBM) score, weighted maximum likelihood estimates (WLE), and expected a posteriori (EAP) estimates. We present a simulation study in which we compared these approaches under conditions typical of socio-emotional skill and personality assessments. We examined the performance of five types of test scores, PV, and SEM with regard to two outcomes: (1) percent bias in regression coefficient of the skill in predicting an outcome; and (2) percent bias in the regression coefficient of a covariate. We varied the number of items, factor loadings/item discriminations, sample size, and relative strength of the relationship of the skill with the outcome. Results revealed that whereas different types of test scores were highly correlated with each other, the ensuing bias in regression coefficients varied considerably. The magnitude of bias was highest for WLE with short scales of low reliability. Bias when using SMS or WLE test scores was sometimes large enough to lead to erroneous research conclusions with potentially adverse implications for policy and practice (up to 55% for the regression coefficient of the skill and 20% for that of the covariate). EAP, EBM, and RFS performed better, producing only small bias in some conditions. Additional analyses showed that the performance of test scores also depended on whether standardized or unstandardized scores were used. Only PV and SEM performed well in all scenarios and emerged as the clearly superior options. We recommend that researchers use SEM, and preferably PV, in studies on the (incremental) predictive power of socio-emotional skills.

scholarly journals Why ability point estimates can be pointless: a primer on using skill measures from large-scale assessments in secondary analyses

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Clemens M. Lechner ◽  
Nivedita Bhaktha ◽  
Katharina Groskurth ◽  
Matthias Bluemke
Keyword(s):  
Measurement Error ◽  
Statistical Models ◽  
Test Scores ◽  
Large Scale ◽  
Equation Modeling ◽  
Model Parameters ◽  
Advantages And Disadvantages ◽  
Point Estimates ◽  
Secondary Analyses ◽  
Large Scale Assessments

AbstractMeasures of cognitive or socio-emotional skills from large-scale assessments surveys (LSAS) are often based on advanced statistical models and scoring techniques unfamiliar to applied researchers. Consequently, applied researchers working with data from LSAS may be uncertain about the assumptions and computational details of these statistical models and scoring techniques and about how to best incorporate the resulting skill measures in secondary analyses. The present paper is intended as a primer for applied researchers. After a brief introduction to the key properties of skill assessments, we give an overview over the three principal methods with which secondary analysts can incorporate skill measures from LSAS in their analyses: (1) as test scores (i.e., point estimates of individual ability), (2) through structural equation modeling (SEM), and (3) in the form of plausible values (PVs). We discuss the advantages and disadvantages of each method based on three criteria: fallibility (i.e., control for measurement error and unbiasedness), usability (i.e., ease of use in secondary analyses), and immutability (i.e., consistency of test scores, PVs, or measurement model parameters across different analyses and analysts). We show that although none of the methods are optimal under all criteria, methods that result in a single point estimate of each respondent’s ability (i.e., all types of “test scores”) are rarely optimal for research purposes. Instead, approaches that avoid or correct for measurement error—especially PV methodology—stand out as the method of choice. We conclude with practical recommendations for secondary analysts and data-producing organizations.

scholarly journals Reliability for Tests With Items Having Different Numbers of Ordered Categories

2019 ◽  
Vol 44 (2) ◽  
pp. 137-149
Author(s):  
Seohyun Kim ◽  
Zhenqiu Lu ◽  
Allan S. Cohen
Keyword(s):  
Structural Equation Modeling ◽  
Simulation Study ◽  
Test Scores ◽  
Structural Equation ◽  
Reliability Coefficient ◽  
Coefficient Alpha ◽  
Equation Modeling ◽  
Ordered Categories

This study describes a structural equation modeling (SEM) approach to reliability for tests with items having different numbers of ordered categories. A simulation study is provided to compare the performance of this reliability coefficient, coefficient alpha and population reliability for tests having items with different numbers of ordered categories, a one-factor and a bifactor structures, and different skewness distributions of test scores. Results indicated that the proposed reliability coefficient was close to the population reliability in most conditions. An empirical example was used to illustrate the performance of the different coefficients for a test of items with two or three ordered categories.

scholarly journals Compound Bias due to Measurement Error When Comparing Regression Coefficients

2019 ◽  
Vol 80 (3) ◽  
pp. 548-577
Author(s):  
William M. Murrah
Keyword(s):  
Measurement Error ◽  
Multiple Regression ◽  
Simulation Study ◽  
Type I Error ◽  
Real Data ◽  
Error Rates ◽  
Regression Coefficients ◽  
Type I ◽  
Type I Error Rates ◽  
The Impact

Multiple regression is often used to compare the importance of two or more predictors. When the predictors being compared are measured with error, the estimated coefficients can be biased and Type I error rates can be inflated. This study explores the impact of measurement error on comparing predictors when one is measured with error, followed by a simulation study to help quantify the bias and Type I error rates for common research situations. Two methods used to adjust for measurement error are demonstrated using a real data example. This study adds to the literature documenting the impact of measurement error on regression modeling, identifying issues particular to the use of multiple regression for comparing predictors, and offers recommendations for researchers conducting such studies.

scholarly journals Type 2 diabetes is more predictable in women than men by multiple anthropometric and biochemical measures

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Tangying Li ◽  
Huibiao Quan ◽  
Huachuan Zhang ◽  
Leweihua Lin ◽  
Lu Lin ◽  
...  
Keyword(s):  
Blood Pressure ◽  
Type 2 Diabetes ◽  
Diastolic Blood Pressure ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Regression Coefficient ◽  
Regression Coefficients ◽  
Sexually Dimorphic ◽  
Fasting Insulin

AbstractMen and women are sexually dimorphic but whether common anthropometric and biochemical parameters predict type 2 diabetes (T2D) in different ways has not been well studied. Here we recruit 1579 participants in Hainan Province, China, and group them by sex. We compared the prediction power of common parameters of T2D in two sexes by association, regression, and Receiver Operating Characteristic (ROC) analysis. HbA1c is associated with FPG stronger in women than in men and the regression coefficient is higher, consistent with higher prediction power for T2D. Age, waist circumference, BMI, systolic and diastolic blood pressure, triglyceride levels, total cholesterol, LDL, HDL, fasting insulin, and proinsulin levels all predict T2D better in women. Except for diastolic blood pressure, all parameters associate or tend to associate with FPG stronger in women than in men. Except for diastolic blood pressure and fasting proinsulin, all parameters associate or tend to associate with HbA1c stronger in women than in men. Except for fasting proinsulin and HDL, the regression coefficients of all parameters with FPG and HbA1c were higher in women than in men. Together, by the above anthropometric and biochemical measures, T2D is more readily predicted in women than men, suggesting the importance of sex-based subgroup analysis in T2D research.

Examination of customer-centric measures among different types of customers in the context of major Canadian ski resort

2018 ◽  
Vol 30 (2) ◽  
pp. 438-459
Author(s):  
Matti J. Haverila ◽  
Kai Christian Haverila
Keyword(s):  
Customer Satisfaction ◽  
Structural Equation ◽  
A Priori ◽  
Equation Modeling ◽  
Marketing Practice ◽  
Content Type ◽  
Ski Resort ◽  
Different Types ◽  
Managerial Implications ◽  
Satisfaction Model

Purpose Customer-centric measures such as customer satisfaction and repurchase intent are important indicators of performance. The purpose of this paper is to examine what is the strength and significance of the path coefficients in a customer satisfaction model consisting of various customer-centric measures for different types of ski resort customer (i.e. day, weekend and ski holiday visitors as well as season pass holders) in a ski resort in Canada. Design/methodology/approach The results were analyzed using the partial least squares structural equation modeling approach for the four different types ski resort visitors. Findings There appeared to differences in the strength and significance in the customer satisfaction model relationships for the four types of ski resort visitors indicating that the a priori managerial classification of the ski resort visitors is warranted. Originality/value The research pinpoints differences in the strength and significance in the relationships between customer-centric measures for four different types ski resort visitors, i.e. day, weekend and ski holiday visitors as well as season pass holders, which have significant managerial implications for the marketing practice of the ski resort.

scholarly journals PENGARUH KEPEMIMPINAN, KEPRIBADIAN, UMUR, DAN PENDIDIKAN TERHADAP KINERJA PEGAWAI MELALUI MOTIVASI PEGAWAI BALAI PELAKSANA TEKNIS BINA MARGA WILAYAH TEGAL

2017 ◽  
Vol 1 (2) ◽  
Author(s):  
Mr. Wasmo ◽  
Mr. Basuki
Keyword(s):  
Structural Equation Modeling ◽  
Structural Equation ◽  
Regression Coefficient ◽  
Employee Performance ◽  
Equation Modeling ◽  
P Value ◽  
Research Use ◽  
Leadership Motivation ◽  
And Performance

The purpose of this research is 1) to describe leadership, personality, age, education, motivation and the employee performance 2) to know the influence of leadership, personality, age, and education motivation employees to 3) to know the influence of leadership, personality, age, and education against the employee performance 4) to know the motivation to performance employees and 5) to know the influence of leadership, personality, age, and education on performance through motivation. Respondents in this study by the 107. These respondents are civil of technical execution Bina Marga Region Tegal. The methods of this research use Structural Equation Modeling (SEM) who run through AMOS as a means of the analysis. Is the between leadership, personality, age and education for employees at the motivation bina marga the tegal. This is evidenced of the value of the terstandar regression (beta) leadership motivation to obtain value of 0,11, to obtain personality motivation value of 0,32, my motivation to obtain value of -0,19 and education motivation to obtain value of 0,23 and more tender testing obtained value p-value very small (< 0,001). There are relations between leadership, personality, age and education of the performance of employees in city bina marga areas tegal. This is evidenced of the value of the regression coefficient terstandar (beta) leadership of the performance of have value of 0,06, personality of the performance of have value of 0,48, age of the performance of get value 0.01 and education on performance have value of 0.09 and from the testing obtained value pvalue very small (< 0,001). the incentives on performance in city bina marga areas tegal. This is evidenced of the value of the regression coefficient terstandar (beta) motivation on performance have value of 0,36 and from the testing obtained value pvalue very small (& lt; 0,001). Is the between leadership, personality, age and education on performance through motivation in city bina marga areas tegal. This is evidenced value the regression coefficient terstandar (beta) leadership, personality, of and education on performance through motivation have value of. 30 and from the testing obtained value p-value very small (< 0,001). There are relations between leadership, personality, age and education through motivation to performance in the region tegal bina marga.This is evidenced value regression coefficient terstandar (beta) leadership, personality, age) and education on performance through motivation for value of 0,30 and more tender testing obtained value p-value very small (< 0,001). Keywords: leadership, personality, age, education, motivation and performance.

scholarly journals ANALISIS FAKTOR-FAKTOR YANG MEMPENGARUHI MINAT BELI KONSUMEN PADA SITUS JUAL-BELI ONLINE

2020 ◽  
Vol 6 (2) ◽  
pp. 179
Author(s):  
I Gede Andri Setiawan ◽  
Sukardi Sukardi
Keyword(s):  
Data Analysis ◽  
Risk Perception ◽  
Linear Regression ◽  
Multiple Linear Regression ◽  
Significant Influence ◽  
Regression Coefficient ◽  
Regression Coefficients ◽  
Test Results ◽  
Price Perception ◽  
Usefulness Perception

The purpose of this study were to analyze (1) the influence of trust of the purchase interest of online shop consumers, (2) the influence of risk perception of the  purchase  interest  of  online  shop  consumers,  (3)  the  influence  of  use fulness perception of the purchase interest of online shop consumers, (4) the influence of price perception of the purchase interest of online shop consumers. The populations in this research are  student of Ahmad Dahlah University Yogyakarta  Campus  One.  The  method  sample  is  purposif  sampling  with  the number  of  sample  as  much  as  100  respondents  from  three  Faculty  that  is, Economy  Faculty,  Psycology  Faculty,  and  FTDI.  Collecting  data  using questionnaires  that  was  done  validity  test  and  realibity  test.  This  research  was done by using multiple linear regression data analysis. The  results  of  this  study  indicate  that  (1)  the  trust  is  not  significant influence on consumer purchase interest of the sites online shop, this is evidenced by the significant value  is bigger  than the probability alpha value of 0.05 (0,206 > 0.05), and the regression coefficient has a  positive value of 0.062; (2)  the  trust is not significant influence on consumer purchase interest of the sites online shop, this  is  evidenced  by  the  significant  value  is  bigger  than  the  probability  alpha value of 0.05 (0, 0,234 > 0.05), and the regression coefficient has a positive value of  0.057;  (3)  the  usefulness  perception  is  significant  influence  on  consumer purchase interest of the sites online shop, this is evidenced by the significant value is  smaller  than  the  probability  alpha  value  of  0.05  (0.000  <  0.05),  and  the regression coefficients has a positive value of 0.456; (4) the price perception is not significant  influence on consumer purchase interest of the sites online shop, this  is  evidenced  by  the  significant  value  is  bigger  than  the  probability  alpha value of 0.05 (0.066 > 0.05), and the regression coefficient has a  positive value of  0.111;  and  (5)  trust,  risk  perception,  usefulness  perception  and  price perception  together  have  an  effect  on  consumer  purchase  interest  of  the  sites online  shop,  this  is  evidenced  by  the  significant  value  is  smaller  than  the probability alpha value of 0.05 (0.000 < 0.05). The R2 test results in this study was obtained  R2 value  of  0.413.  This  shows  that  purchase  interest  is  influenced  by trust, risk perception, use perception, and price  perception by 41.3%, while the remaining 58.7% is influenced by other factors not observed by the researcher or considered fixed.

When love becomes hate: how different consumer-brand relationships interact with crises to influence consumers' reactions

2020 ◽  
Vol 25 (3) ◽  
pp. 357-375 ◽  
Author(s):  
Liang Ma
Keyword(s):  
Public Relations ◽  
Crisis Communication ◽  
Behavioral Intentions ◽  
Structural Equation ◽  
Equation Modeling ◽  
Future Research ◽  
Brand Relationships ◽  
Content Type ◽  
Different Types ◽  
Buffering Effects

PurposeA well-accepted proposition in the literature of corporate strategic communication and public relations is that consumer-brand relationships (CBRs) affect corporate crisis communication. However, it is inconclusive whether CBRs protect or work against brands, because both buffering effects and love-becomes-hate effects have been found. This study attempts to explain and bridge the seemingly inconsistent findings by clarifying the effects of different types of CBRs in different brand transgressions.Design/methodology/approachRe-conceptualizing CBRs into non-identifying relationships and identifying relationships, this study examined the possible interaction effects of CBRs and crises on consumers' attitudes and emotions, which then influence their behavioral intentions. A three-step multi-group structural equation modeling (SEM) was used to analyze the data collected from an online experiment with nearly 900 consumers of two brands.FindingsAlthough non-identifying relationships offer buffering effects, identifying relationships primarily offer love-becomes-hate effects by intensifying negative emotions such as anger and disappointment, which in turn affect consumers' behavioral intentions. Such patterns hold regardless of whether a crisis directly threatens the core meaning of the brand.Originality/valueThis study clarifies the effects of different types of CBRs in crises and shows that deep psychological connections (i.e. identifying relationships) offer love-becomes-hate effects. It suggests that one promising future research direction for crisis communication and public relations scholars is to examine how to mitigate such love-becomes-hate effects so that brands can keep their loyal consumers.

Sign in / Sign up

Export Citation Format

Share Document