What Happens When Data Communication Fails? - A Simulation Study

2011 ◽  
Author(s):  
Sehchang Hah ◽  
Ben Willems ◽  
Kenneth Schulz
Keyword(s):  
Simulation Study ◽  
Data Communication

An Alternative to Cohen's κ

2006 ◽  
Vol 11 (1) ◽  
pp. 12-24 ◽  
Author(s):  
Alexander von Eye
Keyword(s):  
Simulation Study ◽  
Null Hypothesis ◽  
Categorical Variables ◽  
Alternative Measure ◽  
Rater Agreement ◽  
Verbal Processing ◽  
Heavy Tailed ◽  
Applicant Selection

At the level of manifest categorical variables, a large number of coefficients and models for the examination of rater agreement has been proposed and used. The most popular of these is Cohen's κ. In this article, a new coefficient, κ s , is proposed as an alternative measure of rater agreement. Both κ and κ s allow researchers to determine whether agreement in groups of two or more raters is significantly beyond chance. Stouffer's z is used to test the null hypothesis that κ s = 0. The coefficient κ s allows one, in addition to evaluating rater agreement in a fashion parallel to κ, to (1) examine subsets of cells in agreement tables, (2) examine cells that indicate disagreement, (3) consider alternative chance models, (4) take covariates into account, and (5) compare independent samples. Results from a simulation study are reported, which suggest that (a) the four measures of rater agreement, Cohen's κ, Brennan and Prediger's κ n , raw agreement, and κ s are sensitive to the same data characteristics when evaluating rater agreement and (b) both the z-statistic for Cohen's κ and Stouffer's z for κ s are unimodally and symmetrically distributed, but slightly heavy-tailed. Examples use data from verbal processing and applicant selection.

Comparison of Principal Component Solutions in Two Populations

Methodology ◽  
2016 ◽  
Vol 12 (1) ◽  
pp. 11-20 ◽  
Author(s):  
Gregor Sočan
Keyword(s):  
Simulation Study ◽  
Principal Components ◽  
Common Factor ◽  
Principal Component ◽  
Component Model ◽  
Bootstrap Procedure ◽  
Factor Loadings ◽  
Component Loadings ◽  
Principal Component Model ◽  
Two Populations

Abstract. When principal component solutions are compared across two groups, a question arises whether the extracted components have the same interpretation in both populations. The problem can be approached by testing null hypotheses stating that the congruence coefficients between pairs of vectors of component loadings are equal to 1. Chan, Leung, Chan, Ho, and Yung (1999) proposed a bootstrap procedure for testing the hypothesis of perfect congruence between vectors of common factor loadings. We demonstrate that the procedure by Chan et al. is both theoretically and empirically inadequate for the application on principal components. We propose a modification of their procedure, which constructs the resampling space according to the characteristics of the principal component model. The results of a simulation study show satisfactory empirical properties of the modified procedure.

Assessing Person Fit With the Information Matrix Test

Methodology ◽  
2015 ◽  
Vol 11 (1) ◽  
pp. 3-12 ◽  
Author(s):  
Jochen Ranger ◽  
Jörg-Tobias Kuhn
Keyword(s):  
Simulation Study ◽  
Type I Error ◽  
Information Matrix ◽  
Small Samples ◽  
Type I ◽  
Person Fit ◽  
Power Of The Test ◽  
Order Expansion ◽  
Trait Stability ◽  
Information Matrix Test

In this manuscript, a new approach to the analysis of person fit is presented that is based on the information matrix test of White (1982) . This test can be interpreted as a test of trait stability during the measurement situation. The test follows approximately a χ2-distribution. In small samples, the approximation can be improved by a higher-order expansion. The performance of the test is explored in a simulation study. This simulation study suggests that the test adheres to the nominal Type-I error rate well, although it tends to be conservative in very short scales. The power of the test is compared to the power of four alternative tests of person fit. This comparison corroborates that the power of the information matrix test is similar to the power of the alternative tests. Advantages and areas of application of the information matrix test are discussed.

Sign in / Sign up

Export Citation Format

Share Document