scholarly journals Principles of Model Specification in ANOVA Designs

2021 ◽  
Author(s):  
Jeffrey Rouder ◽  
Martin Schnuerch ◽  
Julia M. Haaf ◽  
Richard Donald Morey
Keyword(s):  
Experimental Psychology ◽  
Bayes Factor ◽  
Mixed Model ◽  
Factor Model ◽  
Source Code ◽  
Model Specification ◽  
Behavioral Sciences ◽  
Behavioral Experiments ◽  
Model Comparisons ◽  
Main Thesis

ANOVA---the workhorse of experimental psychology--seems well understood in that behavioral sciences have agreed-upon contrasts and reporting conventions. Yet, we argue this consensus hides considerable flaws in common ANOVA procedures, and these flaws become especially salient in the within-subject and mixed-model cases. The main thesis is that these flaws are in model specification. The specifications underlying common use are deficient from a substantive perspective, that is, they do not match reality in behavioral experiments. The problem, in particular, is that specifications rely on coincidental rather than robust statements about reality. We provide specifications that avoid making arguments based on coincidences, and note these Bayes factor model comparisons among these specifications are already convenient in the BayesFactor package. Finally, we argue that model specification necessarily and critically reflects substantive concerns, and, consequently, is ultimately the responsibility of substantive researchers. Source code for this project is at github/PerceptionAndCognitionLab/stat_aov2

scholarly journals Bayes Factor Model Comparisons Across Parameter Values for Mixed Models

2021 ◽  
Author(s):  
Maximilian Linde ◽  
Don van Ravenzwaaij
Keyword(s):  
Repeated Measures ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Simulated Data ◽  
Experimental Manipulation ◽  
Data Set ◽  
Model Comparisons ◽  
Multiple Data ◽  
Multiple Data Sets

Nested data structures, in which conditions include multiple trials, are often analyzed using repeated-measures analysis of variance or mixed effects models. Typically, researchers are interested in determining whether there is an effect of the experimental manipulation. Unfortunately, these kinds of analyses have different appropriate specifications for the null and alternative models, and a discussion on which is to be preferred and when is sorely lacking. van Doorn et al. (2021) performed three types of Bayes factor model comparisons on a simulated data set in order to examine which model comparison is most suitable for quantifying evidence for or against the presence of an effect of the experimental manipulation. Here we extend their results by simulating multiple data sets for various scenarios and by using different prior specifications. We demonstrate how three different Bayes factor model comparison types behave under changes in different parameters, and we make concrete recommendations on which model comparison is most appropriate for different scenarios.

scholarly journals Bayes Factor Model Comparisons Across Parameter Values for Mixed Models

2021 ◽  
Author(s):  
Maximilian Linde ◽  
Don van Ravenzwaaij
Keyword(s):  
Repeated Measures ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Simulated Data ◽  
Experimental Manipulation ◽  
Data Set ◽  
Model Comparisons ◽  
Multiple Data ◽  
Multiple Data Sets

AbstractNested data structures, in which conditions include multiple trials and are fully crossed with participants, are often analyzed using repeated-measures analysis of variance or mixed-effects models. Typically, researchers are interested in determining whether there is an effect of the experimental manipulation. These kinds of analyses have different appropriate specifications for the null and alternative models, and a discussion on which is to be preferred and when is sorely lacking. van Doorn et al. (2021) performed three types of Bayes factor model comparisons on a simulated data set in order to examine which model comparison is most suitable for quantifying evidence for or against the presence of an effect of the experimental manipulation. Here, we extend their results by simulating multiple data sets for various scenarios and by using different prior specifications. We demonstrate how three different Bayes factor model comparison types behave under changes in different parameters, and we make concrete recommendations on which model comparison is most appropriate for different scenarios.

The expressivity dimension of speech is the basis of the expression dimension. Evidence from behavioural and neuroimaging studies

2020 ◽  
Author(s):  
Isabelle Hesling
Keyword(s):  
Language Acquisition ◽  
Experimental Psychology ◽  
Right Hemisphere ◽  
Word List ◽  
Behavioral Experiments ◽  
Brain Areas ◽  
Language Communication ◽  
Neuroimaging Data ◽  
List Processing ◽  
The Right

The modalities of communication are the sum of the expression dimension (linguistics) and the expressivity dimension (prosody), both being equally important in language communication. The expressivity dimension which comes first in the act of speech, is the basis on which phonemes, syllables, words, grammar and morphosyntax, i.e., the expression dimension of speech is superimposed. We will review evidence (1) revealing the importance of prosody in language acquisition and (2) showing that prosody triggers the involvement of specific brain areas dedicated to sentences and word-list processing. To support the first point, we will not only rely on experimental psychology studies conducted in newborns and young children but also on neuroimaging studies that have helped to validate these behavioral experiments. Then, neuroimaging data on adults will allow for concluding that the expressivity dimension of speech modulates both the right hemisphere prosodic areas and the left hemisphere network in charge of the expression dimension

scholarly journals Identification of genetic loci affecting body mass index through interaction with multiple environmental factors using structured linear mixed model

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Hae-Un Jung ◽  
Won Jun Lee ◽  
Tae-Woong Ha ◽  
Ji-One Kang ◽  
Jihye Kim ◽  
...  
Keyword(s):  
Body Mass Index ◽  
Environmental Factors ◽  
Body Mass ◽  
Bayes Factor ◽  
Mixed Model ◽  
Linear Mixed Model ◽  
Interaction Analysis ◽  
Carbohydrate Intake ◽  
Calorie Intake ◽  
Environment Interaction

AbstractMultiple environmental factors could interact with a single genetic factor to affect disease phenotypes. We used Struct-LMM to identify genetic variants that interacted with environmental factors related to body mass index (BMI) using data from the Korea Association Resource. The following factors were investigated: alcohol consumption, education, physical activity metabolic equivalent of task (PAMET), income, total calorie intake, protein intake, carbohydrate intake, and smoking status. Initial analysis identified 7 potential single nucleotide polymorphisms (SNPs) that interacted with the environmental factors (P value < 5.00 × 10−6). Of the 8 environmental factors, PAMET score was excluded for further analysis since it had an average Bayes Factor (BF) value < 1 (BF = 0.88). Interaction analysis using 7 environmental factors identified 11 SNPs (P value < 5.00 × 10−6). Of these, rs2391331 had the most significant interaction (P value = 7.27 × 10−9) and was located within the intron of EFNB2 (Chr 13). In addition, the gene-based genome-wide association study verified EFNB2 gene significantly interacting with 7 environmental factors (P value = 5.03 × 10−10). BF analysis indicated that most environmental factors, except carbohydrate intake, contributed to the interaction of rs2391331 on BMI. Although the replication of the results in other cohorts is warranted, these findings proved the usefulness of Struct-LMM to identify the gene–environment interaction affecting disease.

scholarly journals The truth revisited: Bayesian analysis of individual differences in the truth effect

2020 ◽  
Author(s):  
Martin Schnuerch ◽  
Lena Nadarevic ◽  
Jeffrey Rouder
Keyword(s):  
Individual Differences ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Cognitive Task ◽  
Data Sets ◽  
Individual Level ◽  
Truth Effect ◽  
Negative Truth ◽  
Positive Truth

The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite – they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine 8 truth-effect data sets where individual-level data are curated. These sets are composed of 1,105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in 5 of the 8 data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements' validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

scholarly journals The truth revisited: Bayesian analysis of individual differences in the truth effect

2020 ◽  
Author(s):  
Martin Schnuerch ◽  
Lena Nadarevic ◽  
Jeffrey N. Rouder
Keyword(s):  
Individual Differences ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Cognitive Task ◽  
Data Sets ◽  
Individual Level ◽  
Truth Effect ◽  
Negative Truth ◽  
Positive Truth

Abstract The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper, we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite—they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine eight truth-effect data sets where individual-level data are curated. These sets are composed of 1105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in five of the eight data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements’ validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

The Evaluation of Bivariate Mixed Models in Meta-analyses of Diagnostic Accuracy Studies with SAS, Stata and R

2018 ◽  
Vol 57 (03) ◽  
pp. 111-119 ◽  
Author(s):  
Felicitas Vogelgesang ◽  
Marc Dewey ◽  
Peter Schlattmann
Keyword(s):  
Diagnostic Accuracy ◽  
Sensitivity And Specificity ◽  
Mixed Models ◽  
Mixed Model ◽  
Mean Squared Error ◽  
Random Effect ◽  
Model Specification ◽  
Bivariate Model ◽  
Statistical Point ◽  
Meta Analyses

Summary Background: Meta-analyses require a thoroughly planned procedure to obtain unbiased overall estimates. From a statistical point of view not only model selection but also model implementation in the software affects the results. Objectives: The present simulation study investigates the accuracy of different implementations of general and generalized bivariate mixed models in SAS (using proc mixed, proc glimmix and proc nlmixed), Stata (using gllamm, xtmelogit and midas) and R (using reitsma from package mada and glmer from package lme4). Both models incorporate the relationship between sensitivity and specificity – the two outcomes of interest in meta-analyses of diagnostic accuracy studies – utilizing random effects. Methods: Model performance is compared in nine meta-analytic scenarios reflecting the combination of three sizes for meta-analyses (89, 30 and 10 studies) with three pairs of sensitivity/specificity values (97%/87%; 85%/75%; 90%/93%). Results: The evaluation of accuracy in terms of bias, standard error and mean squared error reveals that all implementations of the generalized bivariate model calculate sensitivity and specificity estimates with deviations less than two percentage points. proc mixed which together with reitsma implements the general bivariate mixed model proposed by Reitsma rather shows convergence problems. The random effect parameters are in general underestimated. Conclusions: This study shows that flexibility and simplicity of model specification together with convergence robustness should influence implementation recommendations, as the accuracy in terms of bias was acceptable in all implementations using the generalized approach.

scholarly journals Developing Constraint in Bayesian Mixed Models

2017 ◽  
Author(s):  
Julia M. Haaf ◽  
Jeffrey Rouder
Keyword(s):  
Mixed Models ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Data Sets ◽  
Order Restrictions ◽  
Nested Models ◽  
Order Constraints ◽  
Order Restricted ◽  
Psychological Tasks

Model comparison in Bayesian mixed models is becoming popular in psychological science. Here we develop a set of nested models that account for order restrictions across individuals in psychological tasks. An order-restricted model addresses the question 'Does Everybody', as in, 'Does everybody show the usual Stroop effect', or ‘Does everybody respond more quickly to intense noises than subtle ones.’ The crux of the modeling is the instantiation of 10s or 100s of order restrictions simultaneously, one for each participant. To our knowledge, the problem is intractable in frequentist contexts but relatively straightforward in Bayesian ones. We develop a Bayes factor model-comparison strategy using Zellner and colleagues’ default g-priors appropriate for assessing whether effects obey equality and order restrictions. We apply the methodology to seven data sets from Stroop, Simon, and Eriksen interference tasks. Not too surprisingly, we find that everybody Stroops—that is, for all people congruent colors are truly named more quickly than incongruent ones. But, perhaps surprisingly, we find these order constraints are violated for some people in the Simon task, that is, for these people spatially incongruent responses occur truly more quickly than congruent ones! Implications of the modeling and conjectures about the task-related differences are discussed.This paper was written in R-Markdown with code for data analysis integrated into the text. The Markdown script isopen and freely available at https://github.com/PerceptionAndCognitionLab/ctx-indiff. The data are also open and freely available at https://github.com/PerceptionCognitionLab/data0/tree/master/contexteffects.

scholarly journals ASSESSING LEARNING OUTCOMES: AN ANALYSIS OF THE GOALS-2 INSTRUMENT

2015 ◽  
Vol 14 (2) ◽  
pp. 93-116
Author(s):  
ANELISE GUIMARAES SABBAG ◽  
ANDREW ZIEFFLER
Keyword(s):  
Undergraduate Students ◽  
Mixed Model ◽  
Factor Model ◽  
Statistics Education ◽  
The United States ◽  
Measurement Models ◽  
Test Information ◽  
Research Journal ◽  
Graded Response ◽  
Two Parameter

The test instrument GOALS-2 was designed primarily to evaluate the effectiveness of the CATALST curriculum. The purpose of this study was to perform a psychometric analysis of this instrument. Undergraduate students from six universities in the United States (n=289) were administered the instrument. Three measurement models were fit and compared: the two-parameter logistic model, the mixed model (comprised of both the two-parameter logistic and the graded-response model), and the bi-factor model. The mixed model was found to most appropriately model students’ responses. The results suggested the revision of some items and the addition of more discriminating items to improve total test information. First published November 2015 at Statistics Education Research Journal Archives

Sign in / Sign up

Export Citation Format

Share Document