scholarly journals Bayes Factor Model Comparisons Across Parameter Values for Mixed Models

2021 ◽  
Author(s):  
Maximilian Linde ◽  
Don van Ravenzwaaij
Keyword(s):  
Repeated Measures ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Simulated Data ◽  
Experimental Manipulation ◽  
Data Set ◽  
Model Comparisons ◽  
Multiple Data ◽  
Multiple Data Sets

AbstractNested data structures, in which conditions include multiple trials and are fully crossed with participants, are often analyzed using repeated-measures analysis of variance or mixed-effects models. Typically, researchers are interested in determining whether there is an effect of the experimental manipulation. These kinds of analyses have different appropriate specifications for the null and alternative models, and a discussion on which is to be preferred and when is sorely lacking. van Doorn et al. (2021) performed three types of Bayes factor model comparisons on a simulated data set in order to examine which model comparison is most suitable for quantifying evidence for or against the presence of an effect of the experimental manipulation. Here, we extend their results by simulating multiple data sets for various scenarios and by using different prior specifications. We demonstrate how three different Bayes factor model comparison types behave under changes in different parameters, and we make concrete recommendations on which model comparison is most appropriate for different scenarios.

scholarly journals Bayes Factor Model Comparisons Across Parameter Values for Mixed Models

2021 ◽  
Author(s):  
Maximilian Linde ◽  
Don van Ravenzwaaij
Keyword(s):  
Repeated Measures ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Simulated Data ◽  
Experimental Manipulation ◽  
Data Set ◽  
Model Comparisons ◽  
Multiple Data ◽  
Multiple Data Sets

Nested data structures, in which conditions include multiple trials, are often analyzed using repeated-measures analysis of variance or mixed effects models. Typically, researchers are interested in determining whether there is an effect of the experimental manipulation. Unfortunately, these kinds of analyses have different appropriate specifications for the null and alternative models, and a discussion on which is to be preferred and when is sorely lacking. van Doorn et al. (2021) performed three types of Bayes factor model comparisons on a simulated data set in order to examine which model comparison is most suitable for quantifying evidence for or against the presence of an effect of the experimental manipulation. Here we extend their results by simulating multiple data sets for various scenarios and by using different prior specifications. We demonstrate how three different Bayes factor model comparison types behave under changes in different parameters, and we make concrete recommendations on which model comparison is most appropriate for different scenarios.

scholarly journals The truth revisited: Bayesian analysis of individual differences in the truth effect

2020 ◽  
Author(s):  
Martin Schnuerch ◽  
Lena Nadarevic ◽  
Jeffrey Rouder
Keyword(s):  
Individual Differences ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Cognitive Task ◽  
Data Sets ◽  
Individual Level ◽  
Truth Effect ◽  
Negative Truth ◽  
Positive Truth

The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite – they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine 8 truth-effect data sets where individual-level data are curated. These sets are composed of 1,105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in 5 of the 8 data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements' validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

Multiple Data Set ILI for Mechanical Damage Assessment

2012 ◽  
Author(s):  
Chris Goller ◽  
James Simek ◽  
Jed Ludlow
Keyword(s):  
Damage Assessment ◽  
Public Awareness ◽  
Mechanical Damage ◽  
Data Sets ◽  
Data Set ◽  
Multiple Data ◽  
Pipe Joints ◽  
Multiple Data Sets ◽  
Multiple Field ◽  
Flux Leakage

The purpose of this paper is to present a non-traditional pipeline mechanical damage ranking system using multiple-data-set in-line inspection (ILI) tools. Mechanical damage continues to be a major factor in reportable incidents for hazardous liquid and gas pipelines. While several ongoing programs seek to limit damage incidents through public awareness, encroachment monitoring, and one-call systems, others have focused efforts on the quantification of mechanical damage severity through modeling, the use of ILI tools, and subsequent feature assessment at locations selected for excavation. Current generation ILI tools capable of acquiring multiple-data-sets in a single survey may provide an improved assessment of the severity of damaged zones using methods developed in earlier research programs as well as currently reported information. For magnetic flux leakage (MFL) type tools, using multiple field levels, varied field directions, and high accuracy deformation sensors enables detection and provides the data necessary for enhanced severity assessments. This paper will provide a review of multiple-data-set ILI results from several pipe joints with simulated mechanical damage locations created mimicing right-of-way encroachment events in addition to field results from ILI surveys using multiple-data-set tools.

scholarly journals The truth revisited: Bayesian analysis of individual differences in the truth effect

2020 ◽  
Author(s):  
Martin Schnuerch ◽  
Lena Nadarevic ◽  
Jeffrey N. Rouder
Keyword(s):  
Individual Differences ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Cognitive Task ◽  
Data Sets ◽  
Individual Level ◽  
Truth Effect ◽  
Negative Truth ◽  
Positive Truth

Abstract The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper, we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite—they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine eight truth-effect data sets where individual-level data are curated. These sets are composed of 1105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in five of the eight data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements’ validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

scholarly journals Estimating observation and model error variances using multiple data sets

2018 ◽  
Vol 11 (7) ◽  
pp. 4239-4260 ◽  
Author(s):  
Richard Anthes ◽  
Therese Rieckh
Keyword(s):  
Error Variance ◽  
Specific Humidity ◽  
Data Sets ◽  
Data Set ◽  
Multiple Data ◽  
Gfs Model ◽  
Multiple Data Sets ◽  
Using Data ◽  
The Tropics ◽  
Estimated Error

Abstract. In this paper we show how multiple data sets, including observations and models, can be combined using the “three-cornered hat” (3CH) method to estimate vertical profiles of the errors of each system. Using data from 2007, we estimate the error variances of radio occultation (RO), radiosondes, ERA-Interim, and Global Forecast System (GFS) model data sets at four radiosonde locations in the tropics and subtropics. A key assumption is the neglect of error covariances among the different data sets, and we examine the consequences of this assumption on the resulting error estimates. Our results show that different combinations of the four data sets yield similar relative and specific humidity, temperature, and refractivity error variance profiles at the four stations, and these estimates are consistent with previous estimates where available. These results thus indicate that the correlations of the errors among all data sets are small and the 3CH method yields realistic error variance profiles. The estimated error variances of the ERA-Interim data set are smallest, a reasonable result considering the excellent model and data assimilation system and assimilation of high-quality observations. For the four locations studied, RO has smaller error variances than radiosondes, in agreement with previous studies. Part of the larger error variance of the radiosondes is associated with representativeness differences because radiosondes are point measurements, while the other data sets represent horizontal averages over scales of ∼ 100 km.

scholarly journals Developing Constraint in Bayesian Mixed Models

2017 ◽  
Author(s):  
Julia M. Haaf ◽  
Jeffrey Rouder
Keyword(s):  
Mixed Models ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Data Sets ◽  
Order Restrictions ◽  
Nested Models ◽  
Order Constraints ◽  
Order Restricted ◽  
Psychological Tasks

Model comparison in Bayesian mixed models is becoming popular in psychological science. Here we develop a set of nested models that account for order restrictions across individuals in psychological tasks. An order-restricted model addresses the question 'Does Everybody', as in, 'Does everybody show the usual Stroop effect', or ‘Does everybody respond more quickly to intense noises than subtle ones.’ The crux of the modeling is the instantiation of 10s or 100s of order restrictions simultaneously, one for each participant. To our knowledge, the problem is intractable in frequentist contexts but relatively straightforward in Bayesian ones. We develop a Bayes factor model-comparison strategy using Zellner and colleagues’ default g-priors appropriate for assessing whether effects obey equality and order restrictions. We apply the methodology to seven data sets from Stroop, Simon, and Eriksen interference tasks. Not too surprisingly, we find that everybody Stroops—that is, for all people congruent colors are truly named more quickly than incongruent ones. But, perhaps surprisingly, we find these order constraints are violated for some people in the Simon task, that is, for these people spatially incongruent responses occur truly more quickly than congruent ones! Implications of the modeling and conjectures about the task-related differences are discussed.This paper was written in R-Markdown with code for data analysis integrated into the text. The Markdown script isopen and freely available at https://github.com/PerceptionAndCognitionLab/ctx-indiff. The data are also open and freely available at https://github.com/PerceptionCognitionLab/data0/tree/master/contexteffects.

scholarly journals Two-stage Linked Component Analysis for Joint Decomposition of Multiple Biologically Related Data Sets

2021 ◽  
Author(s):  
By Huan Chen ◽  
Brian Caffo ◽  
Genevieve Stein-O’Brien ◽  
Jinrui Liu ◽  
Ben Langmead ◽  
...  
Keyword(s):  
R Package ◽  
Component Analysis ◽  
Biological Data ◽  
Joint Analysis ◽  
Data Sets ◽  
Biological Processes ◽  
Two Stage ◽  
Data Set ◽  
Multiple Data ◽  
Multiple Data Sets

SummaryIntegrative analysis of multiple data sets has the potential of fully leveraging the vast amount of high throughput biological data being generated. In particular such analysis will be powerful in making inference from publicly available collections of genetic, transcriptomic and epigenetic data sets which are designed to study shared biological processes, but which vary in their target measurements, biological variation, unwanted noise, and batch variation. Thus, methods that enable the joint analysis of multiple data sets are needed to gain insights into shared biological processes that would otherwise be hidden by unwanted intra-data set variation. Here, we propose a method called two-stage linked component analysis (2s-LCA) to jointly decompose multiple biologically related experimental data sets with biological and technological relationships that can be structured into the decomposition. The consistency of the proposed method is established and its empirical performance is evaluated via simulation studies. We apply 2s-LCA to jointly analyze four data sets focused on human brain development and identify meaningful patterns of gene expression in human neurogenesis that have shared structure across these data sets. The code to conduct 2s-LCA has been complied into an R package “PJD”, which is available at https://github.com/CHuanSite/PJD.

scholarly journals Beyond Overall Effects: A Bayesian Approach to Finding Constraints Across A Collection Of Studies In Meta-Analysis

2017 ◽  
Author(s):  
Jeffrey Rouder ◽  
Julia M. Haaf ◽  
Clintin Stober ◽  
Joseph Hilgard
Keyword(s):  
Video Games ◽  
Bayesian Approach ◽  
Model Comparison ◽  
Bayes Factor ◽  
Factor Model ◽  
Meta Analysis ◽  
Violent Video Games ◽  
Underlying Distribution ◽  
Means Testing ◽  
Meta Analyses

Most meta-analyses focus on meta-analytic means, testing whether they are significantly different from zero and how they depend on covariates. This mean is difficult to defend as a construct because the underlying distribution of studies reflects many factors such as how we choose to run experiments. We argue that the fundamental questions of meta-analysis should not be about the aggregated mean; instead, one should ask which relations are stable across all the studies. In a typical meta-analysis, there is a preferred or hypothesized direction (e.g., that violent video games increase, rather than decrease, agressive behavior). We ask whether all studies in a meta-analysis have true effects in a common direction. If so, this is an example of a stable relation across all the studies. We propose four models: (i) all studies are truly null; (ii) all studies share a single true nonzero effect; (iii) studies differ, but all true effects are in the same direction; and (iv) some study effects are truly positive while others are truly negative. We develop Bayes factor model comparison for these models and apply them to four extant meta-analyses to show their usefulness.

Nonlinear Model Identification From Multiple Data Sets Using an Orthogonal Forward Search Algorithm

2013 ◽  
Vol 8 (4) ◽  
Author(s):  
Ping Li ◽  
Hua-Liang Wei ◽  
Stephen A. Billings ◽  
Michael A. Balikhin ◽  
Richard Boynton
Keyword(s):  
Search Algorithm ◽  
Model Identification ◽  
Identification Problem ◽  
Data Sets ◽  
Nonlinear Dynamic Model ◽  
Data Set ◽  
Multiple Data ◽  
Orthogonal Search ◽  
Data Points ◽  
Multiple Data Sets

A basic assumption on the data used for nonlinear dynamic model identification is that the data points are continuously collected in chronological order. However, there are situations in practice where this assumption does not hold and we end up with an identification problem from multiple data sets. The problem is addressed in this paper and a new cross-validation-based orthogonal search algorithm for NARMAX model identification from multiple data sets is proposed. The algorithm aims at identifying a single model from multiple data sets so as to extend the applicability of the standard method in the cases, such as the data sets for identification are obtained from multiple tests or a series of experiments, or the data set is discontinuous because of missing data points. The proposed method can also be viewed as a way to improve the performance of the standard orthogonal search method for model identification by making full use of all the available data segments in hand. Simulated and real data are used in this paper to illustrate the operation and to demonstrate the effectiveness of the proposed method.

scholarly journals Significance levels of common frequencies extracted from multiple data sets

2020 ◽  
Vol 493 (1) ◽  
pp. 48-54
Author(s):  
Chris Koen
Keyword(s):  
Time Series ◽  
Time Base ◽  
Data Sets ◽  
Frequency Interval ◽  
Data Set ◽  
Frequency Spectra ◽  
Multiple Data ◽  
Significance Levels ◽  
System Data ◽  
Multiple Data Sets

ABSTRACT Large monitoring campaigns, particularly those using multiple filters, have produced replicated time series of observations for literally millions of stars. The search for periodicities in such replicated data can be facilitated by comparing the periodograms of the various time series. In particular, frequency spectra can be searched for common peaks. The sensitivity of this procedure to various parameters (e.g. the time base of the data, length of the frequency interval searched, number of replicate series, etc.) is explored. Two additional statistics that could sharpen results are also discussed: the closeness (in frequency) of peaks identified as common to all data sets, and the sum of the ranks of the peaks. Analytical expressions for the distributions of these two statistics are presented. The method is illustrated by showing that a ‘dubious’ periodicity in an 'Asteroid Terrestrial-impact Last Alert System' data set is highly significant.

Sign in / Sign up

Export Citation Format

Share Document