Hyperbolic trade-off: the importance of balancing trial and subject sample sizes in neuroimaging

AbstractThis work considers the use of classifiers in a downstream aggregation task estimating class proportions, such as estimating the percentage of reviews for a movie with positive sentiment. We derive the bias and variance of the class proportion estimator when taking classification error into account to determine how to best trade off different error types when tuning a classifier for these tasks. Additionally, we propose a method for constructing confidence intervals that correctly adjusts for classification error when estimating these statistics. We conduct experiments on four document classification tasks comparing our methods to prior approaches across classifier thresholds, sample sizes, and label distributions. Prior approaches have focused on providing the most accurate point estimate while this work focuses on the creation of correct confidence intervals that appropriately account for classifier error. Compared to the prior approaches, our methods provide lower error and more accurate confidence intervals.

Download Full-text

A Flow-Preserving Algorithm for the Time-Cost Trade-Off Problem

IIE Transactions ◽

10.1080/05695558208974589 ◽

1982 ◽

Vol 14 (2) ◽

pp. 109-113 ◽

Cited By ~ 4

Author(s):

Suleyman Tufekci

Keyword(s):

Time Cost ◽

Trade Off

Download Full-text

Is Red Really Romantic?

Social Psychology ◽

10.1027/1864-9335/a000296 ◽

2017 ◽

Vol 48 (3) ◽

pp. 174-183 ◽

Cited By ~ 9

Author(s):

Gabrielle K. Lehmann ◽

Robert J. Calin-Jageman

Keyword(s):

Opposite Direction ◽

Positive Control ◽

Weak Effect ◽

Sample Sizes ◽

Quality Controls

Abstract. Red has been reported to enhance attraction for women rating men ( Elliot et al., 2010 ) and men rating women ( Elliot & Niesta, 2008 ). We replicated one of these studies online and in-person. To ensure rigor, we obtained original materials, planned for informative sample sizes, pre-registered our study, used a positive control, and adopted quality controls. For men, we found a very weak effect in the predicted direction (d = 0.09, 95% CI [−0.17, 0.34], N = 242). For women, we found a very weak effect in the opposite direction (d = −0.09, 95% CI [−0.30, 0.12], N = 360). The original studies may have overestimated the red effect, our studies may be an underestimate, or there could be strong moderation of the effect of red on attraction.

Download Full-text

Dual Goals for Speed and Accuracy on the Same Performance Task

Journal of Personnel Psychology ◽

10.1027/1866-5888/a000063 ◽

2012 ◽

Vol 11 (3) ◽

pp. 118-126 ◽

Cited By ~ 3

Author(s):

Olive Emil Wetter ◽

Jürgen Wegge ◽

Klaus Jonas ◽

Klaus-Helmut Schmidt

Keyword(s):

Memory Scanning ◽

Performance Task ◽

Performance Tasks ◽

Trade Off ◽

Test Experiment ◽

Trade Offs ◽

New Finding ◽

Sternberg Paradigm ◽

Speed Accuracy ◽

Speed And Accuracy

In most work contexts, several performance goals coexist, and conflicts between them and trade-offs can occur. Our paper is the first to contrast a dual goal for speed and accuracy with a single goal for speed on the same task. The Sternberg paradigm (Experiment 1, n = 57) and the d2 test (Experiment 2, n = 19) were used as performance tasks. Speed measures and errors revealed in both experiments that dual as well as single goals increase performance by enhancing memory scanning. However, the single speed goal triggered a speed-accuracy trade-off, favoring speed over accuracy, whereas this was not the case with the dual goal. In difficult trials, dual goals slowed down scanning processes again so that errors could be prevented. This new finding is particularly relevant for security domains, where both aspects have to be managed simultaneously.

Download Full-text