Probabilistic and Systematic Coverage of Consecutive Test-Method Pairs for Detecting Order-Dependent Flaky Tests

AbstractSoftware developers frequently check their code changes by running a set of tests against their code. Tests that can nondeterministically pass or fail when run on the same code version are called flaky tests. These tests are a major problem because they can mislead developers to debug their recent code changes when the failures are unrelated to these changes. One prominent category of flaky tests is order-dependent (OD) tests, which can deterministically pass or fail depending on the order in which the set of tests are run. By detecting OD tests in advance, developers can fix these tests before they change their code. Due to the high cost required to explore all possible orders (n! permutations for n tests), prior work has developed tools that randomize orders to detect OD tests. Experiments have shown that randomization can detect many OD tests, and that most OD tests depend on just one other test to fail. However, there was no analysis of the probability that randomized orders detect OD tests. In this paper, we present the first such analysis and also present a simple change for sampling random test orders to increase the probability. We finally present a novel algorithm to systematically explore all consecutive pairs of tests, guaranteeing to detect all OD tests that depend on one other test, while running substantially fewer orders and tests than simply running all test pairs.

Download Full-text

Equitable Allocations of Indivisible Goods

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/40 ◽

2019 ◽

Cited By ~ 2

Author(s):

Rupert Freeman ◽

Sujoy Sikdar ◽

Rohit Vaish ◽

Lirong Xia

Keyword(s):

Economic Efficiency ◽

Pareto Optimality ◽

Real World ◽

Fair Division ◽

Prior Work ◽

Indivisible Goods ◽

Preference Data ◽

Novel Algorithm

In fair division, equitability dictates that each participant receives the same level of utility. In this work, we study equitable allocations of indivisible goods among agents with additive valuations. While prior work has studied (approximate) equitability in isolation, we consider equitability in conjunction with other well-studied notions of fairness and economic efficiency. We show that the Leximin algorithm produces an allocation that satisfies equitability up to any good and Pareto optimality. We also give a novel algorithm that guarantees Pareto optimality and equitability up to one good in pseudopolynomial time. Our experiments on real-world preference data reveal that approximate envy-freeness, approximate equitability, and Pareto optimality can often be achieved simultaneously.

Download Full-text

Statistical Evaluation of the Blind Test Method for Water Quality Control

Water Science & Technology ◽

10.2166/wst.1983.0128 ◽

1983 ◽

Vol 15 (6-7) ◽

pp. 35-46 ◽

Cited By ~ 1

Author(s):

G Bousquet ◽

J Ouvrard ◽

S Rigal ◽

R Vilagines

Keyword(s):

Water Quality ◽

Statistical Evaluation ◽

Test Method ◽

Chemical Analyses ◽

Threshold Values ◽

Taste Threshold ◽

Water Quality Control ◽

Blind Test ◽

Water Origin ◽

Simple Change

Each of the three taste panels, consisting of ten persons, participated in 18 water-tasting sessions which included a duplicate estimation of 4 randomly distributed dilutions. The statistical analysis of the 1 440 scores obtained showed a good correlation (r = 0.87) in between scores of the first water tasting and the scores of the second one. Factorial analysis showed that the panelists as a whole find a flavour in the same samples and are able to arrange the dilutions in a proper sequence. This analysis also showed that the differences in the estimation of the intensity of flavour carried out in the absence of periodical calibrations lead to a significant interval between taste threshold values given by each panel. The study of score-dilution contingency tables displayed the sensitivity of each panelist as well as the homogeneity or the heterogeneity of his panel. A change in the water origin and a simple change of the taste panel room reduced temporarily the coherence of scores. The data showed that water tasting is a reliable and reproducible technique in which the observed discrepancies among taste panels are similar to differences found in chemical analyses of palatable substances among laboratories.

Download Full-text

Interoperability move by Microsoft to help software developers

First Break ◽

10.1046/j.1365-2397.1998.00016.x ◽

1998 ◽

Vol 16 (3) ◽

pp. v-vi

Keyword(s):

Software Developers

Download Full-text

The Role of Visual Distractors in the Simon Effect

Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) ◽

10.1027/1618-3169/a000382 ◽

2017 ◽

Vol 64 (6) ◽

pp. 387-397 ◽

Cited By ~ 1

Author(s):

Luisa Lugli ◽

Stefania D’Ascenzo ◽

Roberto Nicoletti ◽

Carlo Umiltà

Keyword(s):

Simon Effect ◽

Simon Task ◽

Automatic Generation ◽

Spatial Code ◽

Attentional Model ◽

Previous Evidence ◽

Spatial Stimulus ◽

Attentional Resources ◽

Code Changes

Abstract. The Simon effect lies on the automatic generation of a stimulus spatial code, which, however, is not relevant for performing the task. Results typically show faster performance when stimulus and response locations correspond, rather than when they do not. Considering reaction time distributions, two types of Simon effect have been individuated, which are thought to depend on different mechanisms: visuomotor activation versus cognitive translation of spatial codes. The present study aimed to investigate whether the presence of a distractor, which affects the allocation of attentional resources and, thus, the time needed to generate the spatial code, changes the nature of the Simon effect. In four experiments, we manipulated the presence and the characteristics of the distractor. Findings extend previous evidence regarding the distinction between visuomotor activation and cognitive translation of spatial stimulus codes in a Simon task. They are discussed with reference to the attentional model of the Simon effect.

Download Full-text