The Position of Distractors in Multiple-Choice Test Items: The Strongest Precede the Weakest

Middle bias has been reported for responses to multiple-choice test items used in educational assessment. It has been claimed that this response bias probably occurs because test developers tend to place correct responses among middle options, tests thus presenting a middle-biased distribution of answer keys. However, this response bias could be driven by strong distractors being more frequently located among middle options. In this study, the frequency of responses to a Chilean national examination used to rank students wanting to access higher education was used to categorize distractors based on attractiveness level. The distribution of different distractor types (best distractor, non-functioning distractors…) was analyzed across 110 tests of 80 five-option items administered to assess several disciplines in five consecutive years. Results showed that the strongest distractors were more frequently found among middle options, most commonly at option C. In contrast, the weakest distractors were more frequently found at the last option (E). This pattern did not substantially vary across disciplines or years. Supplementary analyses revealed that a similar position bias for distractors could be observed in tests administered in countries other than Chile. Thus, the location of different types of distractors might provide an alternative explanation for the middle bias reported in literature for tests’ responses. Implications for test developers, test takers, and researchers in the field are discussed.

Unifying Online and Counterfactual Learning to Rank: A Novel Counterfactual Estimator that Effectively Utilizes Online Interventions (Extended Abstract)

State-of-the-art Learning to Rank (LTR) methods for optimizing ranking systems based on user interactions are divided into online approaches – that learn by direct interaction – and counterfactual approaches – that learn from historical interactions. We propose a novel intervention-aware estimator to bridge this online/counterfactual division. The estimator corrects for the effect of position bias, trust bias, and item-selection bias by using corrections based on the behavior of the logging policy and on online interventions: changes to the logging policy made during the gathering of click data. Our experimental results show that, unlike existing counterfactual LTR methods, the intervention-aware estimator can greatly benefit from online interventions. To the best of our knowledge, this is the first method that is shown to be highly effective in both online and counterfactual scenarios.

Systematic biases in the on-board navigation solution of a CubeSat GNSS payload board

<p>In December 2018 and April 2019, two 3-unit cube satellites of the company Astrocast were launched into orbit. Both satellites are equipped with our low-cost single-frequency multi-GNSS payload board, which provides almost continuous on-board receiver solutions containing the position from GNSS code observations and the velocity from Doppler measurements. We make use of these independent observation types (positions and velocities) to identify and analyse systematic biases in the receiver solution. Therefore, we estimate the parameters of a dynamic orbit model using three different approaches: fitting the orbit model (1) to the positions only, (2) to the velocities only and (3) to both, positions and velocities.</p><p>After removing outliers, the position residuals from the position-only approach are at a level of about 5 m, the velocity residuals from the velocity-only approach at about 15 cm/s. When computing the positions with the velocity-only approach, however, the residuals are much larger and show a once-per revolution periodicity with amplitudes of up to 40 m. Besides that, we identify two offsets in the residuals which are independent of the observation type: a radial position bias of -3 m and an along-track velocity bias of -1.2 cm/s. Additionally, we observe two offsets which are dependent on the observation type: an along-track offset of 13 m in the position residuals when using the velocity-only approach and a radial offset of 1.3 cm/s in radial velocities when using the position-only approach.</p><p>The periodicity in radial and along-track direction is related to the orbit eccentricity and may be due to a general deficiency, when using velocities to estimate geometric orbit parameters. When comparing the orbits from the position-only and the velocity-only approach, we find an offset in the right ascension of the ascending node, which corresponds to a maximum cross-track position difference of 40 m at the equator. We show that this effect is caused by a periodic bias in the velocity solutions with a maximum at the poles. A possible cause for such a periodicity in the velocity solutions may be dynamic effects in the receiver tracking loops related to the LEO satellite velocity relative to the GNSS constellation, which can vary strongly within one revolution.</p><p>Our results show that both, the radial position offset and the along-track velocity offset are dependent on the altitude of the satellite and are likely to be caused by ionospheric refraction. The explanation for the along-track position offset and the along-track velocity offset, however, is not that obvious. We found that these two offsets are geometrically related and, thus, must have the same physical cause. Based on the combined position-and-velocity approach we demonstrate that they originate from a velocity bias rather than from a position bias. To explain the physical cause of such a radial velocity offset, we will study the ionospheric effects on GNSS code and Doppler measurements in more detail, where we use a 3D-ionosphere model and take also the altitude of the two satellites into account.</p>

Outer Cutoff Value for the Box-Counting Method for Fractal Analysis of the Nucleus Using Kirsch Edge Detection

<b><i>Objective:</i></b> The complexity of chromatin (i.e., irregular geometry and distribution) is one of the important factors considered in the cytological diagnosis of cancer. Fractal analysis with Kirsch edge detection is a known technique to detect irregular geometry and distribution in an image. We examined the outer cutoff value for the box-counting (BC) method for fractal analysis of the complexity of chromatin using Kirsch edge detection. <b><i>Materials:</i></b> The following images were used for the analysis: (1) image of the nucleus for Kirsch edge detection measuring 97 × 122 pix (10.7 × 13.4 μm) with a Feret diameter of chromatin mesh (<i>n</i> = 50) measuring 17.3 ± 1.8 pix (1.9 ± 0.5 μm) and chromatin network distance (<i>n</i> = 50) measuring 4.4 ± 1.6 pix (0.49 ± 0.18 μm), and (2) sample images for Kirsch edge detection with varying diameters (10.4, 15.9, and 18.1 μm) and network width of 0.4 μm. <b><i>Methods:</i></b> Three types of bias that can affect the outcomes of fractal analysis in cytological diagnosis were defined. (1) Nuclear position bias: images of 9 different positions generated by shifting the original position of the nucleus in the middle of a 256 × 256 pix (28.1 μm) square frame in 8 compass directions. (2) Nuclear rotation bias: images of 8 different rotations obtained by rotating the original position of the nucleus in 45° increments (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). (3) Nuclear size bias: images of varying size (diameter: 190 pix [10.4 μm], 290 pix [15.9 μm], and 330 pix [18.1 μm]) with the same mesh pattern (network width: 8 pix [0.4 μm]) within a 512 × 512 pix square. Different outer cutoff values for the BC method (256, 128, 64, 32, 16, and 8 pix) were applied for each bias to assess the fractal dimension and to compare the coefficient of variation (CV). <b><i>Results:</i></b> The BC method with the outer cutoff value of 32 pix resulted in the least variation of fractal dimension. Specifically, with the cutoff value of 32 pix, the CV of nuclear position bias, nuclear rotation bias, and nuclear size bias were &#x3c;1% (0.1, 0.4, and 0.3%, respectively), with no significant difference between the position and rotation bias (<i>p</i> = 0.19). Our study suggests that the BC method with the outer cutoff value of 32 pix is suitable for the analysis of the complexity of chromatin with chromatin mesh.

Misperceptions of income distributions: cross-country evidence from a randomized survey experiment

Can imperfect information, as revealed in individual misperceptions about income distributions, explain the demand for redistribution? I conduct a representative survey experiment in Brazil, France, Germany, Russia, Spain and the USA, providing a personalized information treatment on income distribution to a randomly chosen subsample. Most respondents misperceive their own position in the income distribution. These biases differ notably by country and the true income position. Correcting misperceptions slightly shifts the demand towards less redistribution in Germany and Russia. This shift appears to be driven by respondents with a negative position bias. The lack of significant treatment effects in other countries may result from different individual reactions that cancel each other out. Thus, the existence of systematic misperceptions underscores their importance for understanding preferences for redistribution.