The Impact of Aberrant Responses and Detection in Forced‐Choice Noncognitive Assessment

Two formats, traditional Likert scale single statements and two-alternative forced choice statements, were administered to samples from the United States and China to examine whether format influences response biases. Examination of response anchor usages and decision making processes from the two samples using an item response tree model yielded conclusions consistent with findings of prior studies--traditional Likert scale single statements exhibited cross-cultural response style differences: Chinese respondents engage in significantly less endpoint endorsement. Such response style differences can produce measurement nonequivalence across cultures. In contrast, a two-alternative forced choice format with Item Response Theory scoring yielded latent trait score estimates substantially correlated with estimates from other measures, demonstrating convergent validity, but with much smaller differences in trait variance across cultural groups. Thus, the two-alternative forced choice format appears to be an effective format for reducing the impact of cross-cultural response style bias. Implications for practice and extensions for further research are discussed.

Download Full-text

Discrimination of 2D wall textures by passive echolocation for different reflected-to-direct level difference configurations

PLoS ONE ◽

10.1371/journal.pone.0251397 ◽

2021 ◽

Vol 16 (5) ◽

pp. e0251397

Author(s):

Léopold Kritly ◽

Yannick Sluyts ◽

David Pelegrín-García ◽

Christ Glorieux ◽

Monika Rychtáriková

Keyword(s):

Forced Choice ◽

Forward Masking ◽

Test Results ◽

Listening Test ◽

Average Performance ◽

Flat Wall ◽

Direct Component ◽

Passive Listening ◽

Level Difference ◽

The Impact

In this work, we study people’s ability to discriminate between different 2D textures of walls by passive listening to a pre-recorded tongue click in an auralized echolocation scenario. In addition, the impact of artificially enhancing the early reflection magnitude by 6dB and of removing the direct component while equalizing the loudness was investigated. Listening test results for different textures, ranging from a flat wall to a staircase, were assessed using a 2 Alternative-Forced-Choice (2AFC) method, in which 14 sighted, untrained participants were indicating 2 equally perceived stimuli out of 3 presented stimuli. The average performance of the listening subjects to discriminate between different textures was found to be significantly higher for walls at 5m distance, without overlap between the reflected and direct sound, compared to the same walls at 0.8m distance. Enhancing the reflections as well as removing the direct sound were found to be beneficial to differentiate textures. This finding highlights the importance of forward masking in the discrimination process. The overall texture discriminability was found to be larger for the walls reflecting with a higher spectral coloration.

Download Full-text

Fit Indices for Measurement Invariance Tests in the Thurstonian IRT Model

Applied Psychological Measurement ◽

10.1177/0146621619893785 ◽

2019 ◽

Vol 44 (4) ◽

pp. 282-295

Author(s):

HyeSun Lee ◽

Weldon Z. Smith

Keyword(s):

Measurement Invariance ◽

Type I Error ◽

Error Rates ◽

Forced Choice ◽

Model Fit ◽

Future Research ◽

Type I ◽

Fit Indices ◽

Irt Model ◽

The Impact

This study examined whether cutoffs in fit indices suggested for traditional formats with maximum likelihood estimators can be utilized to assess model fit and to test measurement invariance when a multiple group confirmatory factor analysis was employed for the Thurstonian item response theory (IRT) model. Regarding the performance of the evaluation criteria, detection of measurement non-invariance and Type I error rates were examined. The impact of measurement non-invariance on estimated scores in the Thurstonian IRT model was also examined through accuracy and efficiency in score estimation. The fit indices used for the evaluation of model fit performed well. Among six cutoffs for changes in model fit indices, only ΔCFI > .01 and ΔNCI > .02 detected metric non-invariance when the medium magnitude of non-invariance occurred and none of the cutoffs performed well to detect scalar non-invariance. Based on the generated sampling distributions of fit index differences, this study suggested ΔCFI > .001 and ΔNCI > .004 for scalar non-invariance and ΔCFI > .007 for metric non-invariance. Considering Type I error rate control and detection rates of measurement non-invariance, ΔCFI was recommended for measurement non-invariance tests for forced-choice format data. Challenges in measurement non-invariance tests in the Thurstonian IRT model were discussed along with the direction for future research to enhance the utility of forced-choice formats in test development for cross-cultural and international settings.

Download Full-text

The Effect of Forced Choice on Choice

Journal of Marketing Research ◽

10.1509/jmkr.40.2.146.19229 ◽

2003 ◽

Vol 40 (2) ◽

pp. 146-160 ◽

Cited By ~ 317

Author(s):

Ravi Dhar ◽

Itamar Simonson

Keyword(s):

Consumer Preferences ◽

Forced Choice ◽

Attraction Effect ◽

Compromise Effect ◽

Implicit Assumption ◽

Industry Studies ◽

General Proposition ◽

The Impact ◽

Choice Option ◽

Practical Implications

Whereas most academic and industry studies of consumer preferences and decision making involve forced choice (i.e., participants are told to choose one of the presented product or service alternatives), buyers usually also have the option not to select any alternative. An implicit assumption in the experimental practice of forcing choice is that the no-choice option draws proportionately from the various available alternatives, such that the qualitative conclusions are unaffected. However, the authors propose that the no-choice option competes most directly with alternatives that buyers tend to select when they are uncertain about their preferences. Building on this general proposition, the authors show that the introduction of the no-choice option strengthens the attraction effect, weakens the compromise effect, and decreases the relative share of an option that is “average” on all dimensions. They also examine the mechanisms underlying the impact of having the option not to choose and the conditions under which the no-choice option is likely to affect relative option shares. The results are consistent with the notion that the no-choice option provides an alternative way of resolving difficult choices that is not available when subjects are forced to choose. The authors discuss the theoretical and practical implications of this research.

Download Full-text

Response outcomes gate the impact of expectations on perceptual decisions

10.1101/433409 ◽

2018 ◽

Cited By ~ 7

Author(s):

Ainhoa Hermoso-Mendizabal ◽

Alexandre Hyafil ◽

Pavel E. Rueda-Orozco ◽

Santiago Jaramillo ◽

David Robbe ◽

...

Keyword(s):

Correct Response ◽

Sensory Information ◽

Forced Choice ◽

Stimulus Sequence ◽

Stimulus Category ◽

Previous Response ◽

Transition Bias ◽

Serial Correlations ◽

Auditory Tasks ◽

The Impact

ABSTRACTPerceptual decisions are not only determined by current sensory information but are also influenced by expectations based on recent experiences. Can the impact of these expectations be flexibly modulated based on the outcome of previous decisions? We trained rats in several two-alternative forced choice auditory tasks, where the probability to repeat the previous stimulus category was varied in blocks of trials. All rats capitalized on the serial correlations of the stimulus sequence by consistently exploiting atransition bias:a tendency to repeat or alternate their previous response using an internal trial-by-trial estimate of the sequence repeating probability. Surprisingly, this bias was null in trials immediately following an error. The internal estimate however was not reset and it became effective again causing a bias after the next correct response. This ability to rapidly activate and inactivate the bias was captured by a non-linear generative model of rat behavior, whereby a reward-driven modulatory signal gated the use of the latent estimate of the environment statistics on the current decision. These results demonstrate that, based on the outcome of previous choices, rats flexibly modulate how expectations influence their current decisions.

Download Full-text

Children aged 7 – 9 prefer cuteness in baby faces, and femininity in women’s faces

10.31234/osf.io/nmg4h ◽

2020 ◽

Author(s):

Tamsin Saxton ◽

Thomas Victor Pollet ◽

Jenny Panagakis ◽

Emily Round ◽

Sam Brown ◽

...

Keyword(s):

Sex Hormones ◽

Forced Choice ◽

Reproductive Age ◽

Facial Features ◽

Age Group ◽

Control Task ◽

Functional Explanations ◽

Infant Faces ◽

The Impact ◽

Better Than

Infant facial features are typically perceived as ‘cute’, provoking caretaking behaviours. Previous research has examined the impact of adult sex hormones and events such as ovulation and menopause on baby cuteness perceptions, working on the basis that cuteness perception is most relevant to individuals during their reproductive lifespan. However, this overlooks the fact that globally, individuals of all ages, including children, provide notable proportions of infant care. Accordingly, we tested 65 children and 330 adults using a forced-choice paradigm to assess preferences for cute newborn and infant faces, and (as a control task) for femininity in women’s faces. Children and adults successfully identified the newborns that had been manipulated to appear cuter, although children’s performance was poorer than adults’ performance, and did not exceed chance in relation to the infant stimulus set. Children also chose the feminised over masculinised women’s faces as more attractive, although again their performance was poorer than adults’. Girls performed better than boys when assessing the newborn and woman stimuli, and women performed better than men when assessing the infant stimuli. There was no evidence that cuteness judgements differed depending upon exposure to infants (children with siblings aged 0 - 2; adults with a baby caregiving role), or depending upon being just younger or older than the average age of menopause. Children as well as grandparents provide notable portions of infant caretaking globally, and the functional explanations given of reactions to cuteness by adults of reproductive age, that it directs appropriate caregiving behaviour, could also apply to this extended age group of potential caregivers.

Download Full-text

Detecting DIF in Multidimensional Forced Choice Measures Using the Thurstonian Item Response Theory Model

Organizational Research Methods ◽

10.1177/1094428120959822 ◽

2020 ◽

pp. 109442812095982

Author(s):

Philseok Lee ◽

Seang-Hwane Joo ◽

Stephen Stark

Keyword(s):

Item Response Theory ◽

Item Response ◽

Test Construction ◽

Theory Model ◽

Forced Choice ◽

Practical Significance ◽

Large Sample Size ◽

Response Theory ◽

Study Results ◽

The Impact

Although modern item response theory (IRT) methods of test construction and scoring have overcome ipsativity problems historically associated with multidimensional forced choice (MFC) formats, there has been little research on MFC differential item functioning (DIF) detection, where item refers to a block, or group, of statements presented for an examinee’s consideration. This research investigated DIF detection with three-alternative MFC items based on the Thurstonian IRT (TIRT) model, using omnibus Wald tests on loadings and thresholds. We examined constrained and free baseline model comparisons strategies with different types and magnitudes of DIF, latent trait correlations, sample sizes, and levels of impact in an extensive Monte Carlo study. Results indicated the free baseline strategy was highly effective in detecting DIF, with power approaching 1.0 in the large sample size and large magnitude of DIF conditions, and similar effectiveness in the impact and no-impact conditions. This research also included an empirical example to demonstrate the viability of the best performing method with real examinees and showed how a DIF and a DTF effect size measure can be used to assess the practical significance of MFC DIF findings.

Download Full-text

Persistent asthma has an accumulative impact on the loss of smell in patients with nasal polyposis

Rhinology Journal ◽

10.4193/rhino10.295 ◽

2011 ◽

Vol 49 (5) ◽

pp. 519-524

Author(s):

Isam Alobid ◽

Sara Cardelus ◽

Pedro Benitez ◽

Jose Maria Guilemany ◽

Jordi Roca-Ferrer ◽

...

Keyword(s):

Nasal Polyposis ◽

Persistent Asthma ◽

Forced Choice ◽

Clinical Tool ◽

Sense Of Smell ◽

Asthmatic Patients ◽

Smell Test ◽

Significant Impairment ◽

The Impact ◽

Intermittent Asthma

Background: Loss of sense of smell is one of the most frequent complaints in patients with nasal polyposis (NP). The aim of this study was to evaluate the impact of asthma and its severity on the sense of smell. Material and methods: Patients with massive NP and healthy controls were included. More than half of patients presented with asthma. Olfactometry by Barcelona Smell Test 24, nasal symptoms score, nasal endoscopy, allergy study, and paranasal sinus CT scan were assessed. Results: NP patients showed a significant impairment in smell detection, identification, and forced choice compared to the control. Asthmatics reported lower scores for detection, identification, and forced choice than non-asthmatic patients. Patients with persistent asthma had an increased impairment of sense of smell on detection, identification, and forced choice than patients with intermittent asthma. No significant differences were found between mild, moderate, and severe persistent asthmatics. Paranasal sinuses opacification was inversely correlated with smell detection, identification and forced choice. Conclusion: These findings suggest that patients with NP have an impaired sense of smell, that asthma -particularly persistent asthma- has a further impact on sense of smell, and that loss of smell may be used as a clinical tool to identify the severity of both NP and asthma.

Download Full-text

Effects of Faking on the Predictive Validity of a Quasi-Ipsative Forced-Choice Personality Inventory: Implications for Sustainable Personnel Selection

Sustainability ◽

10.3390/su13084398 ◽

2021 ◽

Vol 13 (8) ◽

pp. 4398

Author(s):

Alexandra Martínez ◽

Silvia Moscoso ◽

Mario Lado

Keyword(s):

Predictive Validity ◽

Personality Assessment ◽

Personnel Selection ◽

Rating Scales ◽

Factor Model ◽

Five Factor Model ◽

Reliability And Validity ◽

Forced Choice ◽

Grade Point ◽

The Impact

Faking behavior is one of the main problems of personality measures. For this reason, determining the potential effects of faking on personality assessment procedures is relevant. The aim of this study has been to examine the impact of faking, induced in a laboratory setting, on the predictive validity of a quasi-ipsative forced-choice (FC) inventory based on the five-factor model. It also examined whether the magnitude of the predictive validity varied depending on the type of criteria analyzed (self-reported performance ratings and grade point average). The participants were 939 students from the University of Santiago de Compostela. As expected, the results showed that: (1) conscientiousness is the best predictor of performance even under faking response conditions; (2) conscientiousness predicts performance better when it is assessed using rating scales; and (3) reliability and validity were attenuated under faking conditions. Finally, we discuss the implications of these findings for the research and practice of personnel selection.

Download Full-text