An Experimental Test of the Impact of Varying Questionnaire Response Format on Prevalence Rates for Sexual Violence Victimization and Perpetration

Objective: This study examined the impact of a dichotomous versus scaled response format on prevalence rates of sexual violence perpetration and victimization, thus conceptually replicating Hamby et al., 2006 and extending those findings to the context of sexual violence. Methods: Two samples were recruited from Amazon Mechanical Turk and were randomly assigned to either a dichotomous or scaled response format of the same questionnaire. Sample 1 was used to examine perpetration and received a perpetration specific version of the Post-Refusal Sexual Persistence Scale (PRSPS). Sample 2 was used to examine victimization and received a victimization specific version of the PRSPS. Results: Men and women who received the scaled response format reported significantly more incidents of sexual perpetration, and for each type of tactic studied, on the order of 1.7-9x more frequently. The association between response format condition and prevalence rates of sexual violence victimization was significant only for men (23.0 vs 39.4%) and for the tactic of verbal coercion (30.1 vs 41.5%), with the scaled response format producing greater responding. Conclusions: The response format of sexual violence items can significantly alter prevalence rates of sexual violence perpetration, with scaled response formats producing greater endorsements than dichotomous formats. Response format also appears to impact prevalence rates of sexual violence victimization, particularly for men.

Interpretation Biases in Pain: Validation of Two New Stimulus Sets

Adults with chronic pain interpret ambiguous information in a pain and illness related fashion. However, limitations have been highlighted with traditional experimental paradigms used to measure interpretation biases. Whilst ambiguous scenarios have been developed to measure interpretation biases in adolescents with pain, no scenario sets exist for use with adults. Therefore, the present study: (i) sought to validate a range of ambiguous scenarios suitable for measuring interpretation biases in adults, whilst also allowing for two response formats (forced-choice and free response); and (ii) investigate paradigm efficacy, by assessing the effects of recent pain experiences on task responding. A novel ambiguous scenarios task was administered to adults (N = 241). Participants were presented with 62 ambiguous scenarios comprising 42 that could be interpreted in a pain/pain-illness or non-pain/non-pain illness manner: and 20 control scenarios. Participants generated their own solutions to each scenario (Word Generation Task), then rated how likely they would be to use two researcher-generated solutions to complete each scenario (Likelihood Ratings Task). Participants also rated their subjective experiences of pain in the last 3 months. Tests of reliability, including inter-rater agreement and internal consistency, produced two ambiguous scenario stimulus sets containing 18 and 20 scenarios, respectively. Further analyses revealed adults who reported more recent pain experiences were more likely to endorse the pain/pain-illness solutions in the Likelihood Ratings Task. This study provides two new stimulus sets for use with adults (including control items) in pain research and/or interventions. Results also provide evidence for a negative endorsement bias in adults.

An Experimental Test of the Impact of Varying Questionnaire Response Format on Prevalence Rates for Sexual Violence Victimization and Perpetration

Objective: This study examined the impact of a dichotomous versus scaled response format on prevalence rates of sexual violence perpetration and victimization, thus conceptually replicating Hamby et al., 2006 and extending those findings to the context of sexual violence. Methods: Two samples were recruited from Amazon Mechanical Turk and were randomly assigned to either a dichotomous or scaled response format of the same questionnaire. Sample 1 was used to examine perpetration and received a perpetration specific version of the Post-Refusal Sexual Persistence Scale (PRSPS). Sample 2 was used to examine victimization and received a victimization specific version of the PRSPS. Results: Men and women who received the scaled response format reported significantly more incidents of sexual perpetration, and for each type of tactic studied, on the order of 1.7-9x more frequently. The association between response format condition and prevalence rates of sexual violence victimization was significant only for men (23.0 vs 39.4%) and for the tactic of verbal coercion (30.1 vs 41.5%), with the scaled response format producing greater responding. Conclusions: The response format of sexual violence items can significantly alter prevalence rates of sexual violence perpetration, with scaled response formats producing greater endorsements than dichotomous formats. Response format also appears to impact prevalence rates of sexual violence victimization, particularly for men.

New insights on respondents’ recall ability and memory effects when repeatedly measuring political efficacy

Many study designs in social science research rely on repeated measurements implying that the same respondents are asked the same (or nearly the same) questions at least twice. An assumption made by such study designs is that respondents second answer does not depend on their first answer. However, if respondents recall their initial answer and base their second answer on it memory effects may affect the survey outcome. In this study, I investigate respondents’ recall ability and memory effects within the same survey and randomly assign respondents to a device type (PC or smartphone) and a response format (response scale or text field) for reporting their previous answer. While the results reveal no differences regarding device types, they reveal differences regarding response formats. Respondents’ recall ability is higher when they are provided with the response scale again than when they are only provided with a text field (without displaying the response scale again). The same finding applies to the size of estimated memory effects. This study provides evidence that the size of memory effects may have been overestimated in previous studies.

Capturing richer information: On establishing the validity of an interval-valued survey response mode

Obtaining quantitative survey responses that are both accurate and informative is crucial to a wide range of fields. Traditional and ubiquitous response formats such as Likert and visual analogue scales require condensation of responses into discrete or point values—but sometimes a range of options may better represent the correct answer. In this paper, we propose an efficient interval-valued response mode, whereby responses are made by marking an ellipse along a continuous scale. We discuss its potential to capture and quantify valuable information that would be lost using conventional approaches, while preserving a high degree of response efficiency. The information captured by the response interval may represent a possible response range—i.e., a conjunctive set, such as the real numbers between 3 and 6. Alternatively, it may reflect uncertainty in respect to a distinct response—i.e., a disjunctive set, such as a confidence interval. We then report a validation study, utilizing our recently introduced open-source software (DECSYS), to explore how interval-valued survey responses reflect experimental manipulations of several factors hypothesised to influence interval width, across multiple contexts. Results consistently indicate that respondents used interval widths effectively, and subjective participant feedback was also positive. We present this as initial empirical evidence for the efficacy and value of interval-valued response capture. Interestingly, our results also provide insight into respondents’ reasoning about the different aforementioned types of intervals—we replicate a tendency towards overconfidence for those representing epistemic uncertainty (i.e., disjunctive sets), but find intervals representing inherent range (i.e., conjunctive sets) to be well-calibrated.

Do different response formats affect how test takers approach a clinical reasoning task? An experimental study on antecedents of diagnostic accuracy using a constructed response and a selected response format

The use of response formats in assessments of medical knowledge and clinical reasoning continues to be the focus of both research and debate. In this article, we report on an experimental study in which we address the question of how much list-type selected response formats and short-essay type constructed response formats are related to differences in how test takers approach clinical reasoning tasks. The design of this study was informed by a framework developed within cognitive psychology which stresses the importance of the interplay between two components of reasoning—self-monitoring and response inhibition—while solving a task or case. The results presented support the argument that different response formats are related to different processing behavior. Importantly, the pattern of how different factors are related to a correct response in both situations seem to be well in line with contemporary accounts of reasoning. Consequently, we argue that when designing assessments of clinical reasoning, it is crucial to tap into the different facets of this complex and important medical process.

Comparison of Two Rating Scales With the Orofacial Esthetic Scale and Practical Recommendations for Its Application.

PurposeWe compared measurement properties of 5-point and 11-point response formats for Orofacial Esthetic Scale (OES) items to determine whether collapsing the format would degrade OES score precision.MethodsData were collected from a consecutive sample of adult dental patients from HealthPartners dental clinics in Minnesota (N=2,078). We fitted an Item Response Theory (IRT) model to the 11-point scale and six, derived 5-point scales. We compared all response formats using test (or scale) information, correlation between the IRT scores, Cronbach’s alpha estimates for each scaling format, correlations based on the observed scores for the seven OES items and the eighth global item, and the relationship of observed and IRT scores to an external criterion using orofacial appearance (OA) indicators from the Oral Health Impact Profile (OHIP).ResultsThe correlations among scores based on the different response formats were uniformly high for observed (0.97-0.99) and IRT scores (0.96-0.99); as were correlations of both observed and IRT scores and the OHIP measure of OA (0.65-0.69). Cronbach’s alpha based on any of the 5-point formats (α = 0.95) was nearly the same as that based on the 11-point format (α = 0.96). The weighted total information area for five of six, 5-point derived formats was 98% of that for the 11-point scale ConclusionsOur results support the use of scores based on a 5-point response format for OES items. The measurement properties of scores based on a 5-point response format are comparable to those of scores based on the 11-point format.

Student and staff expectations and experiences of a UK–China Transnational Education collaboration

PurposeThis study reports the results of a survey of teaching staff in the UK and China and two cohorts of Chinese students. We explored perceptions of a Transnational Education (TNE) course taught by UK teachers at a Chinese university to ascertain similarities and differences in perceptions and to help inform future TNE design and provision.Design/methodology/approachTeachers in the UK (N = 10) and Chinese University (N = 20) and two cohorts of Chinese students (N = 102) completed questionnaires about their perceptions of the collaboration, including why they thought the university had developed the course and the benefits and risks of the course. They were also asked why they personally or why they thought the students had enrolled in the course. Questions were a mix of forced choice and open response formats.FindingsPrior to the course commencing, differences were identified between UK and Chinese teachers in their perception of the risks and benefits of the course and the challenges students might face. Differences were also seen in teachers' and students' views about why students enrolled and their expectations about the course. The TNE is no longer running and many of the reasons for this were identified by our participants before the course had begun.Originality/valueThis suggests the importance of engaging with various stakeholders in the setup of TNE to ensure a close match between staff, student and institutional expectations of the course. This is likely to increase the likelihood of success of such programmes.