scholarly journals Using cognitive interviewing to bridge the intent‐interpretation gap for nutrition coverage survey questions in India

2021 ◽  
Author(s):  
Sattvika Ashok ◽  
Sunny S. Kim ◽  
Rebecca A. Heidkamp ◽  
Melinda K. Munos ◽  
Purnima Menon ◽  
...  
Keyword(s):  
Cognitive Interviewing ◽  
Survey Questions

scholarly journals Open-ended versus Closed Probes: Assessing Different Formats of Web Probing

2021 ◽  
pp. 004912412110312
Author(s):  
Cornelia E. Neuert ◽  
Katharina Meitinger ◽  
Dorothée Behr
Keyword(s):  
Cognitive Processes ◽  
Educational Level ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Web Surveys ◽  
Time Efficiency ◽  
Survey Questions ◽  
Item Functioning ◽  
Reduced Costs ◽  
Interviewing Techniques

The method of web probing integrates cognitive interviewing techniques into web surveys and is increasingly used to evaluate survey questions. In a usual web probing scenario, probes are administered immediately after the question to be tested (concurrent probing), typically as open-ended questions. A second possibility of administering probes is in a closed format, whereby the response categories for the closed probes are developed during previously conducted qualitative cognitive interviews. Using closed probes has several benefits, such as reduced costs and time efficiency, because this method does not require manual coding of open-ended responses. In this article, we investigate whether the insights gained into item functioning when implementing closed probes are comparable to the insights gained when asking open-ended probes and whether closed probes are equally suitable to capture the cognitive processes for which traditionally open-ended probes are intended. The findings reveal statistically significant differences with regard to the variety of themes, the patterns of interpretation, the number of themes per respondent, and nonresponse. No differences in number of themes across formats by sex and educational level were found.

scholarly journals Comparing Traditional and Crowdsourcing Methods for Pretesting Survey Questions

SAGE Open ◽  
2016 ◽  
Vol 6 (4) ◽  
pp. 215824401667177 ◽  
Author(s):  
Jennifer Edgar ◽  
Joe Murphy ◽  
Michael Keating
Keyword(s):  
Collective Intelligence ◽  
Amazon Mechanical Turk ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Quality Of Data ◽  
The Public ◽  
Survey Questions ◽  
Participant Characteristics ◽  
Cost Efficient

Cognitive interviewing is a common method used to evaluate survey questions. This study compares traditional cognitive interviewing methods with crowdsourcing, or “tapping into the collective intelligence of the public to complete a task.” Crowdsourcing may provide researchers with access to a diverse pool of potential participants in a very timely and cost-efficient way. Exploratory work found that crowdsourcing participants, with self-administered data collection, may be a viable alternative, or addition, to traditional pretesting methods. Using three crowdsourcing designs (TryMyUI, Amazon Mechanical Turk, and Facebook), we compared the participant characteristics, costs, and quantity and quality of data with traditional laboratory-based cognitive interviews. Results suggest that crowdsourcing and self-administered protocols may be a viable way to collect survey pretesting information, as participants were able to complete the tasks and provide useful information; however, complex tasks may require the skills of an interviewer to administer unscripted probes.

Cognitive interviewing and responses to EU kids online survey questions

2018 ◽  
pp. 33-44
Author(s):  
Christine Ogan ◽  
Türkan Karakuş ◽  
Engin Kurşun ◽  
Kürşat Çağiltay ◽  
Duygu Kaşikçi
Keyword(s):  
Online Survey ◽  
Cognitive Interviewing ◽  
Survey Questions

Applicability of a Salient Belief Elicitation to Measure Abortion Beliefs

2021 ◽  
Vol 45 (1) ◽  
pp. 81-94
Author(s):  
Julie M. Maier ◽  
Kristen N. Jozkowski ◽  
Danny Valdez ◽  
Brandon L. Crawford ◽  
Ronna C. Turner ◽  
...  
Keyword(s):  
Health Behavior ◽  
Reasoned Action ◽  
Behavioral Control ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Reasoned Action Approach ◽  
Belief Elicitation ◽  
Survey Questions

Objectives: Salient belief elicitations (SBEs), informed by the Reasoned Action Approach (RAA), are used to identify 3 sets of beliefs – behavioral, control, and normative – that influence attitudes toward a health behavior. SBEs ask participants about their own beliefs through open-ended questions. We adapted a SBE by focusing on abortion, which is infrequently examined through SBEs; we also included a survey version that asked participants their views on what a hypothetical woman would do if contemplating an abortion. Given these deviations from traditional SBEs, the purpose of this study was to assess if the adapted SBE was understood by participants in English and Spanish through cognitive interviewing. Methods: We examined participants' interpretations of SBE items about abortion to determine if they aligned with the corresponding RAA construct. We administered SBE surveys and conducted cognitive interviews with US adults in both English and Spanish. Results: Participants comprehended the SBE questions as intended. Participants' interpretations of most questions were also in line with the respective RAA construct. Conclusions: SBE survey questions were comprehended well by participants. We discuss areas in which SBE questions can be modified to improve alignment with the underlying RAA construct to assess abortion beliefs.

scholarly journals Design and Analysis of Cognitive Interviews for Comparative Multinational Testing

Field Methods ◽  
2011 ◽  
Vol 23 (4) ◽  
pp. 379-396 ◽  
Author(s):  
Kristen Miller ◽  
Rory Fitzgerald ◽  
José-Luis Padilla ◽  
Stephanie Willson ◽  
Sally Widdop ◽  
...  
Keyword(s):  
Survey Data ◽  
Cross Cultural ◽  
Cognitive Testing ◽  
Cognitive Interviewing ◽  
Evidence Based ◽  
Cognitive Interviews ◽  
Survey Questions ◽  
Method Bias

This article summarizes the work of the Comparative Cognitive Testing Workgroup, an international coalition of survey methodologists interested in developing an evidence-based methodology for examining the comparability of survey questions within cross-cultural or multinational contexts. To meet this objective, it was necessary to ensure that the cognitive interviewing (CI) method itself did not introduce method bias. Therefore, the workgroup first identified specific characteristics inherent in CI methodology that could undermine the comparability of CI evidence. The group then developed and implemented a protocol addressing those issues. In total, 135 cognitive interviews were conducted by participating countries. Through the process, the group identified various interpretive patterns resulting from sociocultural and language-related differences among countries as well as other patterns of error that would impede comparability of survey data.

Cognitive interviewing and responses to EU Kids Online survey questions

2012 ◽  
pp. 33-44 ◽  
Author(s):  
Christine Ogan ◽  
Türkan Karakuş ◽  
Engin Kurşun ◽  
Kürsat Çağiltay ◽  
Duygu Kaşikçi
Keyword(s):  
Online Survey ◽  
Cognitive Interviewing ◽  
Survey Questions

scholarly journals Using Cognitive Interviewing and Behavioral Coding to Determine Measurement Equivalence across Linguistic and Cultural Groups

Field Methods ◽  
2011 ◽  
Vol 23 (4) ◽  
pp. 439-460 ◽  
Author(s):  
James F. Thrasher ◽  
Anne C. K. Quah ◽  
Gregory Dominick ◽  
Ron Borland ◽  
Pete Driezen ◽  
...  
Keyword(s):  
United States ◽  
Cognitive Interviewing ◽  
Measurement Bias ◽  
Cultural Group ◽  
Cultural Groups ◽  
Potential Response ◽  
Protocol Development ◽  
Behavioral Coding ◽  
Survey Questions ◽  
Response Errors

This study examined and compared results from two questionnaire pretesting methods (i.e., behavioral coding and cognitive interviewing [CI]) to assess systematic measurement bias in survey questions for adult smokers across six countries (United States, Australia, Uruguay, Mexico, Malaysia, and Thailand). Protocol development and translation involved multiple bilingual partners in each linguistic/cultural group. The study was conducted with convenience samples of 20 adult smokers in each country. Behavioral coding and CI methods produced similar conclusions regarding measurement bias for some questions; however, CI was more likely to identify potential response errors than behavioral coding. Coordinated qualitative pretesting of survey questions (or postsurvey evaluation) is feasible across cultural groups and can provide important information on comprehension and comparability. The CI appears to be a more robust technique than behavioral coding, although combinations of the two might be even better.

scholarly journals Bridging the Gap Between Intent & Interpretation: Enhancing Survey Questions on Maternal & Child Nutrition Intervention Coverage Through Cognitive Interviewing in India

2021 ◽  
Vol 5 (Supplement_2) ◽  
pp. 869-869
Author(s):  
Sattvika Ashok ◽  
Sunny S Kim ◽  
Rasmi Avula ◽  
Rebecca A Heidkamp ◽  
Melinda K Munos ◽  
...  
Keyword(s):  
Child Nutrition ◽  
Large Scale ◽  
Food Policy ◽  
Survey Design ◽  
Policy Research ◽  
Meat Products ◽  
Nutrition Intervention ◽  
Cognitive Interviewing ◽  
Intervention Coverage ◽  
Survey Questions

Abstract Objectives Designing survey questions that clearly and precisely communicate the question's intent and elicit responses based on the intended interpretation is critical but often undervalued. We used cognitive interviewing to qualitatively assess respondents’ interpretation and responses to questions pertaining to maternal and child nutrition intervention coverage. Methods We conducted interviews with mothers (N = 21) with children less than one year in Madhya Pradesh, India, to cognitively test 25 survey questions. Each question was followed by probes to capture information on four cognitive stages - comprehension, retrieval, judgement, and response. Interviews were recorded and notes were taken on verbal and non-verbal cues. Data were analyzed for common and unique patterns across the survey questions within the cognitive domains and grouped into challenges. Results We identified four types of cognitive challenges: 1) Poor retention of multiple concepts in long questions: difficulty in comprehending and retaining questions with three or more key concepts; 2) Temporal confusion: difficulty in conceptualizing recall periods such as “in the last 6 months” as compared to life stages such as pregnancy; 3) Misinterpretation of concepts: misinterpretation of the information being asked; meaning of certain terms such as “animal-source foods” was considered as referring to meat products only and not milk and eggs; scope of intervention using the phrase “talk with you” in referring to counseling was interpreted in different ways by respondents; and 4) Poor understanding of technical terms: difficulty in understanding even commonly-used technical words such as “breastfeeding” and “antenatal care” requiring the use of plain and simple alternative language. Conclusions Findings from this study will be useful for stakeholders involved in survey design and implementation, especially those conducting large-scale household surveys to improve coverage data of essential nutrition interventions, which is critical for policy actions Funding Sources Bill & Melinda Gates Foundation through the DataDENT initiative and the Improving Measurement and Program Design grant, and the CGIAR Research Program on Agriculture for Nutrition and Health (A4NH), led by the International Food Policy Research Institute.

Sign in / Sign up

Export Citation Format

Share Document