The Cognitive Interviewing Reporting Framework (CIRF)

Methodology ◽  
2013 ◽  
Vol 9 (3) ◽  
pp. 87-95 ◽  
Author(s):  
Hennie Boeije ◽  
Gordon Willis
Keyword(s):  
Qualitative Research ◽  
Cognitive Testing ◽  
Cognitive Interviewing ◽  
Comprehensive Description ◽  
Widespread Adoption ◽  
Reporting Framework ◽  
Common Framework ◽  
Survey Questionnaires ◽  
Complete Reporting

Cognitive interviewing is an important qualitative tool for the testing, development, and evaluation of survey questionnaires. Despite the widespread adoption of cognitive testing, there remain large variations in the manner in which specific procedures are implemented, and it is not clear from reports and publications that have utilized cognitive interviewing exactly what procedures have been used, as critical details are often missing. Especially for establishing the effectiveness of procedural variants, it is essential that cognitive interviewing reports contain a comprehensive description of the methods used. One approach to working toward more complete reporting would be to develop and adhere to a common framework for reporting these results. In this article we introduce the Cognitive Interviewing Reporting Framework (CIRF), which applies a checklist approach, and which is based on several existing checklists for reviewing and reporting qualitative research. We propose that researchers apply the CIRF in order to test its usability and to suggest potential adjustments. Over the longer term, the CIRF can be evaluated with respect to its utility in improving the quality of cognitive interviewing reports.

Reflections on the Cognitive Interviewing Reporting Framework

Methodology ◽  
2013 ◽  
Vol 9 (3) ◽  
pp. 123-128 ◽  
Author(s):  
Gordon Willis ◽  
Hennie Boeije
Keyword(s):  
Research Design ◽  
Cognitive Testing ◽  
Level Of Detail ◽  
Cognitive Interviewing ◽  
Research Groups ◽  
Research Project ◽  
Reporting Framework ◽  
Full Explanation ◽  
Key Steps

Based on the experiences of three research groups using and evaluating the Cognitive Interviewing Reporting Framework (CIRF), we draw conclusions about the utility of the CIRF as a guide to creating cognitive testing reports. Authors generally found the CIRF checklist to be usable, and that it led to a more complete description of key steps involved. However, despite the explicit direction by the CIRF to include a full explanation of major steps and features (e.g., research objectives and research design), the three cognitive testing reports tended to simply state what was done, without further justification. Authors varied in their judgments concerning whether the CIRF requires the appropriate level of detail. Overall, we believe that current cognitive interviewing practice will benefit from including, within cognitive testing reports, the 10 categories of information specified by the CIRF. Future use of the CIRF may serve to direct the overall research project from the start, and to further the goal of evaluation of specific cognitive interviewing procedures.

Cross-Cultural Cognitive Interviewing

Field Methods ◽  
2011 ◽  
Vol 23 (4) ◽  
pp. 331-341 ◽  
Author(s):  
Gordon B. Willis ◽  
Kristen Miller
Keyword(s):  
Qualitative Method ◽  
Cross Cultural ◽  
Cognitive Testing ◽  
Self Report ◽  
Cognitive Interviewing ◽  
Field Methods ◽  
The Cross ◽  
Linguistic Groups ◽  
Current Special Issue ◽  
Survey Questionnaires

Cognitive interviewing (CI) has emerged as a key qualitative method for the pretesting and evaluation of self-report survey questionnaires. This article defines CI, describes its key features, and outlines the data analysis techniques that are commonly used. The authors then consider recent extensions of cognitive testing to the cross-cultural survey research realm, where the major practical objectives are: (1) to facilitate inclusion of a range of cultural and linguistic groups and (2) for purposes of comparative analysis, to produce survey questionnaire items that exhibit comparability of measurement, across groups. Challenges presented by this extension to the cross-cultural and multilingual areas are discussed. Finally, the authors introduce the articles contained within the current special issue of Field Methods (2011), which endeavor to apply cognitive testing in specific cross-cultural survey projects, and to both identify and suggest solutions to the unique problems that face questionnaire designers and researchers more generally, in the practice of survey pretesting and evaluation methods as these endeavor to cover the sociocultural spectrum.

Examining the Personal Experience of Aging Scale With the Three Step Test Interview

Methodology ◽  
2013 ◽  
Vol 9 (3) ◽  
pp. 96-103 ◽  
Author(s):  
Christina Bode ◽  
Harrie Jansen
Keyword(s):  
Rheumatic Diseases ◽  
Personal Experience ◽  
Cognitive Interview ◽  
Step Test ◽  
Cognitive Interviewing ◽  
Physical Decline ◽  
Reporting Framework ◽  
Interview Methodology ◽  
Statistical Consistency

A study on aging experience in patients with rheumatic diseases showed low statistical consistency (Cronbach’s alpha) for the Physical Decline subscale of the Personal Experience of Aging Scale whereas the scale functioned very well in preceding surveys. This discrepancy led to the decision to examine the contextual validity of the subscale with a cognitive interview methodology. In the current study we applied the Three Step Test Interview as a cognitive interviewing device to explore misinterpretations and other problems in answering the PD questionnaire items, with the aim of identifying item improvements that might also be beneficial to the statistical quality of the PD subscale. Some problems were identified that were not related to the statistical inter-item correlation rates, nor to the arthritis context of this sample, but were rather due to faulty item formulation. Reformulations were designed to improve the items. The report of this study follows the Cognitive Interviewing Reporting Framework. Experiences with this new guideline are discussed.

scholarly journals “We tried to, but life gets in the way”: The Value of Cognitive Interviewing for Testing a Questionnaire on Antibiotic Consumption Behaviours

2019 ◽  
Author(s):  
Alistair Anderson
Keyword(s):  
Social Research ◽  
Antibiotic Use ◽  
Antibiotic Consumption ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Drug Resistant ◽  
Socially Desirable Responding ◽  
Pet Owners ◽  
Survey Questionnaires

The public’s perceptions and behaviours regarding antibiotics and antibiotic stewardship are often measured using survey questionnaires. Previous qualitative testing of questionnaire instruments that include questions about antibiotics or drug-resistant infections has suggested that questions on this topic are subject to measurement error for multiple reasons. This study used 13 cognitive interviews with pet owners to identify issues with a questionnaire instrument for a survey examining pet owners’ knowledge and behaviour around antibiotic use both personally and regarding their pets. Key findings from the study are that there are notable differences in quality of recollection between personal and pet-focused antibiotic use, and that socially desirable responding is only applied to certain behaviour questions. This article argues that cognitive interviews can provide substantial benefits as part of mixed-methods social research by supplementing dimension-reduced survey questionnaires with participants’ explanations, narratives, and experiences relating to the survey questions, with further benefits for qualitative theme generation.

Evaluating the Cognitive Interviewing Reporting Framework (CIRF) by Rewriting a Dutch Pretesting Report of a European Health Survey Questionnaire *The views expressed in this paper are those of the authors and do not necessarily reflect the policies of Statistics Netherlands.

Methodology ◽  
2013 ◽  
Vol 9 (3) ◽  
pp. 104-112 ◽  
Author(s):  
Rachel Vis-Visschers ◽  
Vivian Meertens
Keyword(s):  
Health Survey ◽  
Cognitive Interviewing ◽  
Survey Questionnaire ◽  
Reporting Framework ◽  
Research Designs ◽  
European Health ◽  
Original Report ◽  
Minimal Standard ◽  
Exhaustive List

We used the Cognitive Interviewing Reporting Framework (CIRF) to restructure the report of a pretest on a European health survey questionnaire. This pretest was conducted by the Questionnaire Laboratory of Statistics Netherlands, and the original report was written according to a standard Statistics Netherlands format for pretesting reports. This article contains the rewritten report with highlights from the case study. The authors reflect on the process of rewriting and the usefulness of the CIRF. We conclude that expanded use of the CIRF as a reporting format for articles on cognitive pretests would enhance international comparability, completeness, and uniformity of research designs, terminology, and reporting. A limitation of the CIRF is that it does not provide an exhaustive list of items that could be included in a report, but it is more a “minimal standard”: that is a report on how a cognitive pretest was conducted should at least contain a description of the CIRF items.

Getting out of the Field

2017 ◽  
Author(s):  
Jeasik Cho
Keyword(s):  
Qualitative Research ◽  
Academic Research ◽  
Scientific Research ◽  
Research Community

This chapter provides a review of the book, which explores how to conceptually understand and practically evaluate the quality of qualitative research. Despite the fact that there are few scholarly pieces regarding qualitative research, the depth and creativity that the pioneering researchers have demonstrated are profound, and the extent to which they cover not only the broad quality of qualitative research but also most of the specific qualities expected by many different kinds of qualitative research is incredible. This chapter summarizes the major topics of this book. Final remarks on this exciting, creative, but difficult topic are preceded by the following summary: Fortunately, There are commonly agreed, bold standards for evaluating the goodness of qualitative research in the academic research community. These standards are a part of what is generally called “scientific research.”

Evaluating Qualitative Research

2017 ◽  
Author(s):  
Jeasik Cho
Keyword(s):  
Qualitative Research ◽  
Large Scale ◽  
Academic Research ◽  
Research Community ◽  
Assessment Tools ◽  
Journal Articles ◽  
Journal Editors ◽  
The Past ◽  
Grant Proposals

This book provides the qualitative research community with some insight on how to evaluate the quality of qualitative research. This topic has gained little attention during the past few decades. We, qualitative researchers, read journal articles, serve on masters’ and doctoral committees, and also make decisions on whether conference proposals, manuscripts, or large-scale grant proposals should be accepted or rejected. It is assumed that various perspectives or criteria, depending on various paradigms, theories, or fields of discipline, have been used in assessing the quality of qualitative research. Nonetheless, until now, no textbook has been specifically devoted to exploring theories, practices, and reflections associated with the evaluation of qualitative research. This book constructs a typology of evaluating qualitative research, examines actual information from websites and qualitative journal editors, and reflects on some challenges that are currently encountered by the qualitative research community. Many different kinds of journals’ review guidelines and available assessment tools are collected and analyzed. Consequently, core criteria that stand out among these evaluation tools are presented. Readers are invited to join the author to confidently proclaim: “Fortunately, there are commonly agreed, bold standards for evaluating the goodness of qualitative research in the academic research community. These standards are a part of what is generally called ‘scientific research.’ ”

Evaluating slogan translation from the readers’ perspective

Babel ◽  
2015 ◽  
Vol 61 (2) ◽  
pp. 283-303 ◽  
Author(s):  
Lily Lim ◽  
Kwok Ying Loi
Keyword(s):  
Public Administration ◽  
English Translation ◽  
Public Authorities ◽  
Word Use ◽  
The Public ◽  
Translation Quality ◽  
Sports Events ◽  
Growing Economy ◽  
Survey Questionnaires

Slogans play an important role in conveying information to targeted audiences, and the translation of slogans tends to be studied under the rubric of public-notice translation. Previous research mainly uses researchers’ expertise to evaluate the quality of this type of translation. Yet, little is known about what the targeted readers think about the translation, although their opinions present key information that helps to determine whether the translation has achieved the intended effect. This paper elicits and systematically analyzes readers’ responses. We investigate the case of Macao, a rapidly growing economy where the demand for English translation has markedly increased in recent decades. Public administration bodies in Macao have commissioned Chinese-to-English translation in varied areas such as tourism, social security and welfare, cultural and sports events. We sampled ten translated slogans that were used in the public sector, and administered survey questionnaires (n=130) to both source-text and target-text readers. The two groups of readers’ evaluations, based on the criteria of fluency, conciseness, persuasiveness and mnemonic effect, reveal that the translations are perceived significantly less favorably than the originals are. Readers most strongly disliked word-for-word translations, and pointed out numerous problems with the translations such as ungrammaticality, inappropriate word use, lack of appeal, and unintelligibility due to insufficient background knowledge. This research demonstrates the tangible value of using readers’ responses to evaluate translation quality. It also has implications for translator training, and recommends that public authorities should institute a rigorous quality assurance system.

Peran Lembaga Pemberdayaan Ekonomi Mustahik Baznas Dalam Meningkatkan Kesejahteraan Ekonomi Mustahik

2021 ◽  
Vol 4 (2) ◽  
pp. 477-494
Author(s):  
Ali Idrus ◽  
Ade Maman
Keyword(s):  
Quality Of Life ◽  
Qualitative Research ◽  
Data Collection ◽  
Small And Medium Enterprises ◽  
Business Development ◽  
Economic Welfare ◽  
Descriptive Data ◽  
Medium Enterprises

One of the solutions proposed by Badan Amil Zakat Nasional is to form Lembaga Pemberdayaan Ekonomi Mustahik which has the task of improving the quality of life of mustahik through its programs. So, the purpose of this study is to find out how the role of Lembaga Pemberdayaan Ekonomi Mustahik Badan Amil Zakat Nasional in improving the economic welfare of mustahik. This study uses qualitative research methods that produce descriptive data. Where this research is based on observations made by the author and then explained according to what the author observed in the field. The data collection that the researchers did was using the method of observation, documentary interviews, and other data. When the data has been collected, the researcher conducts an analysis and then draws conclusions from the analysis. The results of this study are Lembaga Pemberdayaan Ekonomi Mustahik BAZNAS plays a role in the development of micro, small and medium enterprises (MSMEs) and focuses on potential creative businesses, empowers by providing business development capital assistance, in accordance with the budget draft proposed by mustahik, and approved by LPEM BAZNAS. Keywords: The Role of LPEM BAZNAS, Mustahik's Economic Welfare

The Relationship of Music-Sound, Technology and Internet

2020 ◽  
Vol 1 (1) ◽  
pp. 160-169
Author(s):  
Clemens Felix Setiyawan ◽  
Dyah Murwaningrum
Keyword(s):  
Qualitative Research ◽  
Research Method ◽  
Music Technology ◽  
The Internet ◽  
Young Generation ◽  
Participant Observer ◽  
Relationship Of ◽  
The Relationship ◽  
Sound Technology

Nowadays, music creation, collaboration, and publication are easier because of technology. Most young generations have sent music data, made, sold, bought music files on the internet. This changed music processes certainly resulted in different outcomes. Listening and creating music by new means, can change music itself. Technology has simplified tools, and the internet has simplified the distance. But new problems and questions have been found. How were the internet and technology influenced the quality of music, music creator, music appreciator and the form of music. The aims of this research to determine the relationship between music, technology, and the internet, through behavior of the young generation. This study was qualitative research that used observations and unstructured interviews. In subsequent observations, participant-observer was chosen as an advanced research method to better understand existing phenomena. The result of observations and interviews were interpreted, then presented descriptively. This research used theory by Don Ihde that technology has three characteristics (1) material (2) used (3)relationship of human and tools. The result of this research is internet influenced music quality and human appreciation. Technology changed the way humans create music.

Sign in / Sign up

Export Citation Format

Share Document