scholarly journals Comparing Traditional and Crowdsourcing Methods for Pretesting Survey Questions

SAGE Open ◽  
2016 ◽  
Vol 6 (4) ◽  
pp. 215824401667177 ◽  
Author(s):  
Jennifer Edgar ◽  
Joe Murphy ◽  
Michael Keating
Keyword(s):  
Collective Intelligence ◽  
Amazon Mechanical Turk ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Quality Of Data ◽  
The Public ◽  
Survey Questions ◽  
Participant Characteristics ◽  
Cost Efficient

Cognitive interviewing is a common method used to evaluate survey questions. This study compares traditional cognitive interviewing methods with crowdsourcing, or “tapping into the collective intelligence of the public to complete a task.” Crowdsourcing may provide researchers with access to a diverse pool of potential participants in a very timely and cost-efficient way. Exploratory work found that crowdsourcing participants, with self-administered data collection, may be a viable alternative, or addition, to traditional pretesting methods. Using three crowdsourcing designs (TryMyUI, Amazon Mechanical Turk, and Facebook), we compared the participant characteristics, costs, and quantity and quality of data with traditional laboratory-based cognitive interviews. Results suggest that crowdsourcing and self-administered protocols may be a viable way to collect survey pretesting information, as participants were able to complete the tasks and provide useful information; however, complex tasks may require the skills of an interviewer to administer unscripted probes.

scholarly journals Open-ended versus Closed Probes: Assessing Different Formats of Web Probing

2021 ◽  
pp. 004912412110312
Author(s):  
Cornelia E. Neuert ◽  
Katharina Meitinger ◽  
Dorothée Behr
Keyword(s):  
Cognitive Processes ◽  
Educational Level ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Web Surveys ◽  
Time Efficiency ◽  
Survey Questions ◽  
Item Functioning ◽  
Reduced Costs ◽  
Interviewing Techniques

The method of web probing integrates cognitive interviewing techniques into web surveys and is increasingly used to evaluate survey questions. In a usual web probing scenario, probes are administered immediately after the question to be tested (concurrent probing), typically as open-ended questions. A second possibility of administering probes is in a closed format, whereby the response categories for the closed probes are developed during previously conducted qualitative cognitive interviews. Using closed probes has several benefits, such as reduced costs and time efficiency, because this method does not require manual coding of open-ended responses. In this article, we investigate whether the insights gained into item functioning when implementing closed probes are comparable to the insights gained when asking open-ended probes and whether closed probes are equally suitable to capture the cognitive processes for which traditionally open-ended probes are intended. The findings reveal statistically significant differences with regard to the variety of themes, the patterns of interpretation, the number of themes per respondent, and nonresponse. No differences in number of themes across formats by sex and educational level were found.

Using Cognitive Interviews to Guide Questionnaire Construction for Cross-National Crime Surveys

2017 ◽  
Author(s):  
Stephen Farrall
Keyword(s):  
Organizational Structures ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Social Scientists ◽  
Intended Meaning ◽  
National Surveys ◽  
Conceptual Equivalence ◽  
Questionnaire Construction ◽  
Cross National

What is a “snowball”? For some, a snowball is a drink made of advocaat and lemonade; for others, a mix of heroin and cocaine injected; for yet others, a handful of packed snow commonly thrown at objects or people; for gamblers, it refers to a cash prize that accumulates over successive games; for social scientists, it is a form of sampling. There are other uses for the term in the stock market and further historical usages that refer to stealing things from washing lines or that are racist. Clearly then, different people in different contexts and different times will have used the term “snowball” to refer to various activities or processes. Problems like this—whereby a particular word or phrase may have various meanings or may be interpreted variously—are just one of the issues for which cognitive interviews can offer insights (and possible solutions). Cognitive interviews can also help researchers designing surveys to identify problems with mistranslation of words, or near-translations that do not quite convey the intended meaning. They are also useful for ensuring that terms are understood in the same way by all sections of society, and that they can be used to assess the degree to which organizational structures are similar in different countries (not all jurisdictions have traffic police, for example). They can also assess conceptual equivalence. Among the issues explored here are the following: • What cognitive interviews are • The background to their development • Why they might be used in cross-national crime and victimization surveys • Some of the challenges associated with cross-national surveys • Ways cognitive interviews can help with these challenges • Different approaches to cognitive interviewing (and the advantages of each) • How to undertake cognitive interviews • A “real-world” example of a cognitive interviewing exercise • Whether different probing styles make any difference to the quality of the data derived.

Technologies for Wellbeing and Healthy Living

2014 ◽  
Vol 5 (1) ◽  
pp. 30-40
Author(s):  
Jochen Meyer
Keyword(s):  
Health Technology ◽  
Quality Of Data ◽  
The Public ◽  
Use Of Data ◽  
Technical Systems ◽  
Navigational Systems ◽  
Behavior Models ◽  
Data Design

It is a strange paradox that the public is talking about health technology but cares more about disease technology: people address chronic diseases, people want to change unhealthy behaviors, people aim to help carers and nurses - but people hardly ever look at those who are and want to remain healthy. This is even stranger, as times of health outnumber periods of disease in most persons` lifetimes. Somewhat surprisingly, technology available today is not yet optimally suited to help staying healthy. The authors discuss challenges with respect to the adaption of health behavior models, long-term interaction, quality of data, design of devices, primary use of data, and life-long data. And the authors suggest understanding technical systems for wellbeing as navigational systems, guiding a person through life on a healthy path.

scholarly journals Regulatory requirements for Vaccine registration in United States

2018 ◽  
Vol 6 (2) ◽  
pp. 85-91 ◽  
Author(s):  
Shailaja Pashikanti ◽  
Sowmya A.N.V.L ◽  
Jyothi Sri Durga V
Keyword(s):  
Biological Origin ◽  
Restrictive Procedure ◽  
The Public ◽  
Rigorous Testing ◽  
Management Procedures ◽  
Good Manufacturing Practices ◽  
Adolescents And Adults ◽  
Cost Efficient ◽  
Assess Quality

Vaccination is one among the foremost cost-efficient health interventions out available, saving a lot of individuals from illness, incapacity, and death annually. No alternative countermeasures are effective in reducing or eliminating the prevalence of infectious diseases reminiscent of measles, mumps, rubella, smallpox, and diphtheria. Vaccines are products of biological origin that exhibit some inherent variability. They are characterized by advanced manufacturing processes and are administered to a huge number of healthy youngsters, adolescents, and adults. Their quality cannot be assessed by testing the ultimate product alone. The vaccine industry is highly regulated. Vaccines development maybe an advanced and long method. Before a new vaccine is approved for release into the market, a rigorous restrictive procedure to assess quality, effectiveness, and safety should be undertaken. The Office of Vaccines Research and Review (OVRR) at the Center for Biologics Evaluation and Research (CBER), U.S. Food and Drug Administration (FDA) are responsible for the regulation of Vaccines. Current authority for the regulation of vaccines is in Section 351(a) of the Public Health Service Act (PHS).  Throughout the lifecycle of development, from preclinical studies to licensure, vaccines are subjected to rigorous testing and oversight. Manufacturers should adhere to good manufacturing practices and management procedures to make sure the quality of vaccines.

GuiMarket

2010 ◽  
pp. 1193-1205
Author(s):  
M. Manuela Cruz-Cunha ◽  
António Tavares ◽  
Isabel Miranda
Keyword(s):  
Social Services ◽  
New Technology ◽  
Pilot Project ◽  
Healthcare Sector ◽  
The Public ◽  
Goal Definition ◽  
The Social ◽  
Cost Efficient ◽  
The U.S

The healthcare sector has been continuously growing in importance in the past years throughout the entire world, and particularly in most Western countries and the U.S., where we witness an increase of expenditure in health per capita every year. This is related to many aspects of contemporary society, including an increase in life expectancy, the public demand for a better quality of life and better health services. This must be met with more cost-efficient approaches, and new technology-based solutions for providing health and other services. The chapter contextualizes the utilization of electronic marketplaces (e-marketplaces) for the social and healthcare sectors, how this field has been evolving in recent years, current challenges and trends, and their contribut to society. The authors also discuss a pilot project of an e-marketplace for healthcare and social services currently being developed in the Guimarães Municipality, including its goal, definition and implementation, as well as the commercially available enabling technology and tools.

Amazon Mechanical Turk

2012 ◽  
pp. 90-102
Author(s):  
Amber Chauncey Strain ◽  
Lucille M. Booker
Keyword(s):  
Mechanical Turk ◽  
Amazon Mechanical Turk ◽  
Quality Of Data ◽  
High Quality ◽  
Web Based ◽  
Potential Benefits ◽  
Amazon's Mechanical Turk ◽  
Short Time ◽  
Time Frames

One of the major challenges of ANLP research is the constant balancing act between the need for large samples, and the excessive time and monetary resources necessary for acquiring those samples. Amazon’s Mechanical Turk (MTurk) is a web-based data collection tool that has become a premier resource for researchers who are interested in optimizing their sample sizes and minimizing costs. Due to its supportive infrastructure, diverse participant pool, quality of data, and time and cost efficiency, MTurk seems particularly suitable for ANLP researchers who are interested in gathering large, high quality corpora in relatively short time frames. In this chapter, the authors first provide a broad description of the MTurk interface. Next, they describe the steps for acquiring IRB approval of MTurk experiments, designing experiments using the MTurk dashboard, and managing data. Finally, the chapter concludes by discussing the potential benefits and limitations of using MTurk for ANLP experimentation.

Applicability of a Salient Belief Elicitation to Measure Abortion Beliefs

2021 ◽  
Vol 45 (1) ◽  
pp. 81-94
Author(s):  
Julie M. Maier ◽  
Kristen N. Jozkowski ◽  
Danny Valdez ◽  
Brandon L. Crawford ◽  
Ronna C. Turner ◽  
...  
Keyword(s):  
Health Behavior ◽  
Reasoned Action ◽  
Behavioral Control ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Reasoned Action Approach ◽  
Belief Elicitation ◽  
Survey Questions

Objectives: Salient belief elicitations (SBEs), informed by the Reasoned Action Approach (RAA), are used to identify 3 sets of beliefs – behavioral, control, and normative – that influence attitudes toward a health behavior. SBEs ask participants about their own beliefs through open-ended questions. We adapted a SBE by focusing on abortion, which is infrequently examined through SBEs; we also included a survey version that asked participants their views on what a hypothetical woman would do if contemplating an abortion. Given these deviations from traditional SBEs, the purpose of this study was to assess if the adapted SBE was understood by participants in English and Spanish through cognitive interviewing. Methods: We examined participants' interpretations of SBE items about abortion to determine if they aligned with the corresponding RAA construct. We administered SBE surveys and conducted cognitive interviews with US adults in both English and Spanish. Results: Participants comprehended the SBE questions as intended. Participants' interpretations of most questions were also in line with the respective RAA construct. Conclusions: SBE survey questions were comprehended well by participants. We discuss areas in which SBE questions can be modified to improve alignment with the underlying RAA construct to assess abortion beliefs.

scholarly journals Design and Analysis of Cognitive Interviews for Comparative Multinational Testing

Field Methods ◽  
2011 ◽  
Vol 23 (4) ◽  
pp. 379-396 ◽  
Author(s):  
Kristen Miller ◽  
Rory Fitzgerald ◽  
José-Luis Padilla ◽  
Stephanie Willson ◽  
Sally Widdop ◽  
...  
Keyword(s):  
Survey Data ◽  
Cross Cultural ◽  
Cognitive Testing ◽  
Cognitive Interviewing ◽  
Evidence Based ◽  
Cognitive Interviews ◽  
Survey Questions ◽  
Method Bias

This article summarizes the work of the Comparative Cognitive Testing Workgroup, an international coalition of survey methodologists interested in developing an evidence-based methodology for examining the comparability of survey questions within cross-cultural or multinational contexts. To meet this objective, it was necessary to ensure that the cognitive interviewing (CI) method itself did not introduce method bias. Therefore, the workgroup first identified specific characteristics inherent in CI methodology that could undermine the comparability of CI evidence. The group then developed and implemented a protocol addressing those issues. In total, 135 cognitive interviews were conducted by participating countries. Through the process, the group identified various interpretive patterns resulting from sociocultural and language-related differences among countries as well as other patterns of error that would impede comparability of survey data.

scholarly journals Evaluation of Data Exchange Process for Interoperability and Impact on Electronic Laboratory Reporting Quality to a State Public Health Agency

2018 ◽  
Vol 10 (2) ◽  
Author(s):  
Sripriya Rajamani ◽  
Ann Kayser ◽  
Emily Emerson ◽  
Sarah Solarz
Keyword(s):  
Public Health ◽  
Data Quality ◽  
Disease Surveillance ◽  
Data Exchange ◽  
Health Program ◽  
Exchange Process ◽  
Quality Of Data ◽  
The Public ◽  
Electronic Laboratory Reporting

Background: Past and present national initiatives advocate for electronic exchange of health data and emphasize interoperability. The critical role of public health in the context of disease surveillance was recognized with recommendations for electronic laboratory reporting (ELR). Many public health agencies have seen a trend towards centralization of information technology services which adds another layer of complexity to interoperability efforts.Objectives: To understand the process of data exchange and its impact on the quality of data being transmitted in the context of electronic laboratory reporting to public health. The study was conducted in context of Minnesota Electronic Disease Surveillance System (MEDSS), the public health information system for supporting infectious disease surveillance in Minnesota. Data Quality (DQ) dimensions by Strong et al., was chosen as the guiding framework for evaluation.Methods: The process of assessing data exchange for electronic lab reporting and its impact was a mixed methods approach with qualitative data obtained through expert discussions and quantitative data obtained from queries of the MEDSS system. Interviews were conducted in an open-ended format from November 2017 through February 2018. Based on these discussions, two high level categories of data exchange process which could impact data quality were identified: onboarding for electronic lab reporting and internal data exchange routing. This in turn comprised of eight critical steps and its impact on quality of data was identified through expert input. This was followed by analysis of data in MEDSS by various criteria identified by the informatics team.Results: All DQ metrics (Intrinsic DQ, Contextual DQ, Representational DQ, and Accessibility DQ) were impacted in the data exchange process with varying influence on DQ dimensions. Some errors such as improper mapping in electronic health records (EHRs) and laboratory information systems had a cascading effect and can pass through technical filters and go undetected till use of data by epidemiologists. Some DQ dimensions such as accuracy, relevancy, value-added data and interpretability are more dependent on users at either end of the data exchange spectrum, the relevant clinical groups and the public health program professionals. The study revealed that data quality is dynamic and on-going oversight is a combined effort by MEDSS Operations Team and Review by Technical and Public Health Program Professionals.Conclusion: With increasing electronic reporting to public health, there is a need to understand the current processes for electronic exchange and their impact on quality of data. This study focused on electronic laboratory reporting to public health and analyzed both on-boarding and internal data exchange processes. Insights gathered from this research can be applied to other public health reporting currently (e.g. immunizations) and will be valuable in planning for electronic case reporting in near future.

scholarly journals Design of a compact grinding machine

2021 ◽  
Vol 2120 (1) ◽  
pp. 012006
Author(s):  
S Ashvinderjit ◽  
W J Kelvin Chew
Keyword(s):  
Ground Surface ◽  
Preparation Process ◽  
The Public ◽  
Proficiency Levels ◽  
Emery Paper ◽  
Test Parameters ◽  
Cost Efficient ◽  
Grinding Machine ◽  
Coarse To Fine

Abstract The availability and cost of tools are among concerns to the public and industries as key operation device used either for commercial or maintenance purposes such precision grinders or polishing machines are usually expensive. Assorted grinding tools in the market differ in terms of cost, required user proficiency levels and have different tool designs that can lead to limitations of flatness of ground surface and significant vibrations with prolonged use. The aim of this study is to develop a design compact grinder that is able to perform precise grinding and polishing while being cost efficient. Both functionality and ergonomic aspects were taken into consideration with market standards derived from datasheets. An initial design of the grinding machine was made using SolidWorks CAD software. A prototype was subsequently constructed and tested by using typical metallurgical sample preparation process with test parameters of grade of grinding paper and usage of lubricant. Microscopic image of ground specimens showed improvement in the quality of grinding with emery paper grades from coarse to fine grits with lubricant during the grinding process. Moreover, the designed compact grinding machine showed better surface finish when compared to using a typical angle grinder on aluminium sample.

Sign in / Sign up

Export Citation Format

Share Document