Are Interviews Costing £0.08 a Waste of Money? Reviewing Google Surveys for ‘Wisdom of the Crowd’ Projects

2017 ◽  
Vol 59 (2) ◽  
pp. 199-220
Author(s):  
G.W. Roughton ◽  
Iain Mackay
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Survey Data ◽  
Response Rates ◽  
Quality Data ◽  
Collection Costs

This paper investigates whether a ‘wisdom of the crowd’ approach might offer an alternative to recent political polls that have raised questions about survey data quality. Data collection costs have become so low that, as well as the question of data quality, concerns have also been raised about low response rates, professional respondents and respondent interaction. There are also uncertainties about self-selecting ‘samples’. This paper looks at more than 100 such surveys and reports that, in five out of the six cases discussed, £0.08p interviews delivered results in line with known outcomes. The results discussed in the paper show that such interviews are not a waste of money.

Abstract WP389: Understanding the Clinicians’ Experiences in Collecting Stroke Rehabilitation Intensity Data Within Ontario

Stroke ◽  
2016 ◽  
Vol 47 (suppl_1) ◽  
Author(s):  
Elizabeth Linkewich ◽  
Janine Theben ◽  
Amy Maebrae-Waller ◽  
Shelley Huffman ◽  
Jenn Fearn ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Thematic Analysis ◽  
Best Practice ◽  
Practice Change ◽  
Quality Data ◽  
Electronic Survey ◽  
Workload Measurement ◽  
Survey Results ◽  
Patients Experience

Background and Issues: The collection and reporting of Rehabilitation Intensity (RI) in a national rehabilitation database was mandated on April 1, 2015 for all stroke patients within Ontario, to support evaluation of stroke best practice implementation. RI includes minutes of direct task-specific therapy patients experience per day. This requires a shift in thinking from capturing the clinician’s time spent in therapy to the patient perspective. To ensure that high quality data is collected, it was important to understand clinicians’ experiences in collecting RI data. Purpose: To identify enablers and barriers to RI data collection in order to inform the development of resources to support clinicians. Methods: A 12-item electronic survey was developed by an Ontario Stroke Network (OSN) task group to evaluate the clinician experience of RI data collection (including: demographics, barriers, enablers, education, resources, and practice change). The survey was distributed via SurveyMonkey® and sent to clinicians from 48 hospitals, 3 weeks post implementation of RI data collection. Analyses involved descriptive statistics and thematic analysis. Results: Three hundred and twenty-one clinicians from 47 hospitals responded to the survey. Survey results suggest RI data collection is feasible; seventy-one percent of clinicians report it takes 10 minutes or less to enter RI data. Thematic analysis identified: 5 common challenges with most frequently reported relating to data quality, 30% (N=358) and 6 common enablers with most frequently reported relating to ease of collecting RI data through workload measurement systems, 50% (N=46). Suggestions for educational resources included tools for identifying what is included in RI and the provision of education (e.g. webinars). Conclusions: RI data collection is feasible for clinicians. Education and resources developed should support key challenges and enablers identified by clinicians - to enhance data quality and the consistency of RI collection. As RI data fields are available through a national rehabilitation database, this work sets the foundation for other provinces interested in the systematic collection and reporting of RI data.

SME Survey Methodology: Response Rates, Data Quality, and Cost Effectiveness

2003 ◽  
Vol 28 (2) ◽  
pp. 163-172 ◽  
Author(s):  
Rick Newby ◽  
John Watson ◽  
David Woodliff
Keyword(s):  
Cost Effectiveness ◽  
Data Collection ◽  
Data Quality ◽  
Survey Methodology ◽  
Methodological Issue ◽  
Cost Effective ◽  
Response Rates ◽  
Mail Surveys ◽  
Medium Enterprise ◽  
Quality Response

Cost effective data collection is an important methodological issue for small and medium enterprise (SME) researchers. There is a generally held view that mail surveys are the most efficient means of collecting empirical data, despite the potential difficulties associated with low response rates. To enhance the usefulness of mail surveys, researchers have suggested a variety of strategies aimed at improving response rates. While previous studies have examined the effect on response rates of many of these strategies, their impact on data quality and on the cost effectiveness of data collection is less well understood. This study evaluates four response–inducing strategies (printing the survey instrument on colored paper, telephone pre–notification, payment of a monetary incentive, and a follow–up mailing) in terms of their effect on data quality, response rates, and cost effectiveness for a population of SMEs.

scholarly journals Survey Fatigue During the COVID-19 Pandemic: An Analysis of Neurosurgery Survey Response Rates

2021 ◽  
Vol 8 ◽  
Author(s):  
Rosaline de Koning ◽  
Abdullah Egiz ◽  
Jay Kotecha ◽  
Ana Catinca Ciuculete ◽  
Setthasorn Zhi Yang Ooi ◽  
...  
Keyword(s):  
Data Collection ◽  
Survey Data ◽  
Data Extraction ◽  
Response Rates ◽  
Quality Of Data ◽  
Research Activity ◽  
Exact Test ◽  
Wilcoxon Rank Sum Test ◽  
Geographical Regions ◽  
Bivariable Analysis

Background: The COVID-19 pandemic has caused a surge in research activity while restricting data collection methods, leading to a rise in survey-based studies. Anecdotal evidence suggests this increase in neurosurgical survey dissemination has led to a phenomenon of survey fatigue, characterized by decreased response rates and reducing the quality of data. This paper aims to analyze the effect of COVID-19 on neurosurgery surveys and their response rates, and suggest strategies for improving survey data collection.Methods: A search was conducted on March 20, 2021, on Medline and EMBASE. This included the terms “neurosurgery,” “cranial surgery,” “spine surgery,” and “survey” and identified surveys written in English, on a neurosurgical topic, distributed to neurosurgeons, trainees, and medical students. Results were screened by two authors according to these inclusion criteria, and included articles were used for data extraction, univariable, and bivariable analysis with Fisher's exact-test, Wilcoxon rank-sum test, and Spearman's correlation.Results: We included 255 articles in our analysis, 32.3% of which were published during the COVID-19 pandemic. Surveys had an average of 25.6 (95% CI = 22.5–28.8) questions and were mostly multiple choice (78.8%). They were disseminated primarily by email (75.3%, 95% CI = 70.0–80.6%) and there was a significant increase in dissemination via social media during the pandemic (OR = 3.50, 95% CI = 1.30–12.0). COVID-19 surveys were distributed to more geographical regions than pre-pandemic surveys (2.1 vs. 1.5, P = 0.01) and had higher total responses (247.0 vs. 206.4, P = 0.01), but lower response rates (34.5 vs. 51.0%, P < 0.001) than pre-COVID-19 surveys.Conclusion: The rise in neurosurgical survey distribution during the COVID-19 pandemic has led to survey fatigue, reduced response rates, and data collection quality. We advocate for population targeting to avoid over-researching, collaboration between research teams to minimize duplicate surveys, and communication with respondents to convey study importance, and we suggest further strategies to improve response rates in neurosurgery survey data collection.

Effects of Sequential Prepaid Incentives on Response Rates, Data Quality, Sample Representativeness, and Costs in a Mail Survey of Physicians

2020 ◽  
pp. 016327872095818
Author(s):  
Jennifer Dykema ◽  
John Stevenson ◽  
Nadia Assad ◽  
Chad Kniss ◽  
Catherine A. Taylor
Keyword(s):  
Data Quality ◽  
Mail Survey ◽  
Response Rates ◽  
Quality Data ◽  
Item Nonresponse ◽  
Sample Representativeness ◽  
Initial Mailing ◽  
Cash Incentive ◽  
Group 10 ◽  
Group 5

While collecting high quality data from physicians is critical, response rates for physician surveys are frequently low. A proven method for increasing response in mail surveys is to provide a small, prepaid monetary incentive in the initial mailing. More recently, researchers have begun experimenting with adding a second cash incentive in a follow-up contact in order to increase participation among more reluctant respondents. To assess the effects of sequential incentives on response rates, data quality, sample representativeness, and costs, physicians (N = 1,500) were randomly assigned to treatments that crossed the amount of a first ($5 or $10) and second ($0, $5, or $10) incentive to form the following groups: Group $5/$5; Group $5/$10; Group $10/$0; Group $10/$5; and Group $10/$10. Overall, second incentives were associated with higher response rates and lower costs per completed survey, and while they had no effect on item nonresponse, they increased sample representativeness.

scholarly journals EIGER detector: application in macromolecular crystallography

2016 ◽  
Vol 72 (9) ◽  
pp. 1036-1048 ◽  
Author(s):  
Arnau Casanas ◽  
Rangana Warshamanage ◽  
Aaron D. Finke ◽  
Ezequiel Panepucci ◽  
Vincent Olieric ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Data Acquisition ◽  
Dead Time ◽  
Single Photon ◽  
Photon Counting ◽  
Quality Data ◽  
Macromolecular Crystallography ◽  
Single Photon Counting ◽  
Optimal Rotation

The development of single-photon-counting detectors, such as the PILATUS, has been a major recent breakthrough in macromolecular crystallography, enabling noise-free detection and novel data-acquisition modes. The new EIGER detector features a pixel size of 75 × 75 µm, frame rates of up to 3000 Hz and a dead time as low as 3.8 µs. An EIGER 1M and EIGER 16M were tested on Swiss Light Source beamlines X10SA and X06SA for their application in macromolecular crystallography. The combination of fast frame rates and a very short dead time allows high-quality data acquisition in a shorter time. The ultrafine φ-slicing data-collection method is introduced and validated and its application in finding the optimal rotation angle, a suitable rotation speed and a sufficient X-ray dose are presented. An improvement of the data quality up to slicing at one tenth of the mosaicity has been observed, which is much finer than expected based on previous findings. The influence of key data-collection parameters on data quality is discussed.

scholarly journals Mode Effects on Data Quality and Measurement in a Mixed-Mode Time-Use Survey

2021 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Elena Mylona
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Time Use ◽  
Mixed Mode ◽  
Large Scale ◽  
New Technologies ◽  
Quality Data ◽  
Mode Effects ◽  
Time Use Survey ◽  
Time Diaries

Recent years have witnessed an increasing interest in the use of new technologies for time-use data collection, driven by their potential to reduce survey administration costs and improve data quality. However, despite the steady growth of studies that employ web and app time diaries, there is little research on their comparability with traditional paper-administered diaries that have long been regarded as the “gold standard” for measurement in time-use research. This paper rectifies this omission by investigating diary mode effects on data quality and measurement, drawing on data from a mixed-mode large-scale time diary study of adolescents in the United Kingdom. After controlling for selection effects, we find that web and app diaries yield higher quality data than paper diaries, which attests to the potential of new technologies in facilitating diary completion. At the same time, our analysis of broad time-use domains does not find substantial mode effects on measurement for the majority of daily activity categories. We conclude by discussing avenues for future methodological research and implications for time-use data collection.

scholarly journals An Overview of ARM Program Climate Research Facility Data Quality Assurance

2008 ◽  
Vol 2 (1) ◽  
pp. 192-216 ◽  
Author(s):  
R.A. Peppler ◽  
C.N. Long ◽  
D.L. Sisterson ◽  
D.D. Turner ◽  
C.P. Bahrmann ◽  
...  
Keyword(s):  
Quality Assurance ◽  
Data Collection ◽  
Data Quality ◽  
Data Stream ◽  
Value Added ◽  
Quality Data ◽  
Quality Assurance Program ◽  
Research Facility ◽  
Data Set ◽  
Climate Research

We present an overview of key aspects of the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) data quality assurance program. Processes described include instrument deployment and calibration; instrument and facility maintenance; data collection and processing infrastructure; data stream inspection and assessment; problem reporting, review and resolution; data archival, display and distribution; data stream reprocessing; engineering and operations management; and the roles of value-added data processing and targeted field campaigns in specifying data quality and characterizing field measurements. The paper also includes a discussion of recent directions in ACRF data quality assurance. A comprehensive, end-to-end data quality assurance program is essential for producing a high-quality data set from measurements made by automated weather and climate networks. The processes developed during the ARM Program offer a possible framework for use by other instrumentation- and geographically-diverse data collection networks and highlight the myriad aspects that go into producing research-quality data.

scholarly journals The shape of and solutions to the MTurk quality crisis

2020 ◽  
Vol 8 (4) ◽  
pp. 614-629 ◽  
Author(s):  
Ryan Kennedy ◽  
Scott Clifford ◽  
Tyler Burleigh ◽  
Philip D. Waggoner ◽  
Ryan Jewell ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Treatment Effects ◽  
Mechanical Turk ◽  
Quality Data ◽  
Social Scientists ◽  
Amazon's Mechanical Turk ◽  
Gain Access

AbstractAmazon's Mechanical Turk is widely used for data collection; however, data quality may be declining due to the use of virtual private servers to fraudulently gain access to studies. Unfortunately, we know little about the scale and consequence of this fraud, and tools for social scientists to detect and prevent this fraud are underdeveloped. We first analyze 38 studies and show that this fraud is not new, but has increased recently. We then show that these fraudulent respondents provide particularly low-quality data and can weaken treatment effects. Finally, we provide two solutions: an easy-to-use application for identifying fraud in the existing datasets and a method for blocking fraudulent respondents in Qualtrics surveys.

Using e-surveys in Malaysia to modernise data collection in short-term economic surveys1

2020 ◽  
Vol 36 (4) ◽  
pp. 1007-1014
Author(s):  
Chee You Lee ◽  
Ab. Rahman bin Mohamad ◽  
Rozita binti Misran ◽  
Halina binti Abdul Hamid
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Success Factors ◽  
Sampling Error ◽  
Response Rates ◽  
Data Confidentiality ◽  
Short Term ◽  
Critical Success ◽  
Economic Survey ◽  
Survey Comparison

Response rates, data quality and data confidentiality were identified as the critical success factors in Malaysia’s short-term economic survey. According to the facts, the Department of Statistics Malaysia (DOSM) has outlined a few initiatives in its strategic plan (2011–2015) and transformation plan (2016–2020) over the past ten years to improve and modernise data collection, processing and dissemination methods. This paper presents the practices of two DOSM short-term economic surveys i.e the monthly manufacturing survey (MM) and the quarterly construction survey (QCS) to raise the response rate, enhance data quality and manage data confidentiality by e-survey. The factors influences the participation of the MM and QCS e-survey are discussed. This empirical study shows that the response rates of the both short-term surveys have improved through the e-survey. Comparison of data received by various survey modes also indicated that the e-survey provides the best prospects for reducing non-sampling error and protecting data against unintentional disclosure.

Sign in / Sign up

Export Citation Format

Share Document