SME Survey Methodology: Response Rates, Data Quality, and Cost Effectiveness

2003 ◽  
Vol 28 (2) ◽  
pp. 163-172 ◽  
Author(s):  
Rick Newby ◽  
John Watson ◽  
David Woodliff

Cost effective data collection is an important methodological issue for small and medium enterprise (SME) researchers. There is a generally held view that mail surveys are the most efficient means of collecting empirical data, despite the potential difficulties associated with low response rates. To enhance the usefulness of mail surveys, researchers have suggested a variety of strategies aimed at improving response rates. While previous studies have examined the effect on response rates of many of these strategies, their impact on data quality and on the cost effectiveness of data collection is less well understood. This study evaluates four response–inducing strategies (printing the survey instrument on colored paper, telephone pre–notification, payment of a monetary incentive, and a follow–up mailing) in terms of their effect on data quality, response rates, and cost effectiveness for a population of SMEs.

BMJ Open ◽  
2020 ◽  
Vol 10 (10) ◽  
pp. e036533
Author(s):  
Hilde Hestad Iversen ◽  
Olaf Holmboe ◽  
Oyvind Bjertnaes

ObjectiveThe standard data-collection procedure in the Norwegian national patient experience survey programme is post-discharge mail surveys, which include a pen-and-paper questionnaire with the option to answer electronically. A purely electronic protocol has not previously been explored in Norway. The aim of this study was to compare response rates, background characteristics, data quality and main study results for a survey of patient experiences with general practitioners (GPs) administered by the standard mail data-collection procedure and a web-based approach.DesignCross-sectional survey.SettingGP offices in Norway.ParticipantsThe sample consisted of 6999 patients aged 16 years and older registered with a GP in November 2018.InterventionBased on a three-stage sampling design, 6999 patients of GPs aged 16 or older were randomised to one of two survey administration protocols: Group A, who were mailed an invitation with both a pen-and-paper including an electronic response option (n=4999) and Group B, who received an email invitation with electronic response option (n=2000).Main outcome measuresResponse rates, background characteristics, data quality and main study results.ResultsThe response rate was markedly higher for the mail survey (42.6%) than for the web-based survey (18.3%). A few of the background variables differed significantly between the two groups, but the data quality and patient-reported experiences were similar.ConclusionsWeb-based surveys are faster and less expensive than standard mail surveys, but their low response rates and coverage problems threaten their usefulness and legitimacy. Initiatives to increase response rates for web-based data collection and strategies for tailoring data collection to different groups should be key elements in future research.


Author(s):  
Michael S. Wogalter ◽  
Meredith F. Yarbrough ◽  
David W. Martin

The use of email and fax communications has increased dramatically over the last decade. They are now commonplace methods of information exchange. Most research involving questionnaires has used postal mail to deliver and return the surveys from recipients who might not otherwise be reached through live administration. A frequent methodological issue with mail surveys is low levels of return rates. The present research compared the return rates of a survey that was sent by mail, email or fax. Participants could return the survey by any of the same three methods. The results showed that postal mail and email exhibited higher return rates than facsimiles and that the method of return tended to be the same method in which the questionnaire was originally sent. Implications of these results for survey research are discussed.


Author(s):  
Cristine D. Delnevo ◽  
Binu Singh

Abstract Background: Achieving a high response rate for physicians has been challenging and with response rates declining in recent years, innovative methods are needed to increase rates. An emerging concept in survey methodology has been web-push survey delivery. In this delivery method, contact is made by mail to request a response by web. This study explored the feasibility of a web-push survey on a national sample of physicians. Methods: 1000 physicians across six specialties were randomly assigned to a mail only or web-push survey delivery. Each mode consisted of four contacts including an initial mailing, reminder postcard, and two additional follow-ups. Response rates were calculated using AAPOR’s response rate 3 calculation. Data collection occurred between Febuary – April 2018 and analyzed March 2019. Results: Overall reponse rates for the mail only vs. web-push survey delivery were comparable (51.2% vs. 52.8%). Higher response rates across all demographics were seen in the web-push delivery with the exception of pulmonary/critical care and physicians over the age of 65. The web-push survey yielded a greater response after the first mailing requiring fewer follow-up contacts resulting in a more cost-effective delivery. Conclusions: A web-push mail survey is effective in achieveing a comparable response rate to traditional mail only delivery for physicians. The web-push survey was more efficient in terms of cost and in receiving responses in a more timely manner. Future research should explore the efficiency of a web-push survey delivery across various health care provider populations.


1986 ◽  
Vol 59 (1) ◽  
pp. 279-283 ◽  
Author(s):  
Melissa Elkind ◽  
Georgiana Shick Tryon ◽  
Anthony J. De Vito

Two variables thought to influence return rates for mail surveys, type of covering envelope (plain or university printed) and type of return envelope (postage stamp or business reply) produced no statistically significant differences in return rates. Therefore, anticipated response rates and cost-effectiveness become important considerations when choosing methods for mail survey.


Blood ◽  
2021 ◽  
Vol 138 (Supplement 1) ◽  
pp. 751-751
Author(s):  
George Goshua ◽  
Pranay Sinha ◽  
Lauren Pischel ◽  
Alfred Ian Lee ◽  
Adam Cuker

Abstract Introduction: Immune thrombocytopenia (ITP) is an autoimmune disorder characterized by accelerated platelet clearance and impaired platelet production. Up to 75% of cases in adults assume a chronic course. Standard second-line treatment options include rituximab, thrombopoietin receptor agonists (TPO-RAs), and splenectomy. The 2019 American Society of Hematology (ASH) guidelines provide three dichomotous evaluations of these treatments with conditional recommendations for 1) splenectomy or TPO-RA, 2) rituximab over splenectomy, and 3) TPO-RA over rituximab. The guideline panel noted that there were no studies available to evaluate the cost-effectiveness of these therapies. We sought to address this knowledge gap by conducting the first cost-effectiveness analysis of second-line therapies for chronic ITP. Methods: We built a Markov model comparing the cost-effectiveness of all six treatment pathways utilizing rituximab, TPO-RA (romiplostim or eltrombopag), and splenectomy. We assumed a median age of 50 at diagnosis and a 20-year time-horizon. Costs were assessed from the health system perspective. Effectiveness was calculated in quality-adjusted life-years (QALYs). The costs of splenectomy treatment included the cost of surgery, postoperative care, accessory spleen imaging and repeat splenectomy, treatment for post-splenectomy sepsis and thromboembolism. The annual risks of post-splenectomy sepsis and thromboembolism were assumed to be the highest reported, every post-splenectomy infectious complication was assumed to be severe septic shock, and patients with thromboembolism accrued the costs of indefinite anticoagulation. To minimize bias against TPO-RAs, TPO-RA therapy was assumed to have no adverse events, a constant high overall response without any loss of effectiveness, and the highest reported rate of successful TPO-RA therapy discontinuation after two years, which we assumed to be permanent. Rituximab treatment was assumed to have no adverse events and overall response rates as previously reported. Cost-effectiveness of each treatment pathway was calculated as the incremental cost-effectiveness ratio (ICER), calculated as ratio of costs to QALYs. The ICER was compared against a 2019 US willingness-to-pay (WTP) threshold of $195,300. We then performed one-way deterministic sensitivity analyses varying all parameters including the costs of splenectomy, TPO-RA, rituximab, splenectomy complications, splenectomy complete response rates, TPO-RA and rituximab overall response rates, utilities of the well and diseases states, annual post-splenectomy septic shock mortality and perioperative splenectomy mortality. We concluded with a probabilistic sensitivity analysis running 10,000 Monte Carlo simulations. Results: The most cost-effective treatment pathway was #5 (splenectomy->rituximab->TPO-RA; Figure). The next most cost-effective pathway was #4 (rituximab followed by splenectomy and then TPO-RA therapy), with an ICER of $369,289. All four remaining treatment pathways (#1-3, 6) utilizing TPO-RA therapy early (first or second) had an ICER above $1 million, far above the US WTP of $195,300, and/or were dominated. Of these, pathways #1 and #3 were externally dominated and #2 was absolutely dominated. No parameter change in one-way deterministic sensitivity analysis in any of the 4 pathways featuring TPO-RA early brought down the ICER to under $1 million. In the probabilistic sensitivity analysis, pathway #5 was favored in 100% of 10,000 Monte Carlo simulations. The cost of TPO-RA would have to be decreased to under $20,000 annually (e.g., >80% reduction in the cost of eltrombopag or romiplostim) before it could become cost-effective in any TPO-early treatment pathway. Conclusion: Four treatment pathways (#1-#4) are consistent and two pathways (#5-#6) are at variance with the ASH guidelines. Although it does not align with the ASH guidelines, pathway #5 (splenectomy->rituximab->TPO-RA) was most cost-effective. Over a 20-year time-horizon, all pathways featuring early use of a TPO-RA exceeded an ICER >$1 million or were dominated. Because our model was designed to maximize the cost-effectiveness of TPO-RA, it is likely that the actual ICER of pathways featuring early use of TPO-RA are higher than what we report here. Preferred second-line treatment strategies in adults with chronic ITP may be worth considering in light of our findings. Figure 1 Figure 1. Disclosures Cuker: Synergy: Consultancy; Novo Nordisk: Research Funding; Alexion: Research Funding; UpToDate: Patents & Royalties; Novartis: Research Funding; Bayer: Research Funding; Takeda: Research Funding; Pfizer: Research Funding; Sanofi: Research Funding; Spark Therapeutics: Research Funding.


2020 ◽  
Author(s):  
Atinkut Alamirrew Zeleke ◽  
Tolga Naziyok ◽  
Fleur Fritz ◽  
Lara Christianson ◽  
Rainer Röhrig

BACKGROUND Population-level survey (PLS) is an essential standard method used in public health research. It supports to quantify sociodemographic events and support public health policy development and intervention designs with evidence. During survey, data collection mechanisms seem the most determinant to avoid mistakes before they happen. The use of electronic devices such as smartphones and tablet computers improve the quality and cost-effectiveness of public health surveys. However, there is a lack of systematically analyzed evidence to show the potential impact of electronic-based data collection tools on data quality and cost reduction in interviewer-administered surveys compared to the standard paper-based data collection system OBJECTIVE This systematic review aims to evaluate the impact of interviewer-administered electronic device data collection methods concerning data quality and cost reduction in PLS compared to the traditional paper-based methods. METHODS A systematic search was conducted in MEDLINE, CINAHL, PsycINFO, the Web of Science, EconLit and Cochrane CENTRAL, and CDSR to identify relevant studies from 2008 to 2018. We included randomized and non-randomized studies that examine data quality and cost reduction outcomes. Moreover, usability, user experience, and usage parameters from the same studies were included. Two independent authors screened the title, abstract, and finally extracted data from the included papers. A third author mediated in case of disagreement. The review authors used EndNote for de-duplication and Rayyan for screening RESULTS The search strategy from the electronic databases found 3,817 articles. After de-duplication, 2,533 articles were screened, and 14 articles fulfilled the inclusion criteria. None of the studies was designed as a randomized control trial. Most of the studies have a quasi-experimental design, like comparative experimental evaluation studies nested on the other ongoing cross-sectional surveys. 4 comparative evaluations, 2 pre-post intervention comparative evaluation, 2 retrospectives comparative evaluation, and 4 one arm non-comparative studies were included in our review. Meta-analysis was not possible because of the heterogeneity in study design, the type, and level of outcome measurements and the study settings. Individual article synthesis showed that data from electronic data collection systems possessed good quality data and delivered faster when compared to the paper-based data collection system. Only two studies linked the cost and data quality outcomes to describe the cost-effectiveness of electronic-based data collection systems. Despite the poor economic evaluation qualities, most of the reported results were in favor of EDC for the large-scale surveys. The field data collectors reported that an electronic data collection system was a feasible, acceptable and preferable tool for their work. Onsite data error prevention, fast data submission, and easy to handle devices were the comparative advantages of electronic data collection systems. Technical difficulties, accidental data loss, device theft, security concerns, power surges, and internet connection problems were reported as challenges during the implementation. CONCLUSIONS Though positive evidence existed about the comparative advantage of electronic data capture over paper-based tools, the included studies were not methodologically rigorous enough to combine. We need more rigorous studies that demonstrate the comparative evidence of paper and electronic-based data collection systems in public health surveys on data quality, work efficiency, and cost reduction CLINICALTRIAL The review protocol is registered in the International Prospective Register for Systematic Reviews (PROSPERO) CRD42018092259. The protocol of this article was also pre-published (JMIR Res Protoc 2019;8(1): e10678 doi:10.2196/10678).


2017 ◽  
Vol 59 (2) ◽  
pp. 199-220
Author(s):  
G.W. Roughton ◽  
Iain Mackay

This paper investigates whether a ‘wisdom of the crowd’ approach might offer an alternative to recent political polls that have raised questions about survey data quality. Data collection costs have become so low that, as well as the question of data quality, concerns have also been raised about low response rates, professional respondents and respondent interaction. There are also uncertainties about self-selecting ‘samples’. This paper looks at more than 100 such surveys and reports that, in five out of the six cases discussed, £0.08p interviews delivered results in line with known outcomes. The results discussed in the paper show that such interviews are not a waste of money.


2018 ◽  
Author(s):  
Atinkut Alamirrew Zeleke ◽  
Tolga Naziyok ◽  
Fleur Fritz ◽  
Rainer Röhrig

BACKGROUND Population-level survey is an essential standard method used in public health research to quantify sociodemographic events and support public health policy development and intervention designs with evidence. Although all steps in the survey can contribute to the data quality parameters, data collection mechanisms seem the most determinant, as they can avoid mistakes before they happen. The use of electronic devices such as smartphones and tablet computers improve the quality and cost-effectiveness of public health surveys. However, there is lack of systematically analyzed evidence to show the potential impact on data quality and cost reduction of electronic-based data collection tools in interviewer-administered surveys. OBJECTIVE This systematic review aims to evaluate the impact of interviewer-administered electronic device data collection methods concerning data quality and cost reduction in population-level surveys compared with the traditional paper-based methods. METHODS We will conduct a systematic search on Medical Literature Analysis and Retrieval System Online, PubMed, CINAHL, PsycINFO, Global Health, Trip, ISI Web of Science, and Cochrane Library for studies from 2007 to 2018 to identify relevant studies. The review will include randomized and nonrandomized studies that examine data quality and cost reduction outcomes. Moreover, usability, user experience, and usage parameters from the same study will be summarized. Two independent authors will screen the title and abstract. A third author will mediate in cases of disagreement. If the studies are considered to be combinable with minimal heterogeneity, we will perform a meta-analysis. RESULTS The preliminary search in PubMed and Web of Science showed 1491 and 979 resulting hits of articles, respectively. The review protocol is registered in the International Prospective Register of Systematic Reviews (CRD42018092259). We anticipate January 30, 2019, to be the finishing date. CONCLUSIONS This systematic review will inform policymakers, investors, researchers, and technologists about the impact of an electronic-based data collection system on data quality, work efficiency, and cost reduction. CLINICALTRIAL PROSPERO CRD42018092259; http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID= CRD42018092259 INTERNATIONAL REGISTERED REPORT PRR1-10.2196/10678


2019 ◽  
Vol 8 (5) ◽  
pp. 821-831
Author(s):  
Matthew Debell ◽  
Natalya Maisel ◽  
Brad Edwards ◽  
Michelle Amsbary ◽  
Vanessa Meldener

Abstract In mail surveys and in advance letters for surveys in other modes, it is common to include a prepaid incentive of a small amount such as $5. However, when letters are addressed generically (such as to “Resident”), advance letters may be thrown away without being opened, so the enclosed cash is wasted and the invitation or advance letter is ineffective. This research note describes results of an experiment using a nationally representative sample of 4,725 residential addresses to test a new way of letting mail recipients know their letter contains cash and is therefore worth opening: an envelope with a window revealing $5, so the cash is clearly visible from outside the sealed envelope. We also tested the USPS for evidence of theft, and we compared First Class and Priority Mail postage. We found no evidence of theft. We found no difference in response rates between Priority Mail and First Class, making First Class much more cost-effective, and we found that visible money increased the response rate to a mail survey from 42.6 to 46.9 percent, at no significant cost.


2020 ◽  
Vol 36 (4) ◽  
pp. 1007-1014
Author(s):  
Chee You Lee ◽  
Ab. Rahman bin Mohamad ◽  
Rozita binti Misran ◽  
Halina binti Abdul Hamid

Response rates, data quality and data confidentiality were identified as the critical success factors in Malaysia’s short-term economic survey. According to the facts, the Department of Statistics Malaysia (DOSM) has outlined a few initiatives in its strategic plan (2011–2015) and transformation plan (2016–2020) over the past ten years to improve and modernise data collection, processing and dissemination methods. This paper presents the practices of two DOSM short-term economic surveys i.e the monthly manufacturing survey (MM) and the quarterly construction survey (QCS) to raise the response rate, enhance data quality and manage data confidentiality by e-survey. The factors influences the participation of the MM and QCS e-survey are discussed. This empirical study shows that the response rates of the both short-term surveys have improved through the e-survey. Comparison of data received by various survey modes also indicated that the e-survey provides the best prospects for reducing non-sampling error and protecting data against unintentional disclosure.


Sign in / Sign up

Export Citation Format

Share Document