scholarly journals Mode Effects on Data Quality and Measurement in a Mixed-Mode Time-Use Survey

2021 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Elena Mylona
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Time Use ◽  
Mixed Mode ◽  
Large Scale ◽  
New Technologies ◽  
Quality Data ◽  
Mode Effects ◽  
Time Use Survey ◽  
Time Diaries

Recent years have witnessed an increasing interest in the use of new technologies for time-use data collection, driven by their potential to reduce survey administration costs and improve data quality. However, despite the steady growth of studies that employ web and app time diaries, there is little research on their comparability with traditional paper-administered diaries that have long been regarded as the “gold standard” for measurement in time-use research. This paper rectifies this omission by investigating diary mode effects on data quality and measurement, drawing on data from a mixed-mode large-scale time diary study of adolescents in the United Kingdom. After controlling for selection effects, we find that web and app diaries yield higher quality data than paper diaries, which attests to the potential of new technologies in facilitating diary completion. At the same time, our analysis of broad time-use domains does not find substantial mode effects on measurement for the majority of daily activity categories. We conclude by discussing avenues for future methodological research and implications for time-use data collection.

Modelling Mode Effects for a Panel Survey in Transition

2021 ◽  
pp. 63-88
Author(s):  
Paul P. Biemer ◽  
Kathleen Mullan Harris ◽  
Dan Liao ◽  
Brian J. Burke ◽  
Carolyn Tucker Halpern
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Mixed Mode ◽  
Good News ◽  
Longitudinal Change ◽  
Nonresponse Bias ◽  
Two Phase ◽  
Mode Effects ◽  
Analytical Results ◽  
Weighting Adjustments

Funding reductions combined with increasing data-collection costs required that Wave V of the USA’s National Longitudinal Study of Adolescent to Adult Health (Add Health) abandon its traditional approach of in-person interviewing and adopt a more cost-effective method. This approach used the mail/web mode in Phase 1 of data collection and in-person interviewing for a random sample of nonrespondents in Phase 2. In addition, to facilitate the comparison of modes, a small random subsample served as the control and received the traditional in-person interview. We show that concerns about reduced data quality as a result of the redesign effort were unfounded based on findings from an analysis of the survey data. In several important respects, the new two-phase, mixed-mode design outperformed the traditional design with greater measurement accuracy, improved weighting adjustments for mitigating the risk of nonresponse bias, reduced residual (or post-adjustment) nonresponse bias, and substantially reduced total-mean-squared error of the estimates. This good news was largely unexpected based upon the preponderance of literature suggesting data quality could be adversely affected by the transition to a mixed mode. The bad news is that the transition comes with a high risk of mode effects for comparing Wave V and prior wave estimates. Analytical results suggest that significant differences can occur in longitudinal change estimates about 60 % of the time purely as an artifact of the redesign. This begs the question: how, then, should a data analyst interpret significant findings in a longitudinal analysis in the presence of mode effects? This chapter presents the analytical results and attempts to address this question.

scholarly journals Data Quality in Web and App Diaries: A Person-Level Comparison

2021 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Elena Mylona
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Large Scale ◽  
New Technologies ◽  
Mode Selection ◽  
Time Period ◽  
Level Data ◽  
Data Collection Instrument ◽  
Respondent Fatigue ◽  
Over Time

The time-use diary is a complex and burdensome data collection instrument. This can negatively affect data quality, leading to less detailed and/or inaccurate activity reporting as the surveyed time period unfolds. However, it can also be argued that data quality may actually improve over time as respondents become more familiar with the diary instrument format and more interested in the diary task. These competing hypotheses have only been partially tested on data from paper and telephone-administered diaries, which are traditionally used for large-scale data collection. Less is known about self-administered modes that make use of new technologies, despite their increasing popularity among researchers. This research note rectifies this omission by comparing diary quality in self-administered web and app diaries, drawing on data from the Millennium Cohort Study. We construct a person-level data quality typology, using information on missing data, episode changes, and reporting of key daily activity domains. Results show significant mode differences on person-level data quality, after controlling for characteristics known to influence diary mode selection and data quality. App diarists were more likely to return two diaries of inconsistent quality. Both respondent fatigue and improvement of completion over time appear more common among app diarists.

scholarly journals Data Quality in Web and App Diaries: A Person-Level Comparison

2021 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Elena Mylona
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Large Scale ◽  
New Technologies ◽  
Mode Selection ◽  
Time Period ◽  
Level Data ◽  
Data Collection Instrument ◽  
Respondent Fatigue ◽  
Over Time

The time-use diary is a complex and burdensome data collection instrument. This can negatively affect data quality, leading to less detailed and/or inaccurate activity reporting as the surveyed time period unfolds. However, it can also be argued that data quality may actually improve over time as respondents become more familiar with the diary instrument format and more interested in the diary task. These competing hypotheses have only been partially tested on data from paper and telephone-administered diaries, which are traditionally used for large-scale data collection. Less is known about self-administered modes that make use of new technologies, despite their increasing popularity among researchers. This research note rectifies this omission by comparing diary quality in self-administered web and app diaries, drawing on data from the Millennium Cohort Study. We construct a person-level data quality typology, using information on missing data, episode changes, and reporting of key daily activity domains. Results show significant mode differences on person-level data quality, after controlling for characteristics known to influence diary mode selection and data quality. App diarists were more likely to return two diaries of inconsistent quality. Both respondent fatigue and improvement of completion over time appear more common among app diarists.

scholarly journals Using New Technologies for Time Diary Data Collection: Instrument Design and Data Quality Findings from a Mixed-Mode Pilot Survey

2017 ◽  
Vol 137 (1) ◽  
pp. 379-390 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Kimberly Fisher ◽  
Emily Gilbert ◽  
Lisa Calderwood ◽  
Tom Huskinson ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Mixed Mode ◽  
New Technologies ◽  
Instrument Design ◽  
Pilot Survey ◽  
Time Diary ◽  
Diary Data ◽  
Data Collection Instrument

scholarly journals How to Seize the Opportunities of New Technologies in Life Cycle Analysis Data Collection: A Case Study of the Dutch Dairy Farming Sector

Challenges ◽  
2019 ◽  
Vol 10 (1) ◽  
pp. 8 ◽  
Author(s):  
Eric Mieras ◽  
Anne Gaasbeek ◽  
Daniël Kan
Keyword(s):  
Life Cycle ◽  
Data Collection ◽  
Large Scale ◽  
Success Factors ◽  
New Technologies ◽  
Social Innovation ◽  
Real Life ◽  
Mass Scale ◽  
Quality Data

Technologies such as blockchain, big data, and the Internet of Things provide new opportunities for improving and scaling up the collection of life cycle inventory (LCI) data. Unfortunately, not all new technologies are adopted, which means that their potential is not fully exploited. The objective of this case study is to show how technological innovations can contribute to the collection of data and the calculation of carbon footprints at a mass scale, but also that technology alone is not sufficient. Social innovation is needed in order to seize the opportunities that these new technologies can provide. The result of the case study is real-life, large-scale data collected from the entire Dutch dairy sector and the calculation of each individual farm’s carbon footprint. To achieve this, it was important to (1) identify how members of a community can contribute, (2) link their activities to the value it brings them, and (3) consider how to balance effort and result. The case study brought forward two key success factors in order to achieve this: (1) make it easy to integrate data collection in farmers’ daily work, and (2) show the benefits so that farmers are motivated to participate. The pragmatic approach described in the case study can also be applied to other situations in order to accelerate the adoption of new technologies, with the goal to improve data collection at scale and the availability of high-quality data.

Abstract WP389: Understanding the Clinicians’ Experiences in Collecting Stroke Rehabilitation Intensity Data Within Ontario

Stroke ◽  
2016 ◽  
Vol 47 (suppl_1) ◽  
Author(s):  
Elizabeth Linkewich ◽  
Janine Theben ◽  
Amy Maebrae-Waller ◽  
Shelley Huffman ◽  
Jenn Fearn ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Thematic Analysis ◽  
Best Practice ◽  
Practice Change ◽  
Quality Data ◽  
Electronic Survey ◽  
Workload Measurement ◽  
Survey Results ◽  
Patients Experience

Background and Issues: The collection and reporting of Rehabilitation Intensity (RI) in a national rehabilitation database was mandated on April 1, 2015 for all stroke patients within Ontario, to support evaluation of stroke best practice implementation. RI includes minutes of direct task-specific therapy patients experience per day. This requires a shift in thinking from capturing the clinician’s time spent in therapy to the patient perspective. To ensure that high quality data is collected, it was important to understand clinicians’ experiences in collecting RI data. Purpose: To identify enablers and barriers to RI data collection in order to inform the development of resources to support clinicians. Methods: A 12-item electronic survey was developed by an Ontario Stroke Network (OSN) task group to evaluate the clinician experience of RI data collection (including: demographics, barriers, enablers, education, resources, and practice change). The survey was distributed via SurveyMonkey® and sent to clinicians from 48 hospitals, 3 weeks post implementation of RI data collection. Analyses involved descriptive statistics and thematic analysis. Results: Three hundred and twenty-one clinicians from 47 hospitals responded to the survey. Survey results suggest RI data collection is feasible; seventy-one percent of clinicians report it takes 10 minutes or less to enter RI data. Thematic analysis identified: 5 common challenges with most frequently reported relating to data quality, 30% (N=358) and 6 common enablers with most frequently reported relating to ease of collecting RI data through workload measurement systems, 50% (N=46). Suggestions for educational resources included tools for identifying what is included in RI and the provision of education (e.g. webinars). Conclusions: RI data collection is feasible for clinicians. Education and resources developed should support key challenges and enablers identified by clinicians - to enhance data quality and the consistency of RI collection. As RI data fields are available through a national rehabilitation database, this work sets the foundation for other provinces interested in the systematic collection and reporting of RI data.

scholarly journals Data Watch The American Time Use Survey

2005 ◽  
Vol 19 (1) ◽  
pp. 221-232 ◽  
Author(s):  
Daniel S Hamermesh ◽  
Harley Frazis ◽  
Jay Stewart
Keyword(s):  
Time Use ◽  
National Income ◽  
Household Survey ◽  
Economic Research ◽  
First Year ◽  
American Time Use Survey ◽  
National Income Accounting ◽  
Time Use Survey ◽  
Time Diaries ◽  
Income Accounting

We discuss the new American Time Use Survey (ATUS), an on-going household survey of roughly 1,200 Americans per month (1,800 per month in the first year, 2003) that collects time diaries as well as demographic interview information from respondents who had recently been in the Current Population Survey. The characteristics of the data are presented, as are caveats and concerns that one might have about them. A number of novel uses of the ATUS in economic research, including in the areas of macroeconomics, national income accounting, labor economics, and others, are proposed to illustrate the magnitude of this new survey's possible applications.

scholarly journals Overcoming challenges to data quality in the ASPREE clinical trial

Trials ◽  
2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Jessica E. Lockery ◽  
◽  
Taya A. Collyer ◽  
Christopher M. Reid ◽  
Michael E. Ernst ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Missing Data ◽  
Data Quality ◽  
Large Scale ◽  
Controlled Trial ◽  
Data Entry ◽  
Quality Data ◽  
Community Based ◽  
Data Set ◽  
The Impact

Abstract Background Large-scale studies risk generating inaccurate and missing data due to the complexity of data collection. Technology has the potential to improve data quality by providing operational support to data collectors. However, this potential is under-explored in community-based trials. The Aspirin in reducing events in the elderly (ASPREE) trial developed a data suite that was specifically designed to support data collectors: the ASPREE Web Accessible Relational Database (AWARD). This paper describes AWARD and the impact of system design on data quality. Methods AWARD’s operational requirements, conceptual design, key challenges and design solutions for data quality are presented. Impact of design features is assessed through comparison of baseline data collected prior to implementation of key functionality (n = 1000) with data collected post implementation (n = 18,114). Overall data quality is assessed according to data category. Results At baseline, implementation of user-driven functionality reduced staff error (from 0.3% to 0.01%), out-of-range data entry (from 0.14% to 0.04%) and protocol deviations (from 0.4% to 0.08%). In the longitudinal data set, which contained more than 39 million data values collected within AWARD, 96.6% of data values were entered within specified query range or found to be accurate upon querying. The remaining data were missing (3.4%). Participant non-attendance at scheduled study activity was the most common cause of missing data. Costs associated with cleaning data in ASPREE were lower than expected compared with reports from other trials. Conclusions Clinical trials undertake complex operational activity in order to collect data, but technology rarely provides sufficient support. We find the AWARD suite provides proof of principle that designing technology to support data collectors can mitigate known causes of poor data quality and produce higher-quality data. Health information technology (IT) products that support the conduct of scheduled activity in addition to traditional data entry will enhance community-based clinical trials. A standardised framework for reporting data quality would aid comparisons across clinical trials. Trial registration International Standard Randomized Controlled Trial Number Register, ISRCTN83772183. Registered on 3 March 2005.

Are Interviews Costing £0.08 a Waste of Money? Reviewing Google Surveys for ‘Wisdom of the Crowd’ Projects

2017 ◽  
Vol 59 (2) ◽  
pp. 199-220
Author(s):  
G.W. Roughton ◽  
Iain Mackay
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Survey Data ◽  
Response Rates ◽  
Quality Data ◽  
Collection Costs

This paper investigates whether a ‘wisdom of the crowd’ approach might offer an alternative to recent political polls that have raised questions about survey data quality. Data collection costs have become so low that, as well as the question of data quality, concerns have also been raised about low response rates, professional respondents and respondent interaction. There are also uncertainties about self-selecting ‘samples’. This paper looks at more than 100 such surveys and reports that, in five out of the six cases discussed, £0.08p interviews delivered results in line with known outcomes. The results discussed in the paper show that such interviews are not a waste of money.

Sign in / Sign up

Export Citation Format

Share Document