Presentation-mode effects in large-scale writing assessments

2020 ◽  
Vol 45 ◽  
pp. 100470
Author(s):  
Thomas Canz ◽  
Lars Hoffmann ◽  
Renate Kania
2019 ◽  
Vol 28 (1-2) ◽  
Author(s):  
Mattias Tagseth ◽  
Erik R. Sund ◽  
Göran T.A. Hallman ◽  
Jostein Holmen ◽  
Kyrre Kvistad ◽  
...  

Background: Availability of data on health and its determinants at the local area level is a prerequisite for developing interventions and public health campaigns locally. Collecting self-reported data by means of telephone interviews may rapidly provide relevant data. The reliability of such data may be questioned. In this study, we sought to compare exact similar questions addressed by a recent telephone survey with a previous large scale and very comprehensive population health survey (The Nord-Trøndelag Health Study 2006-08 – HUNT3), conducted a few years earlier in the same geographical region. This was done in order to examine the reliability of telephone interviews as a method to provide data on health and determinants to enable municipal authorities to get a sufficient overview.Methods: One rural and one urban municipality covered by HUNT3 using paper questionnaires were resurveyed through computer assisted telephone interviews. The weighted results for 34 dichotomized variables were compared using chi square tests.Results: The comparison of results between the rural and the urban samples and HUNT3 involved 68 chi square tests, 25 of which (38%) displayed significant differences. The ability of the telephone survey to replicate the results from HUNT3 was only moderate, but with differences between survey themes. Comparability was poor for adverse life events and mental health factors, fair for behavioural and risk factors, and skewed for general health and life satisfaction. The replication was good for reports on the less sensitive and subjective theme of cultural participation.Conclusion: The comparability of the data differed between themes. The differences may be ascribed to mode effects and to some extent the time lag between the surveys. Because replicability on issues that may be more embarrassing or stressful to recall appears to be poorer, and the more subjective self-assessments of health and well-being appear skewed, it is reasonable to conclude that there is an interviewer effect in the telephone survey. The use of a questionnaire through mail or web to monitor public health in municipalities should be considered as an alternative.


2021 ◽  
Author(s):  
Stella Chatzitheochari ◽  
Elena Mylona

Recent years have witnessed an increasing interest in the use of new technologies for time-use data collection, driven by their potential to reduce survey administration costs and improve data quality. However, despite the steady growth of studies that employ web and app time diaries, there is little research on their comparability with traditional paper-administered diaries that have long been regarded as the “gold standard” for measurement in time-use research. This paper rectifies this omission by investigating diary mode effects on data quality and measurement, drawing on data from a mixed-mode large-scale time diary study of adolescents in the United Kingdom. After controlling for selection effects, we find that web and app diaries yield higher quality data than paper diaries, which attests to the potential of new technologies in facilitating diary completion. At the same time, our analysis of broad time-use domains does not find substantial mode effects on measurement for the majority of daily activity categories. We conclude by discussing avenues for future methodological research and implications for time-use data collection.


Sign in / Sign up

Export Citation Format

Share Document