A Methodological Framework for the Analysis of Panel Conditioning Effects

Panel conditioning refers to the phenomenon whereby respondents’ attitudes, behaviour, reporting of behaviour and/or knowledge are changed by repeated participation in a panel survey. Uncovering such effects, however, is difficult due to three major methodological challenges. First, researchers need to disentangle changes in behaviour and attitudes from changes in the reporting of behaviour and attitudes as panel conditioning may result in both, even at the same time and in opposite directions. Second, the identification of the causal effect of panel participation on the various forms of change mentioned above is complicated as it requires comparisons of panel respondents with control groups of people who have not been interviewed before. Third, other sources of error in (panel) surveys may easily be mistaken for panel conditioning if not properly accounted for. Such error sources are panel attrition, mode effects, and interviewer effects. In this chapter the challenges mentioned above are reviewed in detail and a methodological framework for the analysis of panel conditioning effects is provided by identifying the strengths and weaknesses of the various designs that researchers have developed to address the challenges. The chapter concludes with a discussion of a future research agenda on panel conditioning effects in longitudinal surveys.

Using a Mobile App When Surveying Highly Mobile Populations: Panel Attrition, Consent, and Interviewer Effects in a Survey of Refugees

Panel attrition poses major threats to the survey quality of panel studies. Many features have been introduced to keep panel attrition as low as possible. Based on a random sample of refugees, a highly mobile population, we investigate whether using a mobile phone application improves address quality and response behavior. Various features, including geo-tracking, collecting email addresses and adress changes, are tested. Additionally, we investigate respondent and interviewer effects on the consent to download the app and sharing GPS geo-positions. Our findings show that neither geo-tracking nor the provision of email addresses nor the collection of address changes through the app improves address quality substantially. We further show that interviewers play an important role in convincing the respondents to install and use the app, whereas respondent characteristics are largely insignificant. Our findings provide new insights into the usability of mobile phone applications and help determine whether they are a worthwhile tool to decrease panel attrition.

Setting up Probability-Based Online Panels of Migrants with a Push-to-Web Approach: Lessons Learned from the German Emigration and Remigration Panel Study (GERPS)

We address two major challenges in setting up probability-based online panels of migrants, using the German Emigration and Remigration Panel Study (GERPS) as an example. The first challenge is potential spatial and social selectivity in unit response when using push-to-web recruitment. To address the first challenge, we draw on a split ballot experiment with return migrants in wave 1 of GERPS. The related analysis uses population register data and geo data. We use logistic regressions to compare unit nonresponse between a push-to-web-only control group (n = 5999) and two sub-samples (each n = 1000) with optional paper and pencil interviews (PAPI). The second challenge is panel attrition. To address the second challenge, we investigate the role of individual-level and survey-related factors for panel consent. The regression analysis uses GERPS data of first-wave respondents, estimating panel consent rates for responding remigrants in general (n = 6395) and in the experiment sample (n = 2130). We find that the provision of an optional paper questionnaire marginally increases the likelihood of response. The positive correlation of PAPI and response rate, however, is counterbalanced by a negative correlation with the likelihood of panel consent. This suggests a trade-off scenario to the detriment of either response rates or panel participation rates.

Predicting Higher Education Grades using Strategies Correcting for Panel Attrition

This study aims to forecast the final grade of the first higher education degree which can be of considerable interest for higher education institutions to implement early warning systems, students themselves, or potential employers. The analysis is based on the National Education Panel Study (NEPS), a large German dataset covering many aspects of students’ (educational) life. Since panel attrition concerns 35% of participants the Heckman correction and the inverse probability weight (IPW) estimator are used to reduce the estimation bias. A distinction is made between two scenarios, excluding dropout students and including them with a grade of 5.0. Some predictors reveal significant parameter estimates in the first but not in the second scenario, or vice versa, which means that dropout and study performance is not driven by the same variables. To get an early prediction of grades only variables of a pre-university episode were included in the first step. Afterward, variables of the early study phase are included. For the IPW estimator, the R2 improves from 0.202 to 0.593 (dropouts included) when adding the additional variables. The best predictors are the grades at secondary school, grades in the first exams, and the type of institution.

Using Administrative Data to Evaluate Sampling Bias in a Business Panel Survey

We examine two sources of bias for the Bank of Italy’s panel business survey of Industrial and Services Firms: 1) the bias caused by panel attrition; and 2) the bias created by delays in the distributional data on the reference population, needed for computing the survey weights. As for the first source of bias, the estimates strongly dependent on big firms’ values are less affected by panel attrition than those representing firms’ average behavior, independent of their sizes. Positive economic results make it easier to enroll new firms in the survey, in order to replace firms dropping out because of bad economic performances. However, the economic results of new entrances become more aligned to those of the population, once they enter the sample. A very different result emerges for the second source of bias, since, when the population size is highly variable, the information delays produce a bias for the estimates influenced by the contribution of great firms, but the effect is negligible for the estimates not dependent on firm size.

Are Prepaid Monetary Incentives Sufficient for Reducing Panel Attrition and Optimizing the Response Rate? An Experiment in the Context of a Multi-Wave Panel with a Sequential Mixed-Mode Design

In this contribution, we evaluate the short- and long-term effects of a prepaid cash incentive on young people’s cooperation and response rate in the fourth and fifth wave of a panel with sequential mixed-mode design (online questionnaire, CATI). Analyses are based on a survey experiment of students from randomly selected school classes of equal shares, which have participated in the third wave. Findings show that a monetary incentive has a direct and positive effect on the response rate in the fourth but not in the subsequent wave. However, the effect of the incentive is not persistent, since the effect weakens and fades away during the field phase and cannot be directly transferred to the second survey mode. As emphasized in the tailored design method (TDM), a monetary incentive can contribute to a shorter field phase and hence lower costs, but it is an insufficient instrument against panel attrition and the optimization of the retention rate when other strategies are disregarded.