The Challenges of Quantitative Surveys Among Hard-to-Survey Populations – A Case for Mixed-Methods Designs Based on a Survey With Unaccompanied Refugee Minors

Surveys of specific target groups that are hard to survey are prone to errors and biases. In this paper, we use the Total Survey Error (TSE) framework and a study on unaccompanied refugee minors (URM) in Germany to discuss how a mixed-methods quantitative-dominant research design can address challenges of quantitative-only surveys of such groups. We show that unit nonresponse and measurement are two main levels of bias that can be partly supplemented by qualitative research. In addition, taking ethical considerations into account when researching URMs affects the quality of quantitative surveys. This effect cannot be avoided, but it should be classified by researchers. We conclude that surveying hard-to-survey populations benefits from a combination of quantitative surveys and semi-structured interviews.

A New Calibration Estimator of Population Mean for Small Area with Nonresponse

This paper considered the challenges of population mean estimation in small area that is characterized by small or no sample size in the presence of unit nonresponse and presents a calibration estimator that produces reliable estimates under stratified random sampling from a class of synthetic estimators using calibration approach with alternative distance measure. Examining the proposed estimator relatively with existing ones under three distributional assumptions: normal, gamma, and exponential distributions with percent average absolute relative bias, percent average coefficient of variation, and average mean squared error as evaluation criteria using simulation analysis technique, the new estimator exhibited a more reliable estimate of the mean with less bias and greater gain in efficiency. Further evaluation using coefficient of variation under varying nonresponse rates to validate the results of variations suggests that the estimator is a suitable alternative for small area estimation. This finding has therefore contributed to the development of an ultimate estimator for small area estimation in the presence of unit nonresponse.

kmr: A command to correct survey weights for unit nonresponse using groups’ response rates

In this article, we describe kmr, a command to estimate a microcompliance function using groups’ nonresponse rates (Korinek, Mistiaen, and Ravallion, 2007, Journal of Econometrics 136: 213–235), which can be used to correct survey weights for unit nonresponse. We illustrate the use of kmr with an empirical example using the current population survey and state-level nonresponse rates.

Setting up Probability-Based Online Panels of Migrants with a Push-to-Web Approach: Lessons Learned from the German Emigration and Remigration Panel Study (GERPS)

We address two major challenges in setting up probability-based online panels of migrants, using the German Emigration and Remigration Panel Study (GERPS) as an example. The first challenge is potential spatial and social selectivity in unit response when using push-to-web recruitment. To address the first challenge, we draw on a split ballot experiment with return migrants in wave 1 of GERPS. The related analysis uses population register data and geo data. We use logistic regressions to compare unit nonresponse between a push-to-web-only control group (n = 5999) and two sub-samples (each n = 1000) with optional paper and pencil interviews (PAPI). The second challenge is panel attrition. To address the second challenge, we investigate the role of individual-level and survey-related factors for panel consent. The regression analysis uses GERPS data of first-wave respondents, estimating panel consent rates for responding remigrants in general (n = 6395) and in the experiment sample (n = 2130). We find that the provision of an optional paper questionnaire marginally increases the likelihood of response. The positive correlation of PAPI and response rate, however, is counterbalanced by a negative correlation with the likelihood of panel consent. This suggests a trade-off scenario to the detriment of either response rates or panel participation rates.

Targeting Key Survey Variables at the Unit Nonresponse Treatment Stage

In the presence of nonresponse, unadjusted estimators are vulnerable to nonresponse bias when the characteristics of the respondents differ from those of the nonrespondents. To reduce the bias, it is common practice to postulate a nonresponse model linking the response indicators and a set of fully observed variables. Estimated response probabilities are obtained by fitting the selected model, which are then used to adjust the base weights. The resulting estimator, referred to as the propensity score-adjusted estimator, is consistent provided the nonresponse model is correctly specified. In this article, we propose a weighting procedure that may improve the efficiency of propensity score estimators for survey variables identified as key variables by making a more extensive use of the auxiliary information available at the nonresponse treatment stage. Results from a simulation study suggest that the proposed procedure performs well in terms of efficiency when the data are missing at random and also achieves an efficient bias reduction when the data are not missing at random. We further apply our proposed methods to 2017–2018 National Health Nutrition and Examination Survey.

Proxy Pattern-Mixture Analysis for a Binary Variable Subject to Nonresponse

Given increasing survey nonresponse, good measures of the potential impact of nonresponse on survey estimates are particularly important. Existing measures, such as the R-indicator, make the strong assumption that missingness is missing at random, meaning that it depends only on variables that are observed for respondents and nonrespondents. We consider assessment of the impact of nonresponse for a binary survey variable Y subject to nonresponse when missingness may be not at random, meaning that missingness may depend on Y itself. Our work is motivated by missing categorical income data in the 2015 Ohio Medicaid Assessment Survey (OMAS), where whether or not income is missing may be related to the income value itself, with low-income earners more reluctant to respond. We assume there is a set of covariates observed for nonrespondents and respondents, which for the item nonresponse (as in OMAS) is often a rich set of variables, but which may be potentially limited in cases of unit nonresponse. To reduce dimensionality and for simplicity we reduce these available covariates to a continuous proxy variable X, available for both respondents and nonrespondents, that has the highest correlation with Y, estimated from a probit regression analysis of respondent data. We extend the previously proposed proxy-pattern mixture (PPM) analysis for continuous outcomes to the binary outcome using a latent variable approach for modeling the joint distribution of Y and X. Our method does not assume data are missing at random but includes it as a special case, thus creating a convenient framework for sensitivity analyses. Maximum likelihood, Bayesian, and multiple imputation versions of PPM analysis are described, and robustness of these methods to model assumptions is discussed. Properties are demonstrated through simulation and with the 2015 OMAS data.