5/7-POINT “LIKERT SCALES” AREN’T ALWAYS THE BEST OPTION Their validity is undermined by lack of reliability, response style bias, long completion times and limitations to permissible statistical procedures

Some research methods are used by default. Even if they undermine the validity of conclusions, authors no longer justify the use of default approaches and reviewers no longer query them. One such default is the use of bipolar ordinal 5/7-point survey answer formats, often incorrectly referred to as 5/7-point “Likert scales”. This Viewpoint argues that default answer formats should be replaced by thorough assessments of the benefit-to-cost ratio of alternative answer formats for every survey question asked. Benefits of answer formats include enabling respondents to meaningfully express themselves, while keeping respondent burden low. Costs include reducing data quality by introducing bias or over-burdening respondents, or by collecting data at a scale level that limits permissible statistical procedures.

Respondent Burden Effects on Item Non-Response and Careless Response Rates: An Analysis of Two Types of Surveys

The respondent burden refers to the effort required by a respondent to answer a questionnaire. Although this concept was introduced decades ago, few studies have focused on the quantitative detection of such a burden. In this paper, a face-to-face survey and a telephone survey conducted in Valencia (Spain) are analyzed. The presence of burden is studied in terms of both item non-response rates and careless response rates. In particular, two moving-window statistics based on the coefficient of unalikeability and the average longstring index are proposed for characterizing careless responding. Item non-response and careless response rates are modeled for each survey by using mixed-effects models, including respondent-level and question-level covariates and also temporal random effects to assess the existence of respondent burden during the questionnaire. The results suggest that the sociodemographic characteristics of the respondents and the typology of the question impact item non-response and careless response rates. Moreover, the estimates of the temporal random effects indicate that item non-response and careless response rates are time-varying, suggesting the presence of respondent burden. In particular, an increasing trend in item non-response rates in the telephone survey has been found, which supports the hypothesis of the burden. Regarding careless responding, despite the presence of some temporal variation, no clear trend has been identified.

Balancing Bias and Burden in Personal Network Studies

Personal network data is increasingly used to answer research questions about the interplay between individuals (i.e., egos) and their social environment (i.e., alters). Researchers designing such data collections face a trade-off: When eliciting a high number of alters, study participation can be particularly burdensome as all data is obtained by surveying the ego. Eliciting a low number of alters, however, may incur bias in network characteristics. In the present study we use a sample of 701 Dutch women and their personal networks of 25 alters to investigate two strategies reducing respondent burden in personal network data collections: (1) eliciting fewer alters and (2) selecting a random subsample from the original set of elicited alters for full assessment. We present the amount of bias in structural and compositional network characteristics connected to applying these strategies as well as the potential study time gain for every possible network size (2 to 24 alters) as a proxy for respondent burden reduction. Our results can aid researchers designing a personal network study to balance respondent burden and bias in estimates for a range of compositional and structural network characteristics.

Measuring non-commercial tourism traffic in Croatia: Challenges of using administrative data

National statistical authorities are under pressure to provide reliable and readily accessible data with minimum costs and respondent burden. To this end, a use of administrative data, defined as those not primarily collected for statistical purposes, is increasingly encouraged to supplement sample and census surveys. In this paper, a possibility to produce new statistics by combining several sources where at least one is the administrative data is tested. This exercise was applied to measure a volume of tourists (residents and non-residents) staying in non-commercial accommodation facilities in Croatia. Four data sources were analysed: (i) administrative data from eVisitor system, a unique online central information system for tourists registration in Croatia that provides insights into tourist traffic and accommodation capacities (commercial and non-commercial), (ii) Croatian Bureau of Statistics Survey on Tourist Activity of the Croatian population, that provides information on number of domestic trips and nights in commercial and noncommercial accommodation establishments, (iii) Croatian National Bank Survey on foreign visitors in Croatia, that estimates number of foreign tourist nights in Croatia in, both, commercial and non-commercial type of accommodation, and (iv) administrative data from Tax Register on paid taxes on second homes – holiday houses and apartments. The results clearly demonstrated that two survey-based data sources and the administrative one regarding the second homes taxes had not provided a quality foundation to improve the current estimations of tourist traffic in non-commercial accommodation provided by the eVisitor system.

Solving School’s Survey Request Overload

Limited information is available on strategies for managing the large number of survey requests that reach an individual nursing school. This article addresses problems identified in managing survey requests and describes the implementation and evaluation of a solution. Identified problems included the appearance of endorsing studies of varying quality and rigor, overlap and competition between external study requests and internal studies, respondent burden, and level of anonymity and confidentiality. The solution included a school-wide policy for tracking and vetting study requests before they were distributed. Evaluation data show the number of requests received (total, by month and source, by target population), their disposition (withdrawn, approved, not approved for distribution), and quality improvement data on meeting a 30-day target turnaround time. Additional considerations are also discussed.

Micro data linking: Addressing new emerging topics without increasing the respondent burden

Micro data linking (MDL) has become an important cornerstone in the production of new statistical insights. MDL is now widely acknowledged as a strategic activity to avoid increasing the respondent burden when meeting new user demands on e.g. globalisation. MDL is a strong method in combining micro data on individual entities such as enterprises, people or in a combination of both (Linked Employer-Employee Data). MDL is one of the most powerful methods to answer urgent questions on emerging policy or research topics such as the interconnectedness of the economies and its consequences for jobs, income and growth. Not only for national purposes, where economic behavior and dynamics by enterprises can be expressed in terms of job dynamics, income and welfare for its citizens, but also from an international perspective where consistent and coherent indicators play an important role. Firstly, the article addresses the central role of the Business Register for any MDL approach related to business statistics; secondly the current official statistics based on MDL such as Trade by Enterprise Characteristics (TEC) are described. Thirdly, the most common use of MDL in terms of producing experimental statistics are described, including linked employer-employee data (LEED).

Toward Reduced Burden in Evidence-Based Assessment of PTSD: A Machine Learning Study

Structured diagnostic interviews involve significant respondent burden and clinician administration time. This study examined whether we can maintain diagnostic accuracy using fewer posttraumatic stress disorder (PTSD) assessment questions. Our study included 1,265 U.S. veterans of the Afghanistan and Iraq conflicts who were assessed for PTSD using the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (SCID-5). We used random forests to assess the importance of each diagnostic item in predicting a SCID-5 PTSD diagnosis. We used variable importance to rank each item and removed the lowest ranking items while maintaining ≥90% accuracy (i.e., efficiency), sensitivity, and other metrics. We eliminated six diagnostic items among the overall sample, four items among male veterans, and six items among female veterans. Our findings demonstrate that we may shorten the SCID-5 PTSD module while maintaining excellent diagnostic performance. These findings have implications for potentially reducing patient and provider burden of PTSD diagnostic assessment.

Improved Estimation of Poisson Rate Distributions Through a Multimode Survey Design

Researchers interested in studying the frequency of events or behaviors among a population must rely on count data provided by sampled individuals. Often, this involves a decision between live event counting, such as a behavioral diary, and recalled aggregate counts. Diaries are generally more accurate, but their greater cost and respondent burden generally yield less data. The choice of survey mode, therefore, involves a potential trade-off between bias and variance of estimators. We use a case study comparing inferences about payment instrument use based on different survey designs to illustrate this dilemma. We then use a simulation study to show how and under what conditions a hybrid survey design can improve efficiency of estimation, in terms of mean-squared error. Overall, our work suggests that such a hybrid design can have considerable benefits, as long as there is nontrivial overlap in the diary and recall samples.