Self-evaluation Interventions

Research has repeatedly shown self-efficacy to be associated with course outcomes in CS and across other fields. CS education research has documented this and has developed CS-specific self-efficacy measurement instruments, but to date there have been only a few studies examining interventions intended to improve students’ self-efficacy in CS, and several types of self-efficacy interventions suggested by previous research remain to be tested in CS. This study attempts to address this lack of research by reporting on the results of a trial intervention intended to improve students’ self-efficacy in an introductory programming course. Students were recruited to complete a self-evaluation task, which previous research has suggested could have a beneficial impact on self-efficacy, which should in turn have a beneficial impact on course performance. Participating students’ course outcomes and self-efficacy were compared with those of the students who did not complete the self-evaluation task, using propensity score weighting adjustments to control for differences between the groups on entering characteristics and prior values of self-efficacy and course outcomes. We found that, whereas there was only marginal evidence for the self-evaluation intervention having a direct effect on self-efficacy, students who completed the self-evaluation task had significantly higher project scores during the weeks they were asked to complete it, compared to the students who did not participate. These findings suggest that there are potential benefits to incorporating self-evaluation tasks into introductory CS courses, although perhaps not by virtue of directly influencing self-efficacy.

Modelling Mode Effects for a Panel Survey in Transition

Funding reductions combined with increasing data-collection costs required that Wave V of the USA’s National Longitudinal Study of Adolescent to Adult Health (Add Health) abandon its traditional approach of in-person interviewing and adopt a more cost-effective method. This approach used the mail/web mode in Phase 1 of data collection and in-person interviewing for a random sample of nonrespondents in Phase 2. In addition, to facilitate the comparison of modes, a small random subsample served as the control and received the traditional in-person interview. We show that concerns about reduced data quality as a result of the redesign effort were unfounded based on findings from an analysis of the survey data. In several important respects, the new two-phase, mixed-mode design outperformed the traditional design with greater measurement accuracy, improved weighting adjustments for mitigating the risk of nonresponse bias, reduced residual (or post-adjustment) nonresponse bias, and substantially reduced total-mean-squared error of the estimates. This good news was largely unexpected based upon the preponderance of literature suggesting data quality could be adversely affected by the transition to a mixed mode. The bad news is that the transition comes with a high risk of mode effects for comparing Wave V and prior wave estimates. Analytical results suggest that significant differences can occur in longitudinal change estimates about 60 % of the time purely as an artifact of the redesign. This begs the question: how, then, should a data analyst interpret significant findings in a longitudinal analysis in the presence of mode effects? This chapter presents the analytical results and attempts to address this question.

Dealing with Inaccurate Measures of Size in Two-Stage Probability Proportional to Size Sample Designs: Applications in African Household Surveys

The units at the early stages of multi-stage area samples are generally sampled with probabilities proportional to their estimated sizes (PPES). With such a design, an overall equal probability (EP) sample design would yield a constant number of final stage units from each final stage cluster if the measures of size used in the PPES selection at each sampling stage were directly proportional to the number of final stage units. However, there are often sizable relative differences between the measures of size used in the PPES selections and the number of final stage units. Two common approaches for dealing with these differences are: (1) to retain a self-weighting sample design, allowing the sample sizes to vary across the sampled primary sampling units (PSUs) and (2) to retain the fixed sample size in each PSU and to compensate for the unequal selection probabilities by weighting adjustments in the analyses. This article examines these alternative designs in the context of two-stage sampling in which PSUs are sampled with PPES at the first stage, and an equal probability sample of final stage units is selected from each sampled PSU at the second stage. Two-stage sample designs of this type are used for household surveys in many countries. The discussion is illustrated with data from the Population-based HIV Impact Assessment surveys that were conducted using this design in several African countries.

The Taiwan Adolescent to Adult Longitudinal Study (TAALS): Methodology and Cohort Description

The objective of the study was to introduce the methodology and report on cohort description of Taiwan Adolescent to Adult Longitudinal Study (TAALS). TAALS is the first nationwide longitudinal survey among Taiwan adolescents, linked with the National Health Insurance Research Database (NHIRD) to obtain complete medical records of respondents in the future. The TAALS project employed the principle of probability proportional to size (PPS) sampling method. Data were collected by questionnaire from 18 064 school students participating in 2015 formal survey, with good sample representation via a goodness-of-fit test after weighting adjustments. Through expert evaluation and statistics tests, TAALS shows a well nationally representation, validity, and reliability. Results indicate that the vocational school students had poor healthy behavior than other education systems, supporting the hypothesis that different learning environment will develop different health behaviors. TAALS can serve as a foundation for analyzing health trajectories of Taiwan adolescents.

Unit Nonresponse and Weighting Adjustments: A Critical Review

This article reviews unit nonresponse in cross-sectional household surveys, the consequences of the nonresponse on the bias of the estimates, and methods of adjusting for it. We describe the development of models for nonresponse bias and their utility, with particular emphasis on the role of response propensity modeling and its assumptions. The article explores the close connection between data collection protocols, estimation strategies, and the resulting nonresponse bias in the estimates. We conclude with some comments on the current state of the art and the need for future developments that expand our understanding of the response phenomenon.