Risk Factors for Cognitive Decline in Older Adults in Puerto Rico: Assessing Bias From Sample Attrition

Many risk factors for cognitive decline are associated with mortality and are common among older adults who cannot complete a survey interview. Our objective was to compare analyses of risk factors for cognitive decline among older adults in Puerto Rico with and without accounting for sample attrition. Data came from the Puerto Rican Elderly: Health Conditions Study. Our sample included 3,437 participants interviewed in 2002/03. Cognitive function was measured using the Mini-Mental Caban (MMC). The outcome was the change in MMC score between 2002/03 and 2006/07. Logistic regression was used to estimate inverse probability weights for being interviewed in 2006/07 (n=3,028) and completing the MMC at follow-up (n=2,601). Linear regression models were used to assess the association between stroke, hypertension, diabetes, smoking status, and cognitive decline with and without the IPWs. In the unweighted analysis, stroke was associated with a significantly greater decline in cognition (b=-0.62, standard error [SE]=0.30, p=0.04). Hypertension (b=-0.02, SE=0.12, p=0.84), diabetes (b=-0.22, SE=0.13, p=0.10) and being a current (b=0.05, SE=0.22, p=0.84) or former smoker (b=0.05, SE=0.14, 0.74) were not associated with cognitive decline in the unweighted analysis. The results were similar when including the IPW for mortality (stroke b=-0.63; hypertension b=-0.03; diabetes: b=-0.20; current smoker: b=0.08; former smoker: b=0.07) and having completed the MMC at follow-up (stroke b=-0.58; hypertension b=-0.03; diabetes: b=-0.20; current smoker: b=0.03; former smoker: b=0.09). These findings indicate that stroke is a risk factor for cognitive decline among older Puerto Rican adults even after accounting for selective attrition.

Response to Adamson et al. (2020): ‘Cognitive behavioural therapy for chronic fatigue and chronic fatigue syndrome: Outcomes from a specialist clinic in the UK’

In a paper published in the Journal of the Royal Society of Medicine, Adamson et al. (2020) interpret data as showing that cognitive behavioural therapy leads to improvement in patients with chronic fatigue syndrome and chronic fatigue. Their research is undermined by several methodological limitations, including: (a) sampling ambiguity; (b) weak measurement; (c) survivor bias; (d) missing data and (e) lack of a control group. Unacknowledged sample attrition renders statements in the published Abstract misleading with regard to points of fact. That the paper was approved by peer reviewers and editors illustrates how non-rigorous editorial processes contribute to systematic publication bias.

Trajectories of change in internalizing symptoms during the COVID-19 pandemic: A longitudinal population-based study

Background: Longitudinal data indicates that the mental health of the general population may not have been as badly affected by the COVID-19 pandemic as some had feared. Most studies examining change in mental health during the pandemic have assumed population homogeneity which may conceal evidence of worsening mental health for some. In this study, we applied a heterogeneous perspective to determine if there were distinct groups in the population characterised by different patterns of change in internalizing symptoms during the pandemic. Methods: Self-report data were collected from a nationally representative sample of Irish adults (N = 1,041) at four time-points between April and December 2020. Results: In the entire sample, mean levels of internalizing symptoms significantly declined from March to December 2020. However, we identified four distinct groups with different patterns of change. The most common response was ‘Resilience’ (66.7%), followed by ‘Improving’ (17.9%), ‘Worsening’ (11.3%), and ‘Sustained’ (4.1%). Belonging to the ‘Worsening’ class was associated with younger age, city dwelling, current and past treatment for a mental health problem, higher levels of empathy, and higher levels of loneliness. Limitations: Sample attrition was relatively high and although this was managed using robust statistical methods, bias associated with non-responses cannot be entirely ruled out. Conclusion: The majority of adults experienced no change, or an improvement in internalizing symptoms during the pandemic, and a relatively small proportion of adults experienced a worsening of internalizing symptoms. Limited public mental health resources should be targeted toward helping these at-risk individuals.

RESPONSE TO ADAMSON ET AL. (2020): “COGNITIVE BEHAVIOURAL THERAPY FOR CHRONIC FATIGUE AND CHRONIC FATIGUE SYNDROME: OUTCOMES FROM A SPECIALIST CLINIC IN THE UK”

In this review, we consider the paper by Adamson et al., published in the October 2020 issue of the Journal of the Royal Society of Medicine. The authors interpret their data as revealing significant improvements following cognitive behavioural therapy in a large sample of patients with chronic fatigue syndrome and chronic fatigue. Overall, the research is hampered by several fundamental methodological limitations that are not acknowledged sufficiently, or at all, by the authors. These include: (a) sampling ambiguity; (b) weak measurement; (c) survivor bias; (d) missing data; and (e) lack of a control group. In particular, the study is critically hampered by sample attrition, rendering the presentation of statements in the Abstract misleading with regard to points of fact, and, in our view, urgently requiring a formal published correction. In light of the fact that the paper was approved by multiple peer-reviewers and editors, we reflect on what its publication can teach us about the nature of contemporary scientific publication practices.

Reliability of Longitudinal Social Surveys of Access to Higher Education: The Case of Next Steps in England

Longitudinal social surveys are widely used to understand which factors enable or constrain access to higher education. One such data resource is the Next Steps survey comprising an initial sample of 16,122 pupils aged 13–14 attending English state and private schools in 2004, with follow up annually to age 19–20 and a further survey at age 25. The Next Steps data is a potentially rich resource for studying inequalities of access to higher education. It contains a wealth of information about pupils’ social background characteristics—including household income, parental education, parental social class, housing tenure and family composition—as well as longitudinal data on aspirations, choices and outcomes in relation to education. However, as with many longitudinal social surveys, Next Steps suffers from a substantial amount of missing data due to item non-response and sample attrition which may seriously compromise the reliability of research findings. Helpfully, Next Steps data has been linked with more robust administrative data from the National Pupil Database (NPD), which contains a more limited range of social background variables, but has comparatively little in the way of missing data due to item non-response or attrition. We analyse these linked datasets to assess the implications of missing data for the reliability of Next Steps. We show that item non-response in Next Steps biases the apparent socioeconomic composition of the Next Steps sample upwards, and that this bias is exacerbated by sample attrition since Next Steps participants from less advantaged social backgrounds are more likely to drop out of the study. Moreover, by the time it is possible to measure access to higher education, the socioeconomic background variables in Next Steps are shown to have very little explanatory power after controlling for the social background and educational attainment variables contained in the NPD. Given these findings, we argue that longitudinal social surveys with much missing data are only reliable sources of data on access to higher education if they can be linked effectively with more robust administrative data sources. This then raises the question—why not just use the more robust datasets?

HIP, RIP, and the robustness of empirical earnings processes

The dispersion of individual returns to experience, often referred to as heterogeneity of income profiles (HIP), is a key parameter in empirical human capital models, in studies of life‐cycle income inequality, and in heterogeneous agent models of life‐cycle labor market dynamics. It is commonly estimated from age variation in the covariance structure of earnings. In this study, I show that this approach is invalid and tends to deliver estimates of HIP that are biased upward. The reason is that any age variation in covariance structures can be rationalized by age‐dependent heteroscedasticity in the distribution of earnings shocks. Once one models such age effects flexibly the remaining identifying variation for HIP is the shape of the tails of lag profiles. Credible estimation of HIP thus imposes strong demands on the data since one requires many earnings observations per individual and a low rate of sample attrition. To investigate empirically whether the bias in estimates of HIP from omitting age effects is quantitatively important, I thus rely on administrative data from Germany on quarterly earnings that follow workers from labor market entry until 27 years into their career. To strengthen external validity, I focus my analysis on an education group that displays a covariance structure with qualitatively similar properties like its North American counterpart. I find that a HIP model with age effects in transitory, persistent and permanent shocks fits the covariance structure almost perfectly and delivers small and insignificant estimates for the HIP component. In sharp contrast, once I estimate a standard HIP model without age‐effects the estimated slope heterogeneity increases by a factor of thirteen and becomes highly significant, with a dramatic deterioration of model fit. I reach the same conclusions from estimating the two models on a different covariance structure and from conducting a Monte Carlo analysis, suggesting that my quantitative results are not an artifact of one particular sample.