Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

Noël Coypel’s educational journey to Rome at the end of 1672 and the beginning of 1673

The aim of this article is to present the circumstances of Noël Coypel’s appointment as rector of the French Academy in Rome and to trace the route of his didactic journey from Paris to Rome with the Prix de Rome scholars entrusted to him. The paper is an attempt to answer the following questions: why a more difficult route through the Alps was chosen (and not, for example, a river and sea route), in what way was the journey educational, and what role did the documents given to Coypel play in securing the expedition. The article is based on an analysis of administrative records during the reign of Louis XIV, lists of superintendents and directors of the French Academy in Rome, accounts of royal buildings, and minutes of the Royal Academy of Painting and Sculpture in Paris. The paper uses the analytical method, the comparative method, the synthetic method, source criticism, argumentum ex silentio inference, and the geographical method when discussing the itinerary. Although the trip was purposeful and related to Coypel’s new position, he designed it in such a way as to not so much get to the destination quickly, but to show his students as much as possible. Coypel introduced the royal scholars to masterpieces of painting and sculpture at centers along a route through Dijon, Lyon, Chambéry, the Mont Cenis Pass, Turin, Milan, Bologna and Florence. The crossing of the Alps, though dangerous, was most often chosen because of the artistic reputation of the cities there. The trip was educational at the expense of comfort or safety. Coypel, as a guide and teacher (paidagōgós – παιδαγωγός) led his charges by overseeing their learning during and through the journey. Wandering to the Eternal City was part of a painter’s education (paideía – παιδεία) in the seventeenth century and was part of Coypel’s didactic work allowing young people to be inspired by direct exposure to masterpieces. The journey had an eminently didactic and artistic character, but also an initiatory one, as it gradually initiated and prepared the students for the experience of Rome, the center of artistic life at that time.

Grey’s Anatomy: Gender Differences in Specialty Choice for Medical Students in China

Gender differences in sub-major choices within the science, technology, engineering, and mathematics (STEM) fields have scarcely been discussed. This study uses administrative records from a top medical school in China to examine gender differences in medical students’ specialty choices. Results showed that, although the gender gap in choosing a clinical track shrinks over time, female students in the clinical track are far less likely to choose highly paid surgical specialties, and this gap persists over time. However, female students outperformed male students in all of the courses. Thus, academic performance cannot explain the underrepresentation of female students in surgery. We further collected questions such as “Why don’t female students choose surgical specialties” and answers to them in “Chinese Quora”, Zhihu.com. A preliminary text analysis showed that ultra-physical load, discrimination in recruitment, women-unfriendly work climates, and difficulties in taking care of family are barriers that prevent women from choosing surgery.

Early Franciscans in England: Sickness, Healing and Salvation

From their first arrival in England in 1224, the Franciscans were concerned with the treatment of ill-health for both practical and spiritual reasons. Many brothers fell sick, and their illnesses required both interpretation and treatment. Some friars practised healing on their brethren and on lay patients. This article will focus on the question of the relationship between the religious vocation of the friars and the exigencies of sickness. Little evidence survives in England in the form of administrative records. But two early Franciscan writings (Tractatus de adventu fratrum minorum in Angliam, and the letters of Adam Marsh OFM, d. 1259) throw significant light on attitudes to illness and practical responses.

Generic Skills, Academic Achievement, and Means of Improving the Former

In Canada, in general – and in the Province of Ontario in particular – academics, employers, and government agencies are concerned with the low generic skill levels of university students and graduates. The assumption is that such deficiencies detract from academic and job success. Despite this concern, in Canada, research has not focused on potential links between objectively measured generic skills and grades recorded in administrative records. In view of this lacuna, the current research has two objectives. First, to assess the net effect of objectively measured generic skills on academic achievement as recorded in administrative records. Second, to determine the efficacy of an online course dedicated to the development of generic skills. Overall, I found that generic skills were better predictors of students’ achievement than high school grades used in admission processes; the relationship between high school grades and generic skill levels was weak; students’ generic skill levels did not improve over time; and an online course devoted to increasing students’ generic skills was effective in boosting skills to an acceptable level. Accordingly, if they are concerned with academic achievement, universities in Ontario and in other jurisdictions in which students are admitted to university primarily based on their secondary school grades might make the development of generic skills a priority; however, unless such skills are demanded across the curriculum, they will atrophy.

Using Taiwanese Universal Health Insurance Data to Estimate LTC Needs with Machine Learning

One of the core issues in long-term care (LTC) policy is the growing imbalance between demand and supply of LTC services due to aging population. To estimate the imbalance and allocate LTC resources, the government regularly conducts surveys. These surveys are expensive given the sample size requirements and imprecise given their subjective nature. This study links the administrative records of the universal health insurance database with LTC program usage records in Taiwan to explore this issue. Machine learning algorithms are used in projecting LTC needs from administrative records. LTC program usage records provide detailed LTC needs information and the amount of service each individual used. In addition, health insurance claim data provides rich health information. Specific LTC needs are predicted for each individual. By further extrapolating to future demographics, long-term LTC needs could be projected. There are several findings in this study. Prediction of difficulties in activities of daily livings (ADL), measured by Barthel index, works best using the Gradient Boosting algorithm. The mean absolute error is 17.67 out of a 0 to 100 scale. In addition to dementia and stroke, diagnosis of pressure ulcer (ICD 10 code: L89) and pneumonia (ICD 10 code: J18) have high predictive power for LTC needs. Prediction of Instrumental ADL (IADL) also performs well with a mean absolute error 1.31. The prediction model suggests high LTC needs and excess demand as the demographics changing. Our study provides a reliable way of using rich information to estimate future LTC needs without conducting additional costly surveys.

Recommendations on Diversity Data in the Social Sector

Administrative records, surveys, and censuses are all important resources for understanding the extent and nature of existing inequities and biases. When individuals, households, and peoples (pueblos) are properly identified based on their ethnicity, race, disability status, sexual orientation, migration status, gender identity, or other characteristics, gaps in socioeconomic indicators across these groups can be quantified. This enables policymakers to focus efforts and resources towards the most disadvantaged and promote equity. The lack of disaggregated data representative of diverse groups limits our understanding of their living conditions and economic opportunities. It also poses a significant challenge. This guide supports individuals interested in collecting or analyzing data on indigenous peoples, African descendants, people with disabilities, and LGBTQ populations by providing conceptual and empirical tools for specialists.

Recommendations on Diversity Data in the Social Sector

Administrative records, surveys, and censuses are all important resources for understanding the extent and nature of existing inequities and biases. When individuals, households, and peoples (pueblos) are properly identified based on their ethnicity, race, disability status, sexual orientation, migration status, gender identity, or other characteristics, gaps in socioeconomic indicators across these groups can be quantified. This enables policymakers to focus efforts and resources towards the most disadvantaged and promote equity. The lack of disaggregated data representative of diverse groups limits our understanding of their living conditions and economic opportunities. It also poses a significant challenge. This guide supports individuals interested in collecting or analyzing data on indigenous peoples, African descendants, people with disabilities, and LGBTQ populations by providing conceptual and empirical tools for specialists.

Zachrisson et al 2021 ECEC Achievement

In this study, we estimate the effects of the scale-up of Norway’s universal ECEC program — expanding access to 1- and 2-year olds starting in the early 2000s—on standardized math and achievement tests in 5th grade (age 10) using a population-based survey sample (Norwegian Mother, Father, and Child Cohort Study, MoBa, n = 102,352), linked with national administrative records of child achievement test scores. These data support using fixed-effects regressions and instrumental variable regressions to make inferences about the causal impact of ECEC scale-up on middle childhood achievement. We find the scale-up of ECEC starting in the second year of life improved test scores, especially for children from families with low levels of parental education, thereby reducing the achievement gap between children ofparents with the highest and lowest education by up to 50%.

Inside Jobs: Salary Setting for Immigrants Crossing Establishment, Organizational, and National Boundaries

Using novel US Department of Labor administrative records, the authors test theoretical mechanisms to account for variation in immigrant workers’ starting salaries following key career transitions. Specifically, they examine differences in the base starting salaries and discretionary starting salary increases above these base starting salaries for 1) same-establishment hires, relative to 2) US-based establishment transfers, 3) international establishment transfers, 4) US-based external hires, and 5) international external hires. In support of the “insider premium” account, findings show that same-establishment hires tend to work in jobs with greater requirements, and thus higher base starting salaries. In partial support of the “outsider premium” account, findings show that US-based external hires receive larger starting salary increases than do same-establishment hires, conditional on the jobs they enter. This said, international external hires receive smaller starting salary increases than do same-establishment hires. Findings reveal distinct mechanisms, acting separately or in tandem, during salary-setting processes.