Using theory and reflexivity to preserve methodological rigour of data collection in qualitative research

This paper examines the place of theory in qualitative medical research. While theory’s place in research planning and data analysis has been well-established, the contribution of theory during qualitative data collection tends to be overlooked. Yet, data collection is not an asocial or apolitical process and requires reflection and analysis in and of itself. Therefore, drawing on an exemplar case study research project which focused on patients’ use of medicines, the paper argues that engaging with theory to think reflexively, throughout a project but particularly during the process of data collection can ensure the rigour and trustworthiness of qualitative data. In this case study, we draw on sociologist Erving Goffman’s theoretical framework of the dramaturgical metaphor to address the multiplicity of roles that healthcare practitioners undertaking qualitative research have to occupy and navigate. Rather than painting researchers out of their research through a naïve search for ‘objectivity’, reflexivity that is scaffolded by theory, offers a way through which researchers’ biases and subjectivities can be made explicit and their data analysis transparent. In making this argument, we encourage medical researchers to engage with, and be attuned to, theoretical perspectives outwith their own discipline.

EnTraP: A factorial randomised controlled trial embedded within world hip trauma evaluation eight COPAL investigating the effect of an enhanced trainee principal investigator package and digital nudge on recruitment rates

Background Randomised controlled trials (RCTs) often struggle with various aspects of participant recruitment, including engaging clinicians to recruit effectively, and subsequently fail to reach their target sample size. Studies evaluating interventions to improve recruitment aimed specifically at recruiters to the trial are limited in number. The RCTs embedded into the World Hip Trauma Evaluation (WHiTE) cohort study use Trainee Principal Investigators (TPIs) to help manage and drive recruitment at trial sites. No formalised training or support is provided by central trials units to the TPIs. Additionally, trial recruiters receive a generic automated email confirming randomisation to the trial with no other communication to influence or incentivise their behaviour to further recruit. The primary aim of this factorial trial was to evaluate the effectiveness of an educational intervention to TPIs and a positive reinforcement intervention via an email (digital) nudge on increasing recruitment. Secondary aims included feasibility of implementing the interventions and surveying TPIs on the educational package quality of content, delivery and ongoing support. Design This was a multicentre, open, cluster, 2x2 factorial RCT embedded in the WHiTE 8 COPAL RCT, in which research sites were randomised 1:1:1:1 to receive the enhanced TPI package, the digital nudge intervention, both, or neither. Results 1215 patients were recruited to the WHiTE 8 COPAL trial across 20 sites during the SWAT between August 2018 and March 2019. There was a statistically significant interaction between the interventions (IRR 2.09, 95% CI 1.64 to 2.68, p < 0.001). There was a statistically significant benefit on recruitment (IRR 1.23 95% 1.09 to 1.40, p=0.001) from utilizing an enhanced TPI education intervention. The digital nudge intervention had no significant impact on recruitment (IRR 0.89 95% CI 0.79 to 1.01, p=0.07). Within enhanced TPI package sites, the digital nudge had a beneficial effect, while in the standard practice TPI sites it had a detrimental effect. Feasibility analysis showed the median time to site digital nudge and enhanced TPI set up were one day and 17 days, respectively. 353 digital nudges were created taking an average of 12 min to construct, log the activity and then disseminate to recruiters. Median induction time for enhanced TPI was 32 min and 100% of the groups were extremely satisfied with the induction content, delivery and ongoing support. Discussion An education and support programme targeted at surgical TPIs involving a digital education package, 1:1 telephone induction and subsequent support package was effective in increasing recruitment in the first 6 months of trial commencement. There was no evidence for the effectiveness of the digital nudge intervention in isolation, although our results show that when combined with an education programme, it leads to enhanced effectiveness of that programme.

THE CABE PROJECT: Developing a model to conceptualise the sexual attitudes, behaviours, and experiences of men who have sex with men and Waria in Bali, Indonesia: Protocol for a mixed-methods design within a community-engaged research study

Introduction The prevalence of sexually transmitted infections (STIs) among MSM (men who have sex with men) and transgender women (waria) in Bali is of significant public health concern. According to the 2015 Integrated Biological and Behavioural Survey (IBBS), HIV prevalence among MSM in Denpasar (the capital city of Bali) was 36% – the highest prevalence in this population reported nationally. In addition, 26% of MSM and 25% of waria in Indonesia were living with HIV in 2015. There is limited research examining the attitudes, behaviours and experiences of MSM in Indonesia, and specifically in Bali. This study will develop a model to help understand the social-cultural context, attitudes, behaviours, and experiences to inform interventions to increase safe sex practices amongst Indonesian MSM and waria who live in Bali. Methods and analysis The community-engaged research (CEnR) study, employing a sequential mixed-methods approach, will engage MSM and waria community throughout all five phases. The first two phases will include partnership building and exploratory research (focus group discussions/FGDs). Phase three will include the administration of a survey ( n = 374 Indonesian MSM and waria). Phases four and five include explanatory (in-depth interviews) and dissemination phases, respectively. Multivariate analysis will be employed for the quantitative data (the survey) and thematic analysis will be used to analyse the qualitative data (FGDs and in-depth interviews). Discussion The findings of this CEnR will inform culturally congruent interventions for organisation working with MSM and transgender to promote safer sexual health practice and improve general well-being of this community.

Assessing quality and agreement of structured data in automatic versus manual abstraction of the electronic health record for a clinical epidemiology study

Objective We evaluate data agreement between an electronic health record (EHR) sample abstracted by automated characterization with a standard abstracted by manual review. Study Design and Setting We obtain data for an epidemiology cohort study using standard manual abstraction of the EHR and automated identification of the same patients using a structured algorithm to query the EHR. Summary measures of agreement (e.g., Cohen’s kappa) are reported for 12 variables commonly used in epidemiological studies. Results Best agreement between abstraction methods is observed among demographic characteristics such as age, sex, and race, and for positive history of disease. Poor agreement is found in missing data and negative history, suggesting potential impact for researchers using automated EHR characterization. EHR data quality depends upon providers, who may be influenced by both institutional and federal government documentation guidelines. Conclusion Automated EHR abstraction discrepancies may decrease power and increase bias; therefore, caution is warranted when selecting variables from EHRs for epidemiological study using an automated characterization approach. Validation of automated methods must also continue to advance in sophistication with other technologies, such as machine learning and natural language processing, to extract non-structured data from the EHR, for application to EHR characterization for clinical epidemiology.

Multilevel mediation analysis on time-to-event outcomes: Exploring racial/ethnic disparities in breast cancer survival in California

Background Third-variable effect refers to the effect from a third-variable that explains an observed relationship between an exposure and an outcome. Depending on whether there is a causal relationship from the exposure to the third variable, the third-variable is called a mediator or a confounder. The multilevel mediation analysis is used to differentiate third-variable effects from data of hierarchical structures. Data Collection and Analysis We developed a multilevel mediation analysis method to deal with time-to-event outcomes and implemented the method in the mlma R package. With the method, third-variable effects from different levels of data can be estimated. The method uses multilevel additive models that allow for transformations of variables to take into account potential nonlinear relationships among variables in the mediation analysis. We apply the proposed method to explore the racial/ethnic disparities in survival among patients diagnosed with breast cancer in California between 2006 and 2017, using both individual risk factors and census tract level environmental factors. The individual risk factors are collected by cancer registries and the census tract level factors are collected by the Public Health Alliance of Southern California in partnership with the Virginia Commonwealth University's Center on Society and Health. The National Cancer Institute work group linked variables at the census tract level with each patient and performed the analysis for this study. Results We found that the racial disparity in survival were mostly explained at the census tract level and partially explained at the individual level. The associations among variables were depicted. Conclusion: The multilevel mediation analysis method can be used to differentiate mediation/confounding effects for factors originated from different levels. The method is implemented in the R package mlma.

Conducting rapid qualitative research amongst people with experience of rough sleeping in London during the COVID-19 pandemic

In March 2020, the ‘Everyone In’ initiative was introduced by the UK government as a public health response to COVID-19. This initiative sought to temporarily accommodate people experiencing rough sleeping in hotels in all local authority areas throughout England. In London, ‘Everyone In’ involved the procurement of vacant accommodation in over 100 hotels and temporarily re-housed approximately 2000 individuals. A rapid qualitative study was undertaken within two hotels to explore experiences of the initiative from the perspective of people accommodated in the hotels. This article describes how standard qualitative methods were adapted and implemented to complete the study whilst meeting COVID-19 social distancing guidelines. The research involved a longitudinal design of a two-stage qualitative interview that sought to capture residents’ experience of ‘Everyone In’ at two points in time (while in the hotel and when residents had left the hotel). Adapted qualitative methods were employed by a team of 13 researchers. These adaptations included socially distanced leaflet dropping, telephone-based participant recruitment, a remote, multistage, longitudinal qualitative telephone interviewing and rapid framework analysis. 35 hotel residents were recruited into the study (two subsequently withdrew participation). A total of 299 (of a possible 330) short interviews were completed by 33 participants (26 male and 7 female) as part of the multi-stage, longitudinal design of the study. This study indicates that adapted qualitative research methods employed during a pandemic can be successfully applied to obtain insights and experiences (of individuals and groups) otherwise difficult to reach and/or complex to understand.

Exploring the external validity of survey data with triangulation: A case study from the Norwegian Offender Mental Health and Addiction (NorMA) Study

Objectives This paper demonstrates how to investigate the external validity of a study sample by triangulating survey and registry data, using data from the Norwegian Offender Mental Health and Addiction (NorMA) Study as a case. Methods We use survey data from the NorMA study ( n = 1495), including the NorMA cohort ( n = 733), and data from the Norwegian Prison Registry on all people imprisoned on 1 September 2013 ( n = 3386). Triangulation was performed by (1) comparing the NorMA cohort to those lost to follow-up ( n = 762), using survey data from the NorMA study. Secondly, we compared the NorMA cohort to the one-day population, using data from the Norwegian Prison Registry. We also stratified the one-day sample by possession of a Norwegian personal identification number (PIN). Results We found differences in birthplace, imprisonment and drug use between the NorMA cohort, lost to follow-up and the one-day population. Twenty-three percent of the one-day population did not have a Norwegian PIN. The NorMA cohort was more similar to those with a Norwegian PIN in the one-day population. Our triangulation demonstrates that 56–62% of the Norwegian prison population had an indication of drug use before imprisonment. Conclusions The NorMA cohort was overall representative of the one-day prison population holding a Norwegian PIN and less representative of prisoners without a Norwegian PIN. Using this method provides tangible inputs on the strengths and limitations of a study sample and can be a feasible method to investigate the external validity of survey data.

Measuring the sensitivity of difference-in-difference estimates to the parallel trends assumption

Background. Difference-in-Difference makes a critical assumption that the changes in the outcomes, over the post-treatment period, are similar between the treated and control groups—the parallel trends assumption. Evaluation of this assumption is often done either by graphical examination or by statistical tests in the pre-treatment period. They result in a binary conclusion about the validity of the assumption. Purpose. This paper proposes a sensitivity analysis that quantifies the departure from parallel trends necessary to meaningfully change the estimated treatment effect. Results. Sensitivity analyses have an advantage over traditional parallel trends tests: they use all available data and thereby work even if only one pre-period is available, and they quantify the strength of unobserved confounder(s) required to change the conclusions of a study. Conclusions. We apply the sensitivity analysis metrics developed by Cinelli and Hazlett (2020) and illustrate them on two studies.