scholarly journals To Include, or Not Include, that is the Question: An Empirical Analysis of Dealing with Patients who are Lost to Follow-up when Developing Prognostic Models Using a Cohort Design

2020 ◽  
Author(s):  
Jenna M Reps ◽  
Peter Rijnbeek ◽  
Alana Cuthbert ◽  
Patrick B Ryan ◽  
Nicole Pratt ◽  
...  
Keyword(s):  
At Risk ◽  
Synthetic Data ◽  
Model Performance ◽  
Test Set ◽  
Loss To Follow Up ◽  
Using Data ◽  
Lost To Follow Up ◽  
The Impact ◽  
Prediction Problems

Abstract Background: Researchers developing prediction models are faced with numerous design choices that may impact model performance. One of the main decisions is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up. Methods: We generate a synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. We investigate four simple strategies for developing models using data containing some patients with loss to follow-up. Three strategies employ a binary classifier with data that: i) include all patients (including those lost to follow-up), ii) exclude all patients lost to follow-up or iii) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. In addition to our synthetic data study, we empirically evaluate the discrimination and calibration performance of these strategies across 21 prediction problems using real-world data. Results: The synthetic data study results show that excluding patients lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on the model performance. Our empirical results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year, but demonstrated differential bias when we looking into 3-year time-at-risk. Removing patients who are lost to follow-up before the outcome but keeping patients who are loss to follow-up after the outcome can bias a model and should be avoided. Conclusion: Based on this study we therefore recommend i) developing models using data that includes patients that are lost to follow-up and ii) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

scholarly journals An empirical analysis of dealing with patients who are lost to follow-up when developing prognostic models using a cohort design

2020 ◽  
Author(s):  
Jenna M Reps ◽  
Peter Rijnbeek ◽  
Alana Cuthbert ◽  
Patrick B Ryan ◽  
Nicole Pratt ◽  
...  
Keyword(s):  
At Risk ◽  
Real World ◽  
Synthetic Data ◽  
Model Performance ◽  
Real World Data ◽  
Test Set ◽  
Loss To Follow Up ◽  
Cohort Design ◽  
Lost To Follow Up

Abstract Background: Researchers developing prediction models are faced with numerous design choices that may impact model performance. One key decision is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up.Methods: We generate a partially synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. In addition to our synthetic data study we investigate 21 real-world data prediction problems. We compare four simple strategies for developing models when using a cohort design that encounters loss to follow-up. Three strategies employ a binary classifier with data that: i) include all patients (including those lost to follow-up), ii) exclude all patients lost to follow-up or iii) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. We empirically evaluate the discrimination and calibration performance.Results: The partially synthetic data study results show that excluding patients who are lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on the model performance. Our empirical real-world data results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year, but demonstrated differential bias when we looked into 3-year time-at-risk. Removing patients who are lost to follow-up before experiencing the outcome but keeping patients who are lost to follow-up after the outcome can bias a model and should be avoided.Conclusion: Based on this study we therefore recommend i) developing models using data that includes patients that are lost to follow-up and ii) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

scholarly journals An empirical analysis of dealing with patients who are lost to follow-up when developing prognostic models using a cohort design

2020 ◽  
Author(s):  
Jenna M Reps ◽  
Peter Rijnbeek ◽  
Alana Cuthbert ◽  
Patrick B Ryan ◽  
Nicole Pratt ◽  
...  
Keyword(s):  
At Risk ◽  
Real World ◽  
Synthetic Data ◽  
Real World Data ◽  
Test Set ◽  
Loss To Follow Up ◽  
World Data ◽  
Cohort Design ◽  
Lost To Follow Up

Abstract Background: Researchers developing prediction models are faced with numerous design choices that may impact model performance. One key decision is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up.Methods: We generate a partially synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. In addition to our synthetic data study we investigate 21 real-world data prediction problems. We compare four simple strategies for developing models when using a cohort design that encounters loss to follow-up. Three strategies employ a binary classifier with data that: i) include all patients (including those lost to follow-up), ii) exclude all patients lost to follow-up or iii) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. We empirically evaluate the discrimination and calibration performance.Results: The partially synthetic data study results show that excluding patients who are lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on model discrimination performance. Our empirical real-world data results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year but demonstrated differential bias when we looked into 3-year time-at-risk. Removing patients who are lost to follow-up before experiencing the outcome but keeping patients who are lost to follow-up after the outcome can bias a model and should be avoided.Conclusion: Based on this study we therefore recommend i) developing models using data that includes patients that are lost to follow-up and ii) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

scholarly journals An empirical analysis of dealing with patients who are lost to follow-up when developing prognostic models using a cohort design

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Jenna M. Reps ◽  
Peter Rijnbeek ◽  
Alana Cuthbert ◽  
Patrick B. Ryan ◽  
Nicole Pratt ◽  
...  
Keyword(s):  
At Risk ◽  
Real World ◽  
Synthetic Data ◽  
Real World Data ◽  
Test Set ◽  
Loss To Follow Up ◽  
World Data ◽  
Cohort Design ◽  
Lost To Follow Up

Abstract Background Researchers developing prediction models are faced with numerous design choices that may impact model performance. One key decision is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up. Methods We generate a partially synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. In addition to our synthetic data study we investigate 21 real-world data prediction problems. We compare four simple strategies for developing models when using a cohort design that encounters loss to follow-up. Three strategies employ a binary classifier with data that: (1) include all patients (including those lost to follow-up), (2) exclude all patients lost to follow-up or (3) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. We empirically evaluate the discrimination and calibration performance. Results The partially synthetic data study results show that excluding patients who are lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on model discrimination performance. Our empirical real-world data results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year but demonstrated differential bias when we looked into 3-year time-at-risk. Removing patients who are lost to follow-up before experiencing the outcome but keeping patients who are lost to follow-up after the outcome can bias a model and should be avoided. Conclusion Based on this study we therefore recommend (1) developing models using data that includes patients that are lost to follow-up and (2) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

scholarly journals Assessing the differences in characteristics of patients lost to follow-up at 2 years: results from the Quality Outcomes Database study on outcomes of surgery for grade I spondylolisthesis

2020 ◽  
Vol 33 (5) ◽  
pp. 643-651
Author(s):  
Erica F. Bisson ◽  
Praveen V. Mummaneni ◽  
John Knightly ◽  
Mohammed Ali Alvi ◽  
Anshit Goyal ◽  
...  
Keyword(s):  
Degenerative Spondylolisthesis ◽  
Leg Pain ◽  
Quality Outcomes ◽  
Loss To Follow Up ◽  
Clinical Registries ◽  
Pain Scores ◽  
Patient Reported ◽  
Lost To Follow Up ◽  
The Impact

OBJECTIVELoss to follow-up has been shown to bias outcomes assessment among studies utilizing clinical registries. Here, the authors analyzed patients enrolled in a national surgical registry and compared the baseline characteristics of patients captured with those lost to follow-up at 2 years.METHODSThe authors queried the Quality Outcomes Database for patients with grade I lumbar degenerative spondylolisthesis undergoing a surgical intervention between July 2014 and June 2016. Only those patients enrolled in a multisite study investigating the impact of fusion on clinical and patient-reported outcomes (PROs) among patients with grade I spondylolisthesis were evaluated.RESULTSOf the 608 patients enrolled in the study undergoing 1- or 2-level decompression (23.0%, n = 140) or 1-level fusion (77.0%, n = 468), 14.5% (n = 88) were lost to follow-up at 2 years. Patients who were lost to follow-up were more likely to be younger (59.6 ± 13.5 vs 62.6 ± 11.7 years, p = 0.031), be employed (unemployment rate: 53.3% [n = 277] for successful follow-up vs 40.9% [n = 36] for those lost to follow-up, p = 0.017), have anxiety (26.1% [n = 23] vs 16.3% [n = 85], p = 0.026), have higher back pain scores (7.4 ± 2.9 vs 6.6 ± 2.8, p = 0.010), have higher leg pain scores (7.4 ± 2.5 vs 6.4 ± 2.9, p = 0.003), have higher Oswestry Disability Index scores (50.8 ± 18.7 vs 46 ± 16.8, p = 0.018), and have lower EQ-5D scores (0.481 ± 0.2 vs 0.547 ± 0.2, p = 0.012) at baseline.CONCLUSIONSTo execute future, high-quality studies, it is important to identify patients undergoing surgery for spondylolisthesis who might be lost to follow-up. In a large, prospective registry, the authors found that those lost to follow-up were more likely to be younger, be employed, have anxiety disorder, and have worse PRO scores.

Implementation of the MASK-Air® App for Rhinitis and Asthma in Older Adults: MASK@Puglia Pilot Study

2021 ◽  
pp. 1-6
Author(s):  
Maria Teresa Ventura ◽  
Antonio Francesco Maria Giuliano ◽  
Rosalba Buquicchio ◽  
Anna Bedbrook ◽  
Wienczyslawa Czarlewski ◽  
...  
Keyword(s):  
Older Adults ◽  
Computer Literacy ◽  
Training Session ◽  
Loss To Follow Up ◽  
Item Questionnaire ◽  
Low Education ◽  
Compliance To Treatment ◽  
Lost To Follow Up ◽  
The Impact

<b><i>Introduction:</i></b> MASK-air® is an app whose aim is to reduce the global burden of allergic rhinitis and asthma. A transfer of innovative practices was performed to disseminate and implement MASK-air® in European regions. The aim of the study was to examine the implementation of the MASK-air® app in older adults of the Puglia TWINNING in order to investigate (i) the rate of acceptance in this population, (ii) the reasons for refusal and (iii) the evaluation of the app after its use. <b><i>Methods:</i></b> All consecutive geriatric patients aged between 65 and 90 years were included by the outpatient clinic of the Bari Geriatric Immunoallergology Unit. After a 1-h training session, older adults used the app for 6 months. A 6-item questionnaire was developed by our unit to evaluate the impact of the app on the management of the disease and its treatment. <b><i>Results:</i></b> Among the 174 recruited patients, 102 accepted to use the app (mean age, SD: 72.4 ± 4.6 years), 6 were lost to follow-up, and 63 had a low education level. The reasons given not to use the app included lack of interest (11%), lack of access to a smartphone or tablet (53%), low computer literacy (28%), and distrust (8%). At follow-up, the overall satisfaction was high (89%), the patient considered MASK-air® “advantageous” (95%), compliance to treatment was improved (81%), and the rate of loss to follow-up had decreased to 6%. <b><i>Conclusion:</i></b> Older adults with a low level of education can use the MASK-air® app after a short training session.

Untreated congenital hypothyroidism due to loss to follow-up: developing preventive strategies through quality improvement

2018 ◽  
Vol 31 (9) ◽  
pp. 987-994
Author(s):  
Kristal Anne Matlock ◽  
Sarah Dawn Corathers ◽  
Nana-Hawa Yayah Jones
Keyword(s):  
At Risk ◽  
Quality Improvement ◽  
Congenital Hypothyroidism ◽  
Practice Guidelines ◽  
Provider Education ◽  
Loss To Follow Up ◽  
Patients Âgés ◽  
Provider Adherence ◽  
Lost To Follow Up

Abstract Background Children with congenital hypothyroidism (CH) are at risk for preventable intellectual disability without adequate medical management. The purpose of this manuscript is to discuss quality improvement (QI)-based processes for improving provider adherence to practice guidelines and ultimately identifying at-risk patients with chronic illness prior to the occurrence of adverse events. Methods Our study population included patients ages ≤3 years diagnosed with CH; lost to follow-up was defined as >180 days since last evaluation by an endocrinology provider. Iterative testing of interventions focused on establishing standardized care through (1) registry-based identification, (2) scheduling future appointments during current visits, (3) outreach to patients lost to follow-up and (4) provider and family education of current practice guidelines. Results A population-validated, electronic medical registry identified approximately 100 patients ages ≤3 years diagnosed with CH; initially, 12% of patients met criteria for lost to follow-up. Through serial testing of interventions, the rate of loss to follow-up declined to the goal of <5% within 8 months. Additional measures showed improvement in provider adherence to standard of care. All patients identified as lost to follow-up initially were seen within the first 3 months of intervention. Conclusions Applying QI methodology, a multidisciplinary team implemented a process to identify and contact high-risk CH patients with inadequate follow-up. Focused interventions targeting population management, scheduling and patient/provider education yield sustained improvement in the percentage of patients with a chronic condition who are lost to follow-up.

scholarly journals Predictors of loss to follow-up in art experienced patients in Nigeria: a 13 year review (2004–2017)

2019 ◽  
Vol 16 (1) ◽  
Author(s):  
Ahmad Aliyu ◽  
Babatunde Adelekan ◽  
Nifarta Andrew ◽  
Eunice Ekong ◽  
Stephen Dapiap ◽  
...  
Keyword(s):  
Patient Adherence ◽  
Cross Sectional Study ◽  
Emergency Plan ◽  
Cross Sectional ◽  
Loss To Follow Up ◽  
Viral Loads ◽  
Review Period ◽  
Lost To Follow Up ◽  
Differentiated Care

Abstract Background Expanded access to antiretroviral therapy (ART) leads to improved HIV/AIDS treatment outcomes in Nigeria, however, increasing rates of loss to follow-up among those on ART is threatening optimal standard achievement. Therefore, this retrospective cross-sectional study is aimed at identifying correlates and predictors of loss to follow-up in patients commencing ART in a large HIV program in Nigeria. Methods Records of all patients from 432 US CDC Presidents Emergency Plan for AIDS Relief (PEPFAR) supported facilities across 10 States and FCT who started ART from 2004 to 2017 were used for this study. Bivariate and multivariate analysis of the demographic and clinical parameters of all patients was conducted using STATA version 14 to determine correlates and predictors of loss to follow-up. Results Within the review period, 245,257 patients were ever enrolled on anti-retroviral therapy. 150,191 (61.2%) remained on treatment, 10,960 (4.5%) were transferred out to other facilities, 6926 (2.8%) died, 2139 (0.9%) self-terminated treatment and 75,041 (30.6%) had a loss to follow-up event captured. Males (OR: 1.16), Non-pregnant female (OR: 4.55), Patients on ≥ 3-monthly ARV refills (OR: 1.32), Patients with un-suppressed viral loads on ART (OR: 4.52), patients on adult 2nd line regimen (OR: 1.23) or pediatric on 1st line regimen (OR: 1.70) were significantly more likely to be lost to follow-up. Conclusion Despite increasing access to anti-retroviral therapy, loss to follow-up is still a challenge in the HIV program in Nigeria. Differentiated care approaches that will focus on males, non-pregnant females and paediatrics is encouraged. Reducing months of Anti-retroviral drug refill to less than 3 months is advocated for increased patient adherence.

Rural Barriers to Surgical Care for Children With Sleep-Disordered Breathing

2021 ◽  
pp. 019459982199338
Author(s):  
Flora Yan ◽  
Dylan A. Levy ◽  
Chun-Che Wen ◽  
Cathy L. Melvin ◽  
Marvella E. Ford ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Regression Analysis ◽  
Sleep Disordered Breathing ◽  
Healthy Children ◽  
Loss To Follow Up ◽  
Cox Regression Analysis ◽  
Obstructive Sleep ◽  
The Impact ◽  
Disordered Breathing

Objective To assess the impact of rural-urban residence on children with obstructive sleep-disordered breathing (SDB) who were candidates for tonsillectomy with or without adenoidectomy (TA). Study Design Retrospective cohort study. Setting Tertiary children’s hospital. Methods A cohort of otherwise healthy children aged 2 to 18 years with a diagnosis of obstructive SDB between April 2016 and December 2018 who were recommended TA were included. Rural-urban designation was defined by ZIP code approximation of rural-urban commuting area codes. The main outcome was association of rurality with time to TA and loss to follow-up using Cox and logistic regression analyses. Results In total, 213 patients were included (mean age 6 ± 2.9 years, 117 [55%] male, 69 [32%] rural dwelling). Rural-dwelling children were more often insured by Medicaid than private insurance ( P < .001) and had a median driving distance of 74.8 vs 16.8 miles ( P < .001) compared to urban-dwelling patients. The majority (94.9%) eventually underwent recommended TA once evaluated by an otolaryngologist. Multivariable logistic regression analysis did not reveal any significant predictors for loss to follow-up in receiving TA. Cox regression analysis that adjusted for age, sex, insurance, and race showed that rural-dwelling patients had a 30% reduction in receipt of TA over time as compared to urban-dwelling patients (hazard ratio, 0.7; 95% CI, 0.50-0.99). Conclusion Rural-dwelling patients experienced longer wait times and driving distance to TA. This study suggests that rurality should be considered a potential barrier to surgical intervention and highlights the need to further investigate geographic access as an important determinant of care in pediatric SDB.

Prospective cohort study of Kaposi sarcoma treated under real-world conditions in Malawi.

2021 ◽  
Vol 39 (15_suppl) ◽  
pp. 11569-11569
Author(s):  
Edwards Kasonkanji ◽  
Yolanda Gondwe ◽  
Morgan Dewey ◽  
Joe Gumulira ◽  
Matthew Painschab ◽  
...  
Keyword(s):  
Cohort Study ◽  
Kaposi Sarcoma ◽  
Median Number ◽  
Sub Saharan Africa ◽  
Worst Case ◽  
Loss To Follow Up ◽  
Presenting Symptoms ◽  
Grade 3 ◽  
Lost To Follow Up

11569 Background: Kaposi sarcoma (KS) is the leading cancer in Malawi (34% of cancers). Outside of clinical trials, prospective KS studies from sub-Saharan Africa (SSA) are few and limited by loss to follow up. We conducted a prospective KS cohort study of standard of care bleomycin/vincristine (BV) at Lighthouse HIV clinic, in Lilongwe, Malawi. Methods: We enrolled pathologically confirmed, newly diagnosed, HIV+ KS patients from Feb 2017 to Jun 2019. We collected clinical and treatment characteristics, toxicity, and outcomes of KS with follow-up censored Jun 2020. Patients were treated with bleomycin (25 mg/m2) and vincristine (0.4 mg/m2) every 14 days for a planned maximum of 16 cycles. STATA v13.0 was used to calculate descriptive statistics and Kaplan Meier survival analysis. Toxicity was graded using NCI CTCAE v5.0. Results: We enrolled 138 participants, median age 36 (IQR 32-44) and 110 (80%) male. By ACTG staging, 107 (78%) were T1 (tumour severity), 46 (33%) were S1 (illness severity) and 46 (33%) had Karnofsky performance status ≤70. Presenting symptoms included edema in 69 (53%), visceral disease in 9 (7%), and oral involvement in 43 (33%). Prior to KS diagnosis, 70 (51%) participants were aware of being HIV+ for median 17 months (IQR 6-60) and had been on ART for median 16 months (IQR 6-60). Median CD4 count was 197 (IQR 99-339), median HIV-viral load was 2.6 log copies/mL (IQR 1.6 – 4.8) and 57% were HIV-suppressed ( < 1000 HIV copies/ml). The median number of cycles was 16 (IQR 7-16). 62 (45%) participants missed at least one dose due to stock out. Amongst patients with missed doses, the median number was 3 (IQR 2-4) for bleomycin and 2 (IQR 1-3) for vincristine. 14 (10%) participants experienced at least one reduced dose due to toxicity. 5 (4%) participants suffered grade ≥3 anaemia, 13 (9%) grade ≥3 neutropenia, and one participant had grade 4 bleomycin-induced dermatitis. There was no reported grade ≥3 bleomycin lung toxicity or vincristine-induced neuropathy. Of 115 evaluable participants, responses at the end of therapy were: complete response in 52 (45%), partial response in 27 (23%) stable disease in 5 (4%), and progressive disease in 31 (28%). Median duration of follow-up was 20 months. At censoring, 69 (50%) were alive, 36 (26%) dead, and 33 (24%) lost to follow-up. Overall survival is shown Table as crude and worst-case scenario; worst-case assumes all participants lost to follow up died. Conclusions: Here, we present one of the most complete characterizations of KS presentation and treatment from SSA. As in other studies from the region, the majority of patients presented with advanced disease, chemotherapy stock-outs and loss to follow up were common, and mortality was high. Studies are planned to understand the virologic characteristics, improve therapies, and better implement existing therapies.[Table: see text]

Sign in / Sign up

Export Citation Format

Share Document