Personalizing Evidence for Shoulder Fracture Patients Using an Extended Instrumental Variable Causal Forest Algorithm

Objective: To assess the ability of an extended Instrumental Variable Causal Forest Algorithm (IV-CFA) to provide personalized evidence of early surgery effects on benefits and detriments for elderly shoulder fracture patients. Data Sources/Study Setting: Population of 72,751 fee-for-service Medicare beneficiaries with proximal humerus fractures (PHFs) in 2011 who survived a 60-day treatment window after an index PHF and were continuously Medicare fee-for-service eligible over the period 12 months prior to index to the minimum of 12 months after index or death. Study Design: IV-CFA estimated early surgery effects on both beneficial and detrimental outcomes for each patient in the study population. Classification and regression trees (CART) were applied to these estimates to create patient reference classes. Two-stage least squares (2SLS) estimators were applied to patients in each reference class to scrutinize the estimates relative to the known 2SLS properties. Principal Findings: This approach uncovered distinct reference classes of elderly PHF patients with respect to early surgery effects on benefit and detriment. Older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to gain benefit and more likely to have detriment from early surgery. Reference classes were characterized by the appropriateness of early surgery rates with respect to benefit and detriment. Conclusions: Extended IV-CFA provides an illuminating method to uncover reference classes of patients based on treatment effects using observational data with a strong instrumental variable. This study isolated reference classes of new PHF patients in which changes in early surgery rates would improve patient outcomes. The inability to measure fracture complexity in Medicare claims means providers will need to discuss the appropriateness of these estimates to patients within a reference class in context of this missing information.

Generalizations and Reference Classes

This chapter examines the implications of the reference-class problem for attempts to model the probative value of evidence in mathematical terms. This examination makes three distinct contributions to evidence scholarship. First, and most importantly, it articulates and explains the problematic relationship between algorithmic tools and legal decision-making. Second, it points out serious pitfalls to be avoided for analytical or empirical studies of juridical proof. Third, it indicates when algorithmic tools may be more or less useful in the evidentiary process. As such, the chapter offers yet another demonstration of the very complex set of relationships involving human knowledge and rationality, on the one hand, and attempts to reduce either to a set of formal concepts, on the other.

Personalizing Evidence for Shoulder Fracture Patients Using an Extended Instrumental Variable Causal Forest Algorithm

Objective: To assess the ability of an extended Instrumental Variable Causal Forest Algorithm (IV-CFA) to provide personalized evidence of early surgery effects on benefits and detriments for elderly shoulder fracture patients. Data Sources/Study Setting: Population of 72,751 fee-for-service Medicare beneficiaries with proximal humerus fractures (PHFs) in 2011 who survived a 60-day treatment window after an index PHF and were continuously Medicare fee-for-service eligible over the period 12 months prior to index to the minimum of 12 months after index or death. Study Design: IV-CFA estimated early surgery effects on both beneficial and detrimental outcomes for each patient in the study population. Classification and regression trees (CART) were applied to these estimates to create patient reference classes. Two-stage least squares (2SLS) estimators were applied to patients in each reference class to scrutinize the estimates relative to the known 2SLS properties. Principal Findings: This approach uncovered distinct reference classes of elderly PHF patients with respect to early surgery effects on benefit and detriment. Older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to gain benefit and more likely to have detriment from early surgery. Reference classes were characterized by the appropriateness of early surgery rates with respect to benefit and detriment. Conclusions: Extended IV-CFA provides an illuminating method to uncover reference classes of patients based on treatment effects using observational data with a strong instrumental variable. This study isolated reference classes of new PHF patients in which changes in early surgery rates would improve patient outcomes. The inability to measure fracture complexity in Medicare claims means providers will need to discuss the appropriateness of these estimates to patients within a reference class in context of this missing information.

Correction to: Graphing and reporting heterogeneous treatment effects through reference classes

An amendment to this paper has been published and can be accessed via the original article.

Graphing and reporting heterogeneous treatment effects through reference classes

Background Exploration and modelling of heterogeneous treatment effects as a function of baseline covariates is an important aspect of precision medicine in randomised controlled trials (RCTs). Randomisation generally guarantees the internal validity of an RCT, but heterogeneity in treatment effect can reduce external validity. Estimation of heterogeneous treatment effects is usually done via a predictive model for individual outcomes, where one searches for interactions between treatment allocation and important patient baseline covariates. However, such models are prone to overfitting and multiple testing and typically demand a transformation of the outcome measurement, for example, from the absolute risk in the original RCT to log-odds of risk in the predictive model. Methods We show how reference classes derived from baseline covariates can be used to explore heterogeneous treatment effects via a two-stage approach. We first estimate a risk score which captures on a single dimension some of the heterogeneity in outcomes of the trial population. Heterogeneity in the treatment effect can then be explored via reweighting schemes along this axis of variation. This two-stage approach bypasses the search for interactions with multiple covariates, thus protecting against multiple testing. It also allows for exploration of heterogeneous treatment effects on the original outcome scale of the RCT. This approach would typically be applied to multivariable models of baseline risk to assess the stability of average treatment effects with respect to the distribution of risk in the population studied. Case study We illustrate this approach using the single largest randomised treatment trial in severe falciparum malaria and demonstrate how the estimated treatment effect in terms of absolute mortality risk reduction increases considerably in higher risk strata. Conclusions ‘Local’ and ‘tilting’ reweighting schemes based on ranking patients by baseline risk can be used as a general approach for exploring, graphing and reporting heterogeneity of treatment effect in RCTs. Trial registration ISRCTN clinical trials registry: ISRCTN50258054. Prospectively registered on 22 July 2005.

More Accurate Learning of k-DNF Reference Classes

In machine learning, predictors trained on a given data distribution are usually guaranteed to perform well for further examples from the same distribution on average. This often may involve disregarding or diminishing the predictive power on atypical examples; or, in more extreme cases, a data distribution may be composed of a mixture of individually “atypical” heterogeneous populations, and the kind of simple predictors we can train may find it difficult to fit all of these populations simultaneously. In such cases, we may wish to make predictions for an atypical point by selecting a suitable reference class for that point: a subset of the data that is “more similar” to the given query point in an appropriate sense. Closely related tasks also arise in applications such as diagnosis or explaining the output of classifiers. We present new algorithms for computing k-DNF reference classes and establish much stronger approximation guarantees for their error rates.

Health and Reference Classes

In this article, I address two objections developed by Kingma against Boorse’s (1977) bio-statistical theory of health, the objections that choice of reference classes renders the theory both circular and problematically value-laden. These objections not only apply to the bio-statistical theory of health but also to other naturalistic theories, like the dispositional theory of health. I present three rejoinders. First, I argue that the circularity objection arises from excessive methodological demands. Second, I argue that naturalists can resist the normativist claim that health and pathology are differentiated on the basis of personal or cultural values. Finally, I show that it is possible to justify choices between rival theories of health without the interference of evaluative commitments. With these rejoinders, I conclude that the bio-statistical theory, as well as other naturalistic theories of health utilizing reference classes, is not undermined by Kingma’s arguments.

Graphing and reporting heterogeneous treatment effects through reference classes

BackgroundExploration and modelling of individual treatment effects and treatment heterogeneity is an important aspect of precision medicine in randomized controlled trials (RCTs). The usual approach is to develop a predictive model for individual outcomes and then look for an interaction effect between treatment allocation and important patient covariates. However, such models are prone to overfitting and multiple testing, and typically demand a transformation of the outcome measurement, for example, from the absolute risk in the original RCT to log-odds of risk in the predictive model.MethodsWe show how reference classes derived from background information can be used to alleviate this problem through a two-stage approach where we first estimate a key aspect of heterogeneity in the trial population and then explore for an interaction with the treatment effect along this axis of variation. This bypasses the search for interactions, protecting against multiple testing, and allows for exploration of heterogeneous treatment effects on the original outcome scale of the RCT. This would typically be applied to multivariate models of baseline risk to assess the stability of average treatment effects with respect to the distribution of risk in the population studied. We show how ‘local’ and ‘tilting’ schemes based on ranking patients by baseline risk can be used as a general approach for exploring heterogeneity of treatment effect.ResultsWe illustrate this approach using the single largest randomised treatment trial in severe falciparum malaria and show how the estimated treatment effect in terms of absolute mortality risk reduction increases considerably for higher risk strata.

A Stage-Theoretical Account of Diachronic Identity

Diachronic identity is understood as an identity holding between something existing at one time and something existing at another time. On the stage view, however, ordinary objects are instantaneous stages that do not exist at other times so diachronic identity is, at best, problematic. On account proposed here a name does not, as Sider and others suggest, denote a stage concurrent with its utterance. Rather, at any time, t, a name of an ordinary object designates a stage-at-t as its primary referent and refers indeterminately over it and all and only those stages counterpart-related to it—its reference class at t. At any time, t, a at t1 is the same object as b at t2 iff for every stage x counterpart-related to a’s stage-at t and every stage y counterpart-related to b’s stage-at-t, x=y. Diachronic identity statements, therefore, assert strict identities—between concurrent stages. Ordinarily names select the same reference classes at every time so, in ordinary cases, identity statements are not ‘occasional’. In fission cases names select different reference classes at different times. Where a becomes b and c, at any pre-fission time ‘a’, ‘b’, and ‘c’ have same the primary referent and so select the same reference class—therefore, before fission b is the same object as c. At any post-fission time ‘b’ and ‘c’ select different reference classes—so after fission b is not the same object as c. Identity is not occasional but, in extraordinary cases, identity statements are—because reference is.