EXPRESS: The More You Ask, the Less You Get: When Additional Questions Hurt External Validity

2021 ◽  
pp. 002224372110735
Author(s):  
Ye Li ◽  
Antonia Krefeld-Schwalb ◽  
Daniel G. Wall ◽  
Eric J. Johnson ◽  
Olivier Toubia ◽  
...  
Keyword(s):  
Real World ◽  
External Validity ◽  
Time Preference ◽  
Preference Elicitation ◽  
Decision Processes ◽  
Preference Measurement ◽  
Target Behaviors ◽  
Conjoint Choice ◽  
The Cost ◽  
Choice Tasks

Researchers and practitioners in marketing, economics, and public policy often use preference elicitation tasks to forecast real-world behaviors. These tasks typically ask a series of similarly-structured questions. The authors posit that every time a respondent answers an additional elicitation question, two things happen: (1) they provide information about some parameter(s) of interest, such as their time preference or the partworth for a product attribute, and (2) the respondent increasingly adapts to the task—i.e., using task-specific decision processes specialized for this task that may or may not apply to other tasks. Importantly, adaptation comes at the cost of potential mismatch between the task-specific decision process and real-world processes that generate the target behaviors, such that asking more questions can reduce external validity. The authors used mouse- and eye-tracking to trace decision processes in time preference measurement and conjoint choice tasks: Respondents increasingly relied on task-specific decision processes as more questions were asked, leading to reduced external validity for both related tasks and real-world behaviors. Importantly, the external validity of measured preferences peaked after as few as seven questions in both types of tasks. When measuring preferences, less can be more.

A Preregistered Validation Study of the Taylor Aggression Paradigm

2017 ◽  
Author(s):  
David Skylan Chester
Keyword(s):  
Aggressive Behavior ◽  
Validation Study ◽  
Real World ◽  
External Validity ◽  
State Level ◽  
Taylor Aggression Paradigm ◽  
Psychological Science ◽  
Laboratory Measure ◽  
Behavioral Tasks ◽  
Preliminary Support

The Taylor Aggression Paradigm (TAP) is a frequently-used laboratory measure of aggressive behavior. However, the flexibility inherent in its implementation and analysis can undermine its validity. To test whether the TAP was a valid aggression measure irrespective of this flexibility, I conducted a preregistered study of a 25-trial version of the TAP using a single scoring approach with 160 diverse undergraduate participants. TAP scores showed agreement with other laboratory aggression measures and were magnified by an experimental provocation manipulation. Mixed evidence was found for associations with aggressive dispositions and real-world violence. These results provide preliminary support for this approach to the TAP to measure state-level aggressive behavior. However, more evidence is needed to assess the TAP’s external validity and ability to measure dispositional forms of aggression. Using preregistered designs, researchers should validate specific variants of their behavioral tasks in order to optimize the veridicality and reproducibility of psychological science.

scholarly journals External validity of somatostatin analogues trials in advanced neuroendocrine neoplasms: the GETNE-TRASGU study

2021 ◽  
Author(s):  
Paula Jimenez-Fonseca ◽  
Alberto Carmona-Bayonas ◽  
Angela Lamarca ◽  
Jorge Barriuso ◽  
Angel Castaño ◽  
...  
Keyword(s):  
Real World ◽  
External Validity ◽  
Randomized Clinical Trials ◽  
Cox Model ◽  
Analytical Data ◽  
Somatostatin Analogues ◽  
Unknown Primary ◽  
Octreotide Lar ◽  
Well Differentiated ◽  
Lanreotide Autogel

Introduction: Somatostatin analogues (SSA) prolong progression-free survival (PFS) in patients with well-differentiated gastroenteropancreatic neuroendocrine tumors (GEP-NETs). However, the eligibility criteria in randomized clinical trials (RCTs) have been restricted, which contrasts with the vast heterogeneity found in NETs. Methods: We identified patients with well-differentiated (Ki67% ≤20%), metastatic GEP-NETs treated in first-line with SSA monotherapy from the Spanish R-GETNE registry. The therapeutic effect was evaluated using a Bayesian Cox model. The objective was to compare survival-based outcomes from real world clinical practice versus RCTs. Results: The dataset contained 535 patients with a median age of 62 years (range: 26-89). The median Ki67% was 4 (range: 0-20). The most common primary tumor sites were: midgut, 46%; pancreas, 34%; unknown primary, 10%; and colorectal, 10%. Half of the patients received octreotide LAR (n=266) and half, lanreotide autogel (n=269). The median PFS was 28.0 months (95% CI, 22.1-32.0) for octreotide vs 30.1 months (95% CI, 23.1-38.0) for lanreotide. The overall hazard ratio for lanreotide vs octreotide was 0.90 (95% credible interval, 0.71-1.12). The probability of effect sizes >30% with lanreotide vs octreotide was 2% and 6% for midgut and foregut NENs, respectively. Conclusion: Our study evaluated the external validity of RCTs examining SSAs in the real world, as well as the main effect-modifying factors (progression status, symptoms, tumor site, specific metastases, and analytical data).. Our results indicate that both octreotide LAR and lanreotide autogel had a similar effect on PFS. Consequently, both represent valid alternatives in patients with well-differentiated, metastatic GEP-NENs.

The Cost-Effectiveness of Pre-school Peanut Oral Immunotherapy in the Real World Setting

2021 ◽  
Author(s):  
Marcus Shaker ◽  
Edmond S. Chan ◽  
Jennifer LP. Protudjer ◽  
Lianne Soller ◽  
Elissa M. Abrams ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Real World ◽  
Oral Immunotherapy ◽  
The Real ◽  
Real World Setting ◽  
The Cost

Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution

2021 ◽  
pp. 1-27 ◽  
Author(s):  
Brandon de la Cuesta ◽  
Naoki Egami ◽  
Kosuke Imai
Keyword(s):  
Conjoint Analysis ◽  
Real World ◽  
External Validity ◽  
Causal Effect ◽  
The Other ◽  
Estimation Methods ◽  
Social Scientists ◽  
Actual Distribution ◽  
Open Source Software Package ◽  
Profile Distribution

Abstract Conjoint analysis has become popular among social scientists for measuring multidimensional preferences. When analyzing such experiments, researchers often focus on the average marginal component effect (AMCE), which represents the causal effect of a single profile attribute while averaging over the remaining attributes. What has been overlooked, however, is the fact that the AMCE critically relies upon the distribution of the other attributes used for the averaging. Although most experiments employ the uniform distribution, which equally weights each profile, both the actual distribution of profiles in the real world and the distribution of theoretical interest are often far from uniform. This mismatch can severely compromise the external validity of conjoint analysis. We empirically demonstrate that estimates of the AMCE can be substantially different when averaging over the target profile distribution instead of uniform. We propose new experimental designs and estimation methods that incorporate substantive knowledge about the profile distribution. We illustrate our methodology through two empirical applications, one using a real-world distribution and the other based on a counterfactual distribution motivated by a theoretical consideration. The proposed methodology is implemented through an open-source software package.

Estimating costs of adverse events (AEs) and healthcare resource use (HRU) in patients with relapsed or refractory diffuse large B-cell lymphoma (r/r DLBCL) receiving tisagenlecleucel (tisa-cel) and axicabtagene ciloleucel (axi-cel): A summary of real-world evidence.

2021 ◽  
Vol 39 (15_suppl) ◽  
pp. e19551-e19551
Author(s):  
Hongbo Yang ◽  
Cynthia Zhengyun Qi ◽  
Anand Dalal ◽  
Vamsi Bollu ◽  
Jie Zhang ◽  
...  
Keyword(s):  
Real World ◽  
B Cell Lymphoma ◽  
The United States ◽  
Median Number ◽  
Cost Burden ◽  
Real World Evidence ◽  
Car T ◽  
Grade 3 ◽  
Cost Implications ◽  
The Cost

e19551 Background: The AE rates and HRU reported in multiple real-world evidence (RWE) studies of chimeric antigen receptor T-cell (CAR-T) therapies tisa-cel and axi-cel in r/r DLBCL have differed from those in their clinical trials. However, the cost implications from these findings are not well understood in existing literature. This study summarizes information from these RWE studies of tisa-cel and axi-cel and quantifies the associated costs. Methods: A literature review was conducted to identify RWE studies reporting AE rates and HRU of tisa-cel and axi-cel in the United States (US). AE rates and HRU were summarized and the associated costs were estimated using a micro-costing approach. Costs of AE management included hospitalization and pharmacy costs, such as intensive care unit (ICU) stays, inpatient admissions, and medications for the treatment of cytokine release syndrome (CRS) and neurotoxicity events (NE). HRU costs included hospitalization, ICU stays, and outpatient visit costs. Unit costs were from public health databases that are representative of US healthcare system and from literature. Costs were inflated to 2020 US dollars. A range was reported to present evidence if inputs are available from multiple studies. Results were summarized for tisa-cel and axi-cel separately. Results: Four publications were identified: Jaglowski 2019, Pasquini 2019, Riedell 2019, and Jacobson 2020. Across studies, grade 3+ CRS and NE occurred in 1%-4% and 0%-5% of tisa-cel-treated patients and 7%-16% and 20%-35% of axi-cel-treated patients, respectively. Tocilizumab usage was reported in 14%-20% of tisa-cel- and 62%-71% of axi-cel-treated patients. CAR-T infusion was inpatient for 36% of tisa-cel- and 92%-100% of axi-cel-treated patients. The median hospitalization days was 2 for tisa-cel and 15-16 for axi-cel. ICU transfer was observed for 7% and 28%-38% of tisa-cel- and axi-cel-treated patients, respectively, with median stays of 4 and 5 days, respectively. The median number of outpatient visits within 28 days after infusion was 6 for tisa-cel and 4 for axi-cel. The total estimated costs for managing AEs per patient were $843-$1,962 for tisa-cel and $5,979-$10,878 for axi-cel. The total estimated HRU costs per patient were $3,321 for tisa-cel and $32,394-33,166 for axi-cel. Conclusions: RWE studies suggest that patients with r/r DLBCL receiving tisa-cel had numerically lower AE rates, HRU, and cost burden than those receiving axi-cel in the US. The additional cost burden for axi-cel was primarily driven by the incremental ICU and hospitalization care due to a higher proportion of inpatient infusion among patients receiving axi-cel. Further research is warranted to compare the costs associated with the two CAR-Ts in r/r DLBCL.

scholarly journals Neural Representation and Learning of Hierarchical 2-additive Choquet Integrals

2020 ◽  
Author(s):  
Roman Bresson ◽  
Johanne Cohen ◽  
Eyke Hüllermeier ◽  
Christophe Labreuche ◽  
Michèle Sebag
Keyword(s):  
Real World ◽  
Choquet Integral ◽  
Marginal Utility ◽  
Decision Makers ◽  
Neural Representation ◽  
Automatic Identification ◽  
Neural Architecture ◽  
Real World Applications ◽  
Choquet Integrals ◽  
The Cost

Multi-Criteria Decision Making (MCDM) aims at modelling expert preferences and assisting decision makers in identifying options best accommodating expert criteria. An instance of MCDM model, the Choquet integral is widely used in real-world applications, due to its ability to capture interactions between criteria while retaining interpretability. Aimed at a better scalability and modularity, hierarchical Choquet integrals involve intermediate aggregations of the interacting criteria, at the cost of a more complex elicitation. The paper presents a machine learning-based approach for the automatic identification of hierarchical MCDM models, composed of 2-additive Choquet integral aggregators and of marginal utility functions on the raw features from data reflecting expert preferences. The proposed NEUR-HCI framework relies on a specific neural architecture, enforcing by design the Choquet model constraints and supporting its end-to-end training. The empirical validation of NEUR-HCI on real-world and artificial benchmarks demonstrates the merits of the approach compared to state-of-art baselines.

scholarly journals The cost-effectiveness of initiating ranibizumab therapy in eyes with neovascular AMD with good vision: an economic model using real-world outcomes

BMJ Open ◽  
2015 ◽  
Vol 5 (5) ◽  
pp. e006535-e006535 ◽  
Author(s):  
T. Butt ◽  
A. Lee ◽  
C. Lee ◽  
A. Tufail ◽  
W. Xing ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Real World ◽  
Economic Model ◽  
Neovascular Amd ◽  
The Cost ◽  
Good Vision

Abstract W P273: The Cost-Effectiveness of Telestroke Varies by Implementation Cost and Stroke Severity: Real World Data From a Pacific Northwest Network

Stroke ◽  
2015 ◽  
Vol 46 (suppl_1) ◽  
Author(s):  
Elizabeth Baraban ◽  
Richard Nelson ◽  
Alexandra Lesko ◽  
Jennifer Majersik ◽  
Archit Bhatt ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Pacific Northwest ◽  
Real World ◽  
Cost Effective ◽  
Stroke Severity ◽  
Real World Data ◽  
World Data ◽  
Life Years ◽  
Implementation Costs ◽  
The Cost

Objective: An obstacle for community hospitals in joining a telestroke network is often the cost of implementation. Yet, previous analyses examining the cost and cost-effectiveness have only used estimates from the literature. Using real-world data from a Pacific Northwest telestroke network, we examined the cost-effectiveness of telestroke for spokes by level of financial responsibility for these costs and how this changes with patient stroke severity. Methods: We constructed a decision analytic model and parameterized it using patient-level clinical and financial data from the Providence Telestroke Network (PTN) pre and post telestroke implementation. Data included patients presenting at 17 spokes within 4.5 hours of symptom onset. Probability inputs included observed IV-tPA treatment rates, transfer status and hospital costs and reimbursements. Effectiveness, measured as quality-adjusted life years (QALYs), and cost per patient were used to calculate incremental cost effectiveness ratios (ICERs). ICER’s of <$50,000-$120,000/QALY are considered cost-effective. Outcomes were generated overall and separately by admit NIHSS, defined as low (0-10), medium (11-20) and high (>20) and percentage of implementation costs paid by spokes (0%, 50%, 100%). Results: Data for 594 patients, 105 pre- and 489 post-implementation, were included. See Table 1. Conclusions: Our results support previous theoretic models showing good value, overall. However, costs and ICERs varied by stroke severity, with telestroke being most cost-effective for severe strokes. Telestroke was least cost effective if spokes paid for half or more of implementation costs.

External Validity Reconsidered

2011 ◽  
Author(s):  
Diana C. Mutz
Keyword(s):  
Political Science ◽  
Real World ◽  
External Validity ◽  
Observational Studies ◽  
Internal Validity ◽  
Conventional Wisdom ◽  
Observational Research ◽  
Convincing Case

This chapter talks about the significance of generalizability. Experimentalists often go to great lengths to argue that student or other convenience samples are not problematic in terms of external validity. Likewise, a convincing case for causality is often elusive with observational research, no matter how stridently one might argue to the contrary. The conventional wisdom is that experiments are widely valued for their internal validity, and experiments lack external validity. These assumptions are so widespread as to go without question in most disciplines, particularly those emphasizing external validity, such as political science and sociology. But observational studies, such as surveys, are still supposed to be better for purposes of maximizing external validity because this method allows studying people in real world settings.

Overview of Decision Support Systems Applied to Construction

2016 ◽  
pp. 77-94
Keyword(s):  
Decision Support ◽  
Decision Support Systems ◽  
Markov Decision Processes ◽  
Support Systems ◽  
Decision Processes ◽  
Rule Based ◽  
Markov Decision ◽  
Knowledge Intensive ◽  
The Cost ◽  
Rule Based Systems

The domain of construction is a very knowledge-intensive domain with so many factors involved. This implies undertaking any action requires an understanding of the different factors and how best to combine them to achieve a favourable and optimal outcome. Thus decision-making has been extensively used in the domain of construction. The aim of this chapter is to undertake a review of various decision support systems and to provide insights into their applications in the domain of construction. Specifically, the principle of cost index, sub-work chaining diagram method, linear regression and cost over-runs in time-overrun context (CCOTOV) model and Markov decision processes (MDP), ontology and rule-based systems have been reviewed. Based on the review the Markov decision processes (MDP), ontology and rule-based systems were chosen as the more suitable for the cost control case considered in this study.

Sign in / Sign up

Export Citation Format

Share Document