Can Machine Learning from Real-World Data Support Drug Treatment Decisions? A Prediction Modeling Case for Direct Oral Anticoagulants

2021 ◽  
pp. 0272989X2110646
Author(s):  
Andreas D. Meid ◽  
Lucas Wirbka ◽  
Andreas Groll ◽  
Walter E. Haefeli ◽  
Keyword(s):  
Decision Making ◽  
Major Bleeding ◽  
Treatment Effects ◽  
Claims Data ◽  
Absolute Risk ◽  
Routine Data ◽  
Composite Endpoint ◽  
Absolute Risk Reduction ◽  
The Other ◽  
Individual Treatment

Background: Decision making for the “best” treatment is particularly challenging in situations in which individual patient response to drugs can largely differ from average treatment effects. By estimating individual treatment effects (ITEs), we aimed to demonstrate how strokes, major bleeding events, and a composite of both could be reduced by model-assisted recommendations for a particular direct oral anticoagulant (DOAC). Methods: In German claims data for the calendar years 2014–2018, we selected 29 901 new users of the DOACs rivaroxaban and apixaban. Random forests considered binary events within 1 y to estimate ITEs under each DOAC according to the X-learner algorithm with 29 potential effect modifiers; treatment recommendations were based on these estimated ITEs. Model performance was evaluated by the c-for-benefit statistics, absolute risk reduction (ARR), and absolute risk difference (ARD) by trial emulation. Results: A significant proportion of patients would be recommended a different treatment option than they actually received. The stroke model significantly discriminated patients for higher benefit and thus indicated improved decisions by reduced outcomes (c-for-benefit: 0.56; 95% confidence interval [0.52; 0.60]). In the group with apixaban recommendation, the model also improved the composite endpoint (ARR: 1.69 % [0.39; 2.97]). In trial emulations, model-assisted recommendations significantly reduced the composite event rate (ARD: −0.78 % [−1.40; −0.03]). Conclusions: If prescribers are undecided about the potential benefits of different treatment options, ITEs can support decision making, especially if evidence is inconclusive, risk-benefit profiles of therapeutic alternatives differ significantly, and the patients’ complexity deviates from “typical” study populations. In the exemplary case for DOACs and potentially in other situations, the significant impact could also become practically relevant if recommendations were available in an automated way as part of decision making. Highlights It was possible to calculate individual treatment effects (ITEs) from routine claims data for rivaroxaban and apixaban, and the characteristics between the groups with recommendation for one or the other option differed significantly. ITEs resulted in recommendations that were significantly superior to usual (observed) treatment allocations in terms of absolute risk reduction, both separately for stroke and in the composite endpoint of stroke and major bleeding. When similar patients from routine data were selected (precision cohorts) for patients with a strong recommendation for one option or the other, those similar patients under the respective recommendation showed a significantly better prognosis compared with the alternative option. Many steps may still be needed on the way to clinical practice, but the principle of decision support developed from routine data may point the way toward future decision-making processes.

scholarly journals Preterm neonates benefit from low prophylactic platelet transfusion threshold despite varying risk of bleeding or death

Blood ◽  
2019 ◽  
Vol 134 (26) ◽  
pp. 2354-2360
Author(s):  
Susanna F. Fustolo-Gunnink ◽  
Karin Fijnvandraat ◽  
David van Klaveren ◽  
Simon J. Stanworth ◽  
Anna Curley ◽  
...  
Keyword(s):  
Risk Reduction ◽  
Major Bleeding ◽  
Platelet Transfusion ◽  
Risk Group ◽  
Absolute Risk ◽  
Absolute Risk Reduction ◽  
Baseline Risk ◽  
Multivariate Logistic Regression Model ◽  
Preterm Neonates ◽  
Transfusion Threshold

Abstract The Platelets for Neonatal Thrombocytopenia (PlaNeT-2) trial reported an unexpected overall benefit of a prophylactic platelet transfusion threshold of 25 × 109/L compared with 50 × 109/L for major bleeding and/or mortality in preterm neonates (7% absolute-risk reduction). However, some neonates in the trial may have experienced little benefit or even harm from the 25 × 109/L threshold. We wanted to assess this heterogeneity of treatment effect in the PlaNet-2 trial, to investigate whether all preterm neonates benefit from the low threshold. We developed a multivariate logistic regression model in the PlaNet-2 data to predict baseline risk of major bleeding and/or mortality for all 653 neonates. We then ranked the neonates based on their predicted baseline risk and categorized them into 4 risk quartiles. Within these quartiles, we assessed absolute-risk difference between the 50 × 109/L- and 25 × 109/L-threshold groups. A total of 146 neonates died or developed major bleeding. The internally validated C-statistic of the model was 0.63 (95% confidence interval, 0.58-0.68). The 25 × 109/L threshold was associated with absolute-risk reduction in all risk groups, varying from 4.9% in the lowest risk group to 12.3% in the highest risk group. These results suggest that a 25 × 109/L prophylactic platelet count threshold can be adopted in all preterm neonates, irrespective of predicted baseline outcome risk. Future studies are needed to improve the predictive accuracy of the baseline risk model. This trial was registered at www.isrctn.com as #ISRCTN87736839.

scholarly journals A Systematic Review and Meta-Analysis of Randomized Trials Examining Extended Thromboprophylaxis with Direct Oral Anticoagulants in Hospitalized Medical Patients

Blood ◽  
2016 ◽  
Vol 128 (22) ◽  
pp. 1436-1436 ◽  
Author(s):  
Derrick Tao ◽  
Jeffery Bien ◽  
Joseph Shatzel
Keyword(s):  
Major Bleeding ◽  
Randomized Trials ◽  
Absolute Risk ◽  
Meta Analysis ◽  
Bleeding Event ◽  
Absolute Risk Reduction ◽  
Pooled Analysis ◽  
Risk Difference ◽  
Medical Patients ◽  
Extended Course

Abstract INTRODUCTION: Thromboprophylaxisduring hospitalization is an accepted intervention to prevent hospital related venous thromboembolism (VTE), however VTE still occurs post discharge. Recently, several randomized trials have compared extended prophylaxis with a direct oral anticoagulant (DOAC) for a prolonged period after hospital discharge to standard short-course prophylaxis with low molecular weight heparin (LMWH) in acutely-ill medical patients. While DOACs offer a convenient means to prevent post-hospitalization VTE, it remains unclear if this intervention has a positive risk-benefit ratio in the overall population of acutely hospitalized medical patients. To better answer this, we performed the following systematic review and meta-analysis. METHODS: We performed a systematic literature search in Medline using PUBMED in an attempt to identify all relevant randomized clinical trials comparing extended prophylaxis with any DOAC to standard course prophylaxis in hospitalized medical (non-surgical) patients. We extracted data on study design and characteristics, total VTE, symptomatic VTE, total bleeding, and major bleeding. Pooled relative risk (RR) was calculated with a corresponding 95% confidence interval (CI) using a Mantel-Haneszelrandom-effects model. Absolute risk differences and the number needed to treat (NNT) or number needed to harm (NNH) were generated along with Forest plots. RESULTS: Our search identified 338 individual publications, of which 313 were excluded on initial screening by title and abstract. Full text were obtained of the remaining 25 studies and independently reviewed by two physicians, of which 22 were excluded for the following reasons: Comments on other articles (12), Study Protocol (1), Review article (8), Data unavailable (1). A total of three trials, enrolling 22,142 patients, were included. Apixaban, rivaroxaban, andbetrixabanwere each evaluated by one study. Among the studies, 11,064 patients were randomized to extended DOAC prophylaxis, while 11,078 were randomized to standard course LMWH prophylaxis. In pooled analysis, VTE (including screened asymptomatic VTE) occurred in 4.30% of the patients receiving extended DOAC prophylaxis and 5.61% of patients treated with standard course LMWH (RR 0.76 [95% CI, 0.67-0.87], P = <0.0001, I2 = 0%). The NNT was 76. Symptomatic VTE occurred in 1.11% of patients treated with extended course DOACs and 1.68% patients treated with standard course LMWH. DOAC extended prophylaxis was associated with a significant reduction in symptomatic VTE (RR 0.66 [95% CI, 0.51-0.86], P = 0.002, I2 = 0%). The pooled absolute risk reduction was 0.57%, with a NNT of 176. A bleeding event occurred in 4.82% of patients receiving extended course DOACs compared to 3.16% of patients receiving standard course LMWH. The pooled analysis demonstrates a statistically relevant increase in bleeding (RR 1.74 [95% CI, 1.05-2.90], P < 0.001, I2 = 92%), with an absolute risk difference of 1.67% and a NNH of 60. A major bleeding event occurred in 0.59% of patients treated with extended course DOACs and 0.35% of patients treated with standard course LMWH. DOAC extended prophylaxis was associated with a significantly increased rate of major bleeding (RR 1.71 [95% CI, 1.07-2.75], P = 0.03, I2 = 23%). The pooled absolute risk difference is 0.24% with a NNH of 417. CONCLUSIONS: In this pooled analysis, extendedthromboprophylaxiswith DOACs was associated with a significant decrease in total and symptomatic VTEin medical patientsas compared to standard course LMWH. However, the low rate of symptomatic VTE, low absolute risk reduction, and significantly increased rate of bleeding with extended prophylaxis calls into questions if this intervention should be used. Table Table. Disclosures No relevant conflicts of interest to declare.

scholarly journals GPs’ understanding of the benefits and harms of treatments for long-term conditions: an online survey

BJGP Open ◽  
2020 ◽  
Vol 4 (1) ◽  
pp. bjgpopen20X101016 ◽  
Author(s):  
Julian Stephen Treadwell ◽  
Geoff Wong ◽  
Coral Milburn-Curtis ◽  
Benjamin Feakins ◽  
Trisha Greenhalgh
Keyword(s):  
Decision Making ◽  
Online Survey ◽  
Treatment Effects ◽  
Clinical Decision Making ◽  
Absolute Risk ◽  
Clinical Decision ◽  
Long Term Conditions ◽  
The Absolute ◽  
The Uk

BackgroundGPs prescribe multiple long-term treatments to their patients. For shared clinical decision-making, understanding of the absolute benefits and harms of individual treatments is needed. International evidence shows that doctors’ knowledge of treatment effects is poor but, to the authors knowledge, this has not been researched among GPs in the UK.AimTo measure the level and range of the quantitative understanding of the benefits and harms of treatments for common long-term conditions (LTCs) among GPs.Design & settingAn online cross-sectional survey was distributed to GPs in the UK.MethodParticipants were asked to estimate the percentage absolute risk reduction or increase conferred by 13 interventions across 10 LTCs on 17 important outcomes. Responses were collated and presented in a novel graphic format to allow detailed visualisation of the findings. Descriptive statistical analysis was performed.ResultsA total of 443 responders were included in the analysis. Most demonstrated poor (and in some cases very poor) knowledge of the absolute benefits and harms of treatments. Overall, an average of 10.9% of responses were correct allowing for ±1% margin in absolute risk estimates and 23.3% allowing a ±3% margin. Eighty-seven point seven per cent of responses overestimated and 8.9% of responses underestimated treatment effects. There was no tendency to differentially overestimate benefits and underestimate harms. Sixty-four point eight per cent of GPs self-reported ‘low’ to ‘very low’ confidence in their knowledge.ConclusionGPs’ knowledge of the absolute benefits and harms of treatments is poor, with inaccuracies of a magnitude likely to meaningfully affect clinical decision-making and impede conversations with patients regarding treatment choices.

Abstract 16612: Efficacy and Safety of Edoxaban for the Management of Elderly Patients With Atrial Fibrillation: Engage AF-TIMI 48

Circulation ◽  
2014 ◽  
Vol 130 (suppl_2) ◽  
Author(s):  
Eri Toda Kato ◽  
Robert P Giugliano ◽  
Christian T Ruff ◽  
Sabina A Murphy ◽  
Francesco Nordio ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Elderly Patients ◽  
Major Bleeding ◽  
Absolute Risk ◽  
Anticoagulation Therapy ◽  
Factor Xa ◽  
Absolute Risk Reduction ◽  
The Elderly ◽  
Efficacy And Safety ◽  
Bleeding Events

Background: Elderly patients with atrial fibrillation (AF) treated with anticoagulants are at higher risk of both ischemic and bleeding events compared to younger patients. Intracranial bleeding (ICH) remains one of the most concerning complications of anticoagulation therapy, and risk is strongly related to age. The ENGAGE AF-TIMI 48 trial showed that both the high (HD) and low dose (LD) regimens of the once daily oral factor Xa inhibitor edoxaban were non-inferior to well-managed warfarin (TTR 68.4%) in preventing stroke and systemic embolic events (SEE) while reducing major bleeding. Methods: 21,105 patients were enrolled in ENGAGE-TIMI 48 trial and stratified into pre-specified age categories: <65 (n=5,497), 65 to 74 (n=7,134), ≥75 (n=8,474) years. The primary endpoints were stroke/SEE and major bleeding as defined by the ISTH and outcomes over 2.8 years of median follow-up compared by age and treatment. Results: Regardless of treatment, the risk of major bleeding and stroke/SEE increased with age (p<0.001), but more markedly so for major bleeding. Both edoxaban regimens were similar to warfarin in effect on stroke/SEE across the age categories (HD vs. warfarin: <65yr HR 0.94, 65-74yr HR 0.89, ≥75yr HR 0.83; LD vs. warfarin: <65yr HR 1.42, 65-74yr HR 1.0, ≥75yr HR 1.12; P-int both NS). When comparing edoxaban with warfarin, a major impact was seen on absolute risk reduction for major bleeding, including ICH (figure). The edoxaban treatment benefits were consistent across ages (P-int NS for both). Conclusion: The efficacy and safety of edoxaban compared to well-managed warfarin are consistent regardless of age in patients with AF. Due to the higher risk of bleeding with increasing age, the absolute benefits of edoxaban are greater in the elderly.

A Phonomotor Approach to Apraxia of Speech Treatment

2020 ◽  
Vol 29 (4) ◽  
pp. 2109-2130
Author(s):  
Lauren Bislick
Keyword(s):  
Phase I ◽  
Treatment Effects ◽  
Single Case ◽  
Motor Planning ◽  
The Other ◽  
Multimodal Approach ◽  
Apraxia Of Speech ◽  
Duration Of Treatment ◽  
Treatment Gains ◽  
Future Work

Purpose This study continued Phase I investigation of a modified Phonomotor Treatment (PMT) Program on motor planning in two individuals with apraxia of speech (AOS) and aphasia and, with support from prior work, refined Phase I methodology for treatment intensity and duration, a measure of communicative participation, and the use of effect size benchmarks specific to AOS. Method A single-case experimental design with multiple baselines across behaviors and participants was used to examine acquisition, generalization, and maintenance of treatment effects 8–10 weeks posttreatment. Treatment was distributed 3 days a week, and duration of treatment was specific to each participant (criterion based). Experimental stimuli consisted of target sounds or clusters embedded nonwords and real words, specific to each participants' deficit. Results Findings show improved repetition accuracy for targets in trained nonwords, generalization to targets in untrained nonwords and real words, and maintenance of treatment effects at 10 weeks posttreatment for one participant and more variable outcomes for the other participant. Conclusions Results indicate that a modified version of PMT can promote generalization and maintenance of treatment gains for trained speech targets via a multimodal approach emphasizing repeated exposure and practice. While these results are promising, the frequent co-occurrence of AOS and aphasia warrants a treatment that addresses both motor planning and linguistic deficits. Thus, the application of traditional PMT with participant-specific modifications for AOS embedded into the treatment program may be a more effective approach. Future work will continue to examine and maximize improvements in motor planning, while also treating anomia in aphasia.

A Bird in the Hand Isn’t Good for Long

2018 ◽  
Vol 65 (1) ◽  
pp. 23-31 ◽  
Author(s):  
Stefan Scherbaum ◽  
Simon Frisch ◽  
Maja Dshemuchadse
Keyword(s):  
Decision Making ◽  
Cognitive Control ◽  
The Other ◽  
Decision Task ◽  
Short Term ◽  
Additional Time ◽  
Control Processes ◽  
Folk Wisdom ◽  
Study Participants ◽  
Insight Into

Abstract. Folk wisdom tells us that additional time to make a decision helps us to refrain from the first impulse to take the bird in the hand. However, the question why the time to decide plays an important role is still unanswered. Here we distinguish two explanations, one based on a bias in value accumulation that has to be overcome with time, the other based on cognitive control processes that need time to set in. In an intertemporal decision task, we use mouse tracking to study participants’ responses to options’ values and delays which were presented sequentially. We find that the information about options’ delays does indeed lead to an immediate bias that is controlled afterwards, matching the prediction of control processes needed to counter initial impulses. Hence, by using a dynamic measure, we provide insight into the processes underlying short-term oriented choices in intertemporal decision making.

Mental Contrasting of Future and Reality

2010 ◽  
Vol 9 (3) ◽  
pp. 138-144 ◽  
Author(s):  
Gabriele Oettingen ◽  
Doris Mayer ◽  
Babette Brinkmann
Keyword(s):  
Decision Making ◽  
Everyday Life ◽  
Brief Intervention ◽  
Middle Level ◽  
The Other ◽  
Metacognitive Strategy ◽  
Mental Contrasting ◽  
Low Expectations

Mental contrasting of a desired future with present reality leads to expectancy-dependent goal commitments, whereas focusing on the desired future only makes people commit to goals regardless of their high or low expectations for success. In the present brief intervention we randomly assigned middle-level managers (N = 52) to two conditions. Participants in one condition were taught to use mental contrasting regarding their everyday concerns, while participants in the other condition were taught to indulge. Two weeks later, participants in the mental-contrasting condition reported to have fared better in managing their time and decision making during everyday life than those in the indulging condition. By helping people to set expectancy-dependent goals, teaching the metacognitive strategy of mental contrasting can be a cost- and time-effective tool to help people manage the demands of their everyday life.

Use of pre-operative calcium and vitamin D supplementation to prevent post-operative hypocalcaemia in patients undergoing thyroidectomy: a systematic review

2021 ◽  
pp. 1-6
Author(s):  
A S Khatiwada ◽  
A S Harris
Keyword(s):  
Systematic Review ◽  
Vitamin D ◽  
Clinical Trials ◽  
Total Thyroidectomy ◽  
Risk Reduction ◽  
Operative Treatment ◽  
Absolute Risk ◽  
Absolute Risk Reduction ◽  
Vitamin D Supplementation ◽  
Absolute Reduction

Abstract Objective This systematic review aimed to establish the evidence behind the use of pre-operative calcium, vitamin D or both calcium and vitamin D to prevent post-operative hypocalcaemia in patients undergoing thyroidectomy. Method This review included prospective clinical trials on adult human patients that were published in English and which studied the effects of pre-operative supplementation with calcium, vitamin D or both calcium and vitamin D on the rate of post-operative hypocalcaemia following total thyroidectomy. Results Seven out of the nine trials included reported statistically significantly reduced rates of post-operative laboratory hypocalcaemia (absolute risk reduction, 13–59 per cent) and symptomatic hypocalcaemia (absolute reduction, 11–40 per cent) following pre-operative supplementation. Conclusion Pre-operative treatment with calcium, vitamin D or both calcium and vitamin D reduces the risk of post-operative hypocalcaemia and should be considered in patients undergoing total thyroidectomy.

scholarly journals Development of a prognostic tool: based on risk factors for tooth loss after active periodontal therapy

2021 ◽  
Author(s):  
Sonja Rahim-Wöstefeld ◽  
Dorothea Kronsteiner ◽  
Shirin ElSayed ◽  
Nihad ElSayed ◽  
Peter Eickholz ◽  
...  
Keyword(s):  
Risk Factors ◽  
Decision Making ◽  
Tooth Loss ◽  
Treatment Plan ◽  
Periodontal Therapy ◽  
Outcome Variable ◽  
Individual Treatment ◽  
Risk Indicator ◽  
Related Factors ◽  
Prognostic Tool

Abstract Objectives The aim of this study was to develop a prognostic tool to estimate long-term tooth retention in periodontitis patients at the beginning of active periodontal therapy (APT). Material and methods Tooth-related factors (type, location, bone loss (BL), infrabony defects, furcation involvement (FI), abutment status), and patient-related factors (age, gender, smoking, diabetes, plaque control record) were investigated in patients who had completed APT 10 years before. Descriptive analysis was performed, and a generalized linear-mixed model-tree was used to identify predictors for the main outcome variable tooth loss. To evaluate goodness-of-fit, the area under the curve (AUC) was calculated using cross-validation. A bootstrap approach was used to robustly identify risk factors while avoiding overfitting. Results Only a small percentage of teeth was lost during 10 years of supportive periodontal therapy (SPT; 0.15/year/patient). The risk factors abutment function, diabetes, and the risk indicator BL, FI, and age (≤ 61 vs. > 61) were identified to predict tooth loss. The prediction model reached an AUC of 0.77. Conclusion This quantitative prognostic model supports data-driven decision-making while establishing a treatment plan in periodontitis patients. In light of this, the presented prognostic tool may be of supporting value. Clinical relevance In daily clinical practice, a quantitative prognostic tool may support dentists with data-based decision-making. However, it should be stressed that treatment planning is strongly associated with the patient’s wishes and adherence. The tool described here may support establishment of an individual treatment plan for periodontally compromised patients.

scholarly journals OP0240 EPIDEMIOLOGY, PREDICTORS OF MORTALITY AND ROLE OF PROPHYLAXIS FOR PNEUMOCYSTITIS JIROVECI PNEUMONIA AMONG RHEUMATIC PATIENTS: A TERRITORY-WIDE STUDY

2021 ◽  
Vol 80 (Suppl 1) ◽  
pp. 147-147
Author(s):  
C. W. S. Chan ◽  
H. Y. Chung ◽  
W. Y. Yeung ◽  
C. S. Lau ◽  
P. H. LI
Keyword(s):  
Lupus Erythematosus ◽  
Absolute Risk ◽  
Absolute Risk Reduction ◽  
Number Needed To Treat ◽  
Healthcare Database ◽  
Pneumocystis Jiroveci ◽  
Predictors Of Mortality ◽  
Prednisolone Dose ◽  
Rheumatic Patients

Background:Pneumocystis jiroveci pneumonia (PJP) is an opportunistic infection affecting immunocompromised individuals. Due to its high mortality, PJP prophylaxis is commonly recommended for many immunocompromising conditions. However, evidence regarding the burden and role of prophylaxis in PJP among rheumatic patients remains limited. There is lack of consensus for when and for whom to initiate prophylaxis. Delineating the epidemiology, predictors of mortality and efficacy of prophylaxis in PJP among rheumatic patients is urgently needed.Objectives:To delineate the epidemiology of PJP, identify predictors of mortality and evaluate the usefulness of prophylaxis in rheumatology patients.Methods:We performed a big-data cohort study based on the territory-wide healthcare database of the Hong Kong Hospital Authority. All patients with a diagnosis of anti-neutrophil cytoplasmic antibody-associated vasculitis (AAV), immune-mediated myositis (IMM), rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), systemic sclerosis (SSc), or spondyloarthritis (SpA) between 2015-2019 were included. PJP were identified based on physician diagnosis and/or positive microbiological results from deep respiratory tract specimens. Prophylaxis was defined as prescription of a prophylactic dose of co-trimoxazole for at least 2 weeks and/or inhaled pentamidine. Prevalence of PJP, prophylaxis and mortality among rheumatic patients were calculated. Demographics, blood parameters and immunosuppressants use was also collected for multivariate analysis. Number needed to treat (NNT) analysis was performed based on absolute risk reduction of PJP in patients with and without prior PJP prophylaxis.Results:A total of 21,587 unique rheumatic patients were analysed (54% RA, 25% SLE, 13% SpA, 5% IMM, 2% AAV and 1% SSc). Between 2015-2019, 1141 (5.3%) patients were prescribed PJP prophylaxis and 48 (0.2%) developed PJP. None of those patients who developed PJP had received prophylaxis prior to infection. The risk of PJP was highest among SSc (1.8%), AAV (1.4%) and IMM (0.7%) patients, with NNT of SSc 36, AAV 48 and IMM 114. Within these disease entities, the majority of PJP occurred at prednisolone dose of 15mg/day (P15) or above (100% in SSc and IIM, 66.7% in AAV). Overall, PJP was associated with a mortality-rate of 39.6%. Glucocorticoid dose (daily prednisolone dose equivalent 29.1±23.5mg vs 11.4±7.2mg, P<0.01) and lymphopenia (0.44x109/L vs 0.90x109/L, P= 0.04) at PJP diagnosis were associated with PJP mortality in rheumatic patients.Conclusion:PJP is an uncommon but important infection in rheumatic patients associated with significant mortality. PJP prophylaxis is effective and should be considered in patients with SSc, AAV and IMM, especially in those receiving a steroid dose above P15.Disclosure of Interests:None declared

Sign in / Sign up

Export Citation Format

Share Document