scholarly journals Synergy conformal prediction applied to large-scale bioactivity datasets and in federated learning

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Ulf Norinder ◽  
Ola Spjuth ◽  
Fredrik Svensson
Keyword(s):  
Decision Making ◽  
Drug Discovery ◽  
Large Scale ◽  
Predictive Performance ◽  
Training Data ◽  
Conformal Prediction ◽  
Valuable Addition ◽  
Conformal Predictors ◽  
Data Synergy ◽  
Training Sets

AbstractConfidence predictors can deliver predictions with the associated confidence required for decision making and can play an important role in drug discovery and toxicity predictions. In this work we investigate a recently introduced version of conformal prediction, synergy conformal prediction, focusing on the predictive performance when applied to bioactivity data. We compare the performance to other variants of conformal predictors for multiple partitioned datasets and demonstrate the utility of synergy conformal predictors for federated learning where data cannot be pooled in one location. Our results show that synergy conformal predictors based on training data randomly sampled with replacement can compete with other conformal setups, while using completely separate training sets often results in worse performance. However, in a federated setup where no method has access to all the data, synergy conformal prediction is shown to give promising results. Based on our study, we conclude that synergy conformal predictors are a valuable addition to the conformal prediction toolbox.

scholarly journals Reply to “Missed opportunities in large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery”

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Nicolas Bosc ◽  
Francis Atkinson ◽  
Eloy Félix ◽  
Anna Gaulton ◽  
Anne Hersey ◽  
...  
Keyword(s):  
Drug Discovery ◽  
Scientific Community ◽  
Large Scale ◽  
Prediction Methods ◽  
Missed Opportunities ◽  
Conformal Prediction ◽  
Letter To The Editor ◽  
Published Paper

Abstract In response to Krstajic’s letter to the editor concerning our published paper, we here take the opportunity to reply, to re-iterate that no errors in our work were identified, to provide further details, and to re-emphasise the outputs of our study. Moreover, we highlight that all of the data are freely available for the wider scientific community (including the aforementioned correspondent) to undertake follow-on studies and comparisons.

scholarly journals The Wisdom of Model Crowds

2021 ◽  
Author(s):  
Lisheng He ◽  
Pantelis P. Analytis ◽  
Sudeep Bhatia
Keyword(s):  
Decision Making ◽  
Large Scale ◽  
Choice Behavior ◽  
Predictive Performance ◽  
Choice Models ◽  
Risky Choice ◽  
Risky Decision Making ◽  
Risky Decision ◽  
Performance Bound ◽  
Individual Level

A wide body of empirical research has revealed the descriptive shortcomings of expected value and expected utility models of risky decision making. In response, numerous models have been advanced to predict and explain people’s choices between gambles. Although some of these models have had a great impact in the behavioral, social and management sciences, there is little consensus about which model offers the best account of choice behavior. In this paper, we conduct a large-scale comparison of 58 prominent models of risky choice, using 19 existing behavioral datasets involving more than 800 participants. This allows us to comprehensively evaluate models in terms of individual-level predictive performance across a range of different choice settings. We also identify the psychological mechanisms that lead to superior predictive performance and the properties of choice stimuli that favor certain types of models over others. Second, drawing on research on the wisdom of crowds, we argue that each of the existing models can be seen as an expert that provides unique forecasts in choice predictions. Consistent with this claim, we find that crowds of risky choice models perform better than individual models and thus provide a performance bound for assessing the historical accumulation of knowledge in our field. Our results suggest that each model captures unique aspects of the decision process, and that existing risky choice models offer complementary rather than competing accounts of behavior. We discuss the implications of our results on theories of risky decision making and the quantitative modeling of choice behavior.

The Wisdom of Model Crowds

2021 ◽  
Author(s):  
Lisheng He ◽  
Pantelis P. Analytis ◽  
Sudeep Bhatia
Keyword(s):  
Decision Making ◽  
Large Scale ◽  
Choice Behavior ◽  
Predictive Performance ◽  
Choice Models ◽  
Risky Choice ◽  
Risky Decision Making ◽  
Risky Decision ◽  
Performance Bound ◽  
Individual Level

A wide body of empirical research has revealed the descriptive shortcomings of expected value and expected utility models of risky decision making. In response, numerous models have been advanced to predict and explain people’s choices between gambles. Although some of these models have had a great impact in the behavioral, social, and management sciences, there is little consensus about which model offers the best account of choice behavior. In this paper, we conduct a large-scale comparison of 58 prominent models of risky choice, using 19 existing behavioral data sets involving more than 800 participants. This allows us to comprehensively evaluate models in terms of individual-level predictive performance across a range of different choice settings. We also identify the psychological mechanisms that lead to superior predictive performance and the properties of choice stimuli that favor certain types of models over others. Moreover, drawing on research on the wisdom of crowds, we argue that each of the existing models can be seen as an expert that provides unique forecasts in choice predictions. Consistent with this claim, we find that crowds of risky choice models perform better than individual models and thus provide a performance bound for assessing the historical accumulation of knowledge in our field. Our results suggest that each model captures unique aspects of the decision process and that existing risky choice models offer complementary rather than competing accounts of behavior. We discuss the implications of our results on theories of risky decision making and the quantitative modeling of choice behavior. This paper was accepted by Yuval Rottenstreich, behavioral economics and decision analysis.

scholarly journals Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Nicolas Bosc ◽  
Francis Atkinson ◽  
Eloy Felix ◽  
Anna Gaulton ◽  
Anne Hersey ◽  
...  
Keyword(s):  
Drug Discovery ◽  
Large Scale ◽  
Prediction Methods ◽  
Conformal Prediction

scholarly journals Predicting Targeted Cancer Therapeutics

2016 ◽  
Author(s):  
Aaron Wise ◽  
Murat Can Cobanoglu
Keyword(s):  
Drug Discovery ◽  
High Throughput ◽  
High Throughput Screening ◽  
De Novo ◽  
Cancer Therapeutics ◽  
Predictive Performance ◽  
Training Data ◽  
Brute Force ◽  
Functional Assays ◽  
Novel Approach

AbstractMotivation: Cancer is a complex and evolving disease, making it difficult to discover effective treatments. Traditional drug discovery relies on high-throughput screening on reductionist models in order to enable the testing of 105 or 106 compounds. These assays lack the complexity of the human disease. Functional assays overcome this limitation by testing drugs on human tumors, however they can only test few drugs, and remain restricted to diagnostic use. An algorithm that identifies hits with fewer experiments could enable the use of functional assays for de novo drug discovery.Results: We developed a novel approach that we termed ‘algorithmic ideation’ (AI) to select experiments, and demonstrated that this approach discovers hits 104 times more effectively than brute-force screening. The algorithm trains on known drug-target-disease associations assembled as a tensor, built from the (public) TCGA and STITCH databases and predicts novel associations. We evaluated our tensor completion approach using a temporal cutoff with data prior to 2012 used as training data, and data from 2012 to 2015 used as testing data. Our approach achieved 104-fold more efficient hit discovery than the traditional brute-force high-throughput screening. We further tested the method in a sparse, low data regime by removing up to 90% of the training data, and demonstrated the robustness of the approach. Finally we test predictive performance on drugs with no previously known interactions, and the algorithm demonstrates 103-fold improvement in this challenging problem. Thus algorithmic ideation can potentially enable targeted antineoplastic discovery on functional assays.Availability: Freely accessible at https://bitbucket.org/aiinc/drugx.Contact:[email protected], [email protected]

scholarly journals Missed opportunities in large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Damjan Krstajic
Keyword(s):  
Drug Discovery ◽  
Ligand Binding ◽  
Large Scale ◽  
Prediction Methods ◽  
Missed Opportunities ◽  
Conformal Prediction

Abstract Recently Bosc et al. (J Cheminform 11(1): 4, 2019), published an article describing a case study that directly compares conformal predictions with traditional QSAR methods for large-scale predictions of target-ligand binding. We consider this study to be very important. Unfortunately, we have found several issues in the authors’ approach as well as in the presentation of their findings.

Large-Scale Assessment of Binding Free Energy Calculations in Active Drug Discovery Projects

2020 ◽  
Author(s):  
Christina Schindler ◽  
Hannah Baumann ◽  
Andreas Blum ◽  
Dietrich Böse ◽  
Hans-Peter Buchstaller ◽  
...  
Keyword(s):  
Free Energy ◽  
Drug Discovery ◽  
Large Scale ◽  
Active Drug ◽  
Binding Free Energy ◽  
Free Energy Calculations ◽  
Energy Calculation ◽  
Large Scale Assessment ◽  
New Public ◽  
Binding Free Energy Calculations

Here we present an evaluation of the binding affinity prediction accuracy of the free energy calculation method FEP+ on internal active drug discovery projects and on a large new public benchmark set.<br>

Reaction-based Enumeration, Active Learning, and Free Energy Calculations to Rapidly Explore Synthetically Tractable Chemical Space and Optimize Potency of Cyclin Dependent Kinase 2 Inhibitors

2019 ◽  
Author(s):  
Kyle Konze ◽  
Pieter Bos ◽  
Markus Dahlgren ◽  
Karl Leswing ◽  
Ivan Tubert-Brohman ◽  
...  
Keyword(s):  
Free Energy ◽  
Drug Discovery ◽  
Active Learning ◽  
Large Scale ◽  
Chemical Space ◽  
Population Based ◽  
Free Energy Calculations ◽  
Computational Technique ◽  
Cyclin Dependent Kinase ◽  
Energy Calculations

We report a new computational technique, PathFinder, that uses retrosynthetic analysis followed by combinatorial synthesis to generate novel compounds in synthetically accessible chemical space. Coupling PathFinder with active learning and cloud-based free energy calculations allows for large-scale potency predictions of compounds on a timescale that impacts drug discovery. The process is further accelerated by using a combination of population-based statistics and active learning techniques. Using this approach, we rapidly optimized R-groups and core hops for inhibitors of cyclin-dependent kinase 2. We explored greater than 300 thousand ideas and identified 35 ligands with diverse commercially available R-groups and a predicted IC<sub>50</sub> < 100 nM, and four unique cores with a predicted IC<sub>50</sub> < 100 nM. The rapid turnaround time, and scale of chemical exploration, suggests that this is a useful approach to accelerate the discovery of novel chemical matter in drug discovery campaigns.

Reaction-based Enumeration, Active Learning, and Free Energy Calculations to Rapidly Explore Synthetically Tractable Chemical Space and Optimize Potency of Cyclin Dependent Kinase 2 Inhibitors

2019 ◽  
Author(s):  
Kyle Konze ◽  
Pieter Bos ◽  
Markus Dahlgren ◽  
Karl Leswing ◽  
Ivan Tubert-Brohman ◽  
...  
Keyword(s):  
Free Energy ◽  
Drug Discovery ◽  
Active Learning ◽  
Large Scale ◽  
Chemical Space ◽  
Population Based ◽  
Free Energy Calculations ◽  
Computational Technique ◽  
Cyclin Dependent Kinase ◽  
Energy Calculations

We report a new computational technique, PathFinder, that uses retrosynthetic analysis followed by combinatorial synthesis to generate novel compounds in synthetically accessible chemical space. Coupling PathFinder with active learning and cloud-based free energy calculations allows for large-scale potency predictions of compounds on a timescale that impacts drug discovery. The process is further accelerated by using a combination of population-based statistics and active learning techniques. Using this approach, we rapidly optimized R-groups and core hops for inhibitors of cyclin-dependent kinase 2. We explored greater than 300 thousand ideas and identified 35 ligands with diverse commercially available R-groups and a predicted IC<sub>50</sub> < 100 nM, and four unique cores with a predicted IC<sub>50</sub> < 100 nM. The rapid turnaround time, and scale of chemical exploration, suggests that this is a useful approach to accelerate the discovery of novel chemical matter in drug discovery campaigns.

Sign in / Sign up

Export Citation Format

Share Document