Uncertain Times Call for Quantitative Uncertainty Metrics: Controlling Error in Neural Network Predictions for Chemical Discovery

2019 ◽  
Author(s):  
Jon Paul Janet ◽  
Chenru Duan ◽  
Tzuhsiung Yang ◽  
Aditya Nandy ◽  
Heather Kulik
Keyword(s):  
Neural Network ◽  
High Throughput Screening ◽  
Error Control ◽  
Large Scale ◽  
Chemical Space ◽  
Low Cost ◽  
Space Exploration ◽  
Prediction Errors ◽  
Model Errors ◽  
Neural Network Models

<p>Machine learning (ML) models, such as artificial neural networks, have emerged as a complement to high-throughput screening, enabling characterization of new compounds in seconds instead of hours. The promise of ML models to enable large-scale, chemical space exploration can only be realized if it is straightforward to identify when molecules and materials are outside the model’s domain of applicability. Established uncertainty metrics for neural network models are either costly to obtain (e.g., ensemble models) or rely on feature engineering (e.g., feature space distances), and each has limitations in estimating prediction errors for chemical space exploration. We introduce the distance to available data in the latent space of a neural network ML model as a low-cost, quantitative uncertainty metric that works for both inorganic and organic chemistry. The calibrated performance of this approach exceeds widely used uncertainty metrics and is readily applied to models of increasing complexity at no additional cost. Tightening latent distance cutoffs systematically drives down predicted model errors below training errors, thus enabling predictive error control in chemical discovery or identification of useful data points for active learning.</p>

scholarly journals A Quantitative Uncertainty Metric Controls Error in Neural Network-Driven Chemical Discovery

2019 ◽  
Author(s):  
Jon Paul Janet ◽  
Chenru Duan ◽  
Tzuhsiung Yang ◽  
Aditya Nandy ◽  
Heather Kulik
Keyword(s):  
Neural Network ◽  
High Throughput Screening ◽  
Error Control ◽  
Large Scale ◽  
Chemical Space ◽  
Low Cost ◽  
Space Exploration ◽  
Prediction Errors ◽  
Model Errors ◽  
Neural Network Models

<p>Machine learning (ML) models, such as artificial neural networks, have emerged as a complement to high-throughput screening, enabling characterization of new compounds in seconds instead of hours. The promise of ML models to enable large-scale, chemical space exploration can only be realized if it is straightforward to identify when molecules and materials are outside the model’s domain of applicability. Established uncertainty metrics for neural network models are either costly to obtain (e.g., ensemble models) or rely on feature engineering (e.g., feature space distances), and each has limitations in estimating prediction errors for chemical space exploration. We introduce the distance to available data in the latent space of a neural network ML model as a low-cost, quantitative uncertainty metric that works for both inorganic and organic chemistry. The calibrated performance of this approach exceeds widely used uncertainty metrics and is readily applied to models of increasing complexity at no additional cost. Tightening latent distance cutoffs systematically drives down predicted model errors below training errors, thus enabling predictive error control in chemical discovery or identification of useful data points for active learning.</p>

scholarly journals A Quantitative Uncertainty Metric Controls Error in Neural Network-Driven Chemical Discovery

2019 ◽  
Author(s):  
Jon Paul Janet ◽  
Chenru Duan ◽  
Tzuhsiung Yang ◽  
Aditya Nandy ◽  
Heather Kulik
Keyword(s):  
Neural Network ◽  
High Throughput Screening ◽  
Error Control ◽  
Large Scale ◽  
Chemical Space ◽  
Low Cost ◽  
Space Exploration ◽  
Prediction Errors ◽  
Model Errors ◽  
Neural Network Models

<p>Machine learning (ML) models, such as artificial neural networks, have emerged as a complement to high-throughput screening, enabling characterization of new compounds in seconds instead of hours. The promise of ML models to enable large-scale, chemical space exploration can only be realized if it is straightforward to identify when molecules and materials are outside the model’s domain of applicability. Established uncertainty metrics for neural network models are either costly to obtain (e.g., ensemble models) or rely on feature engineering (e.g., feature space distances), and each has limitations in estimating prediction errors for chemical space exploration. We introduce the distance to available data in the latent space of a neural network ML model as a low-cost, quantitative uncertainty metric that works for both inorganic and organic chemistry. The calibrated performance of this approach exceeds widely used uncertainty metrics and is readily applied to models of increasing complexity at no additional cost. Tightening latent distance cutoffs systematically drives down predicted model errors below training errors, thus enabling predictive error control in chemical discovery or identification of useful data points for active learning.</p>

SHEDR: An End-to-End Deep Neural Event Detection and Recommendation Framework for Hyperlocal News Using Social Media

2021 ◽  
Author(s):  
Yuheng Hu ◽  
Yili Hong
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Deep Learning ◽  
Event Detection ◽  
Large Scale ◽  
Short Term Memory ◽  
State Of The Art ◽  
Neural Network Models ◽  
Neural Event ◽  
End To End

Residents often rely on newspapers and television to gather hyperlocal news for community awareness and engagement. More recently, social media have emerged as an increasingly important source of hyperlocal news. Thus far, the literature on using social media to create desirable societal benefits, such as civic awareness and engagement, is still in its infancy. One key challenge in this research stream is to timely and accurately distill information from noisy social media data streams to community members. In this work, we develop SHEDR (social media–based hyperlocal event detection and recommendation), an end-to-end neural event detection and recommendation framework with a particular use case for Twitter to facilitate residents’ information seeking of hyperlocal events. The key model innovation in SHEDR lies in the design of the hyperlocal event detector and the event recommender. First, we harness the power of two popular deep neural network models, the convolutional neural network (CNN) and long short-term memory (LSTM), in a novel joint CNN-LSTM model to characterize spatiotemporal dependencies for capturing unusualness in a region of interest, which is classified as a hyperlocal event. Next, we develop a neural pairwise ranking algorithm for recommending detected hyperlocal events to residents based on their interests. To alleviate the sparsity issue and improve personalization, our algorithm incorporates several types of contextual information covering topic, social, and geographical proximities. We perform comprehensive evaluations based on two large-scale data sets comprising geotagged tweets covering Seattle and Chicago. We demonstrate the effectiveness of our framework in comparison with several state-of-the-art approaches. We show that our hyperlocal event detection and recommendation models consistently and significantly outperform other approaches in terms of precision, recall, and F-1 scores. Summary of Contribution: In this paper, we focus on a novel and important, yet largely underexplored application of computing—how to improve civic engagement in local neighborhoods via local news sharing and consumption based on social media feeds. To address this question, we propose two new computational and data-driven methods: (1) a deep learning–based hyperlocal event detection algorithm that scans spatially and temporally to detect hyperlocal events from geotagged Twitter feeds; and (2) A personalized deep learning–based hyperlocal event recommender system that systematically integrates several contextual cues such as topical, geographical, and social proximity to recommend the detected hyperlocal events to potential users. We conduct a series of experiments to examine our proposed models. The outcomes demonstrate that our algorithms are significantly better than the state-of-the-art models and can provide users with more relevant information about the local neighborhoods that they live in, which in turn may boost their community engagement.

Detailed Simulation of Large Scale Neural Network Models

1997 ◽  
pp. 931-935 ◽  
Author(s):  
Anders Lansner ◽  
Örjan Ekeberg ◽  
Erik Fransén ◽  
Per Hammarlund ◽  
Tomas Wilhelmsson
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
Network Models ◽  
Neural Network Models ◽  
Detailed Simulation

scholarly journals ChemSpaX: Exploration of Chemical Space by Automated Functionalization of Molecular Scaffold

2021 ◽  
Author(s):  
Adarsh Kalikadien ◽  
Evgeny A. Pidko ◽  
Vivek Sinha
Keyword(s):  
Force Field ◽  
High Throughput ◽  
High Throughput Screening ◽  
Chemical Space ◽  
Space Exploration ◽  
Molecular Structures ◽  
Data Driven ◽  
Pincer Complexes ◽  
Cobalt Porphyrin ◽  
Input Structure

<div>Local chemical space exploration of an experimentally synthesized material can be done by making slight structural</div><div>variations of the synthesized material. This generation of many molecular structures with reasonable quality,</div><div>that resemble an existing (chemical) purposeful material, is needed for high-throughput screening purposes in</div><div>material design. Large databases of geometry and chemical properties of transition metal complexes are not</div><div>readily available, although these complexes are widely used in homogeneous catalysis. A Python-based workflow,</div><div>ChemSpaX, that is aimed at automating local chemical space exploration for any type of molecule, is introduced.</div><div>The overall computational workflow of ChemSpaX is explained in more detail. ChemSpaX uses 3D information,</div><div>to place functional groups on an input structure. For example, the input structure can be a catalyst for which one</div><div>wants to use high-throughput screening to investigate if the catalytic activity can be improved. The newly placed</div><div>substituents are optimized using a computationally cheap force-field optimization method. After placement of</div><div>new substituents, higher level optimizations using xTB or DFT instead of force-field optimization are also possible</div><div>in the current workflow. In representative applications of ChemSpaX, it is shown that the structures generated by</div><div>ChemSpaX have a reasonable quality for usage in high-throughput screening applications. Representative applications</div><div>of ChemSpaX are shown by investigating various adducts on functionalized Mn-based pincer complexes,</div><div>hydrogenation of Ru-based pincer complexes, functionalization of cobalt porphyrin complexes and functionalization</div><div>of a bipyridyl functionalized cobalt-porphyrin trapped in a M2L4 type cage complex. Descriptors such as</div><div>the Gibbs free energy of reaction and HOMO-LUMO gap, that can be used in data-driven design and discovery</div><div>of catalysts, were selected and studied in more detail for the selected use cases. The relatively fast GFN2-xTB</div><div>method was used to calculate these descriptors and a comparison was done against DFT calculated descriptors.</div><div>ChemSpaX is open-source and aims to bolster the efforts of the scientific community towards data-driven material</div><div>discovery.</div>

scholarly journals Computational High-Throughput Screening of Polymeric Photocatalysts: Exploring the Effect of Composition, Sequence Isomerism and Conformational Degrees of Freedom

2018 ◽  
Author(s):  
isabelle Heath-Apostolopoulos ◽  
Liam Wilbraham ◽  
Martijn Zwijnenburg
Keyword(s):  
High Throughput ◽  
High Throughput Screening ◽  
Degrees Of Freedom ◽  
Chemical Space ◽  
Low Cost ◽  
Computational Screening ◽  
Computational Workflow ◽  
Conformational Degrees Of Freedom

We discuss a low-cost computational workflow for the high-throughput screening of polymeric photocatalysts and demonstrate its utility by applying it to a number of challenging problems that would be difficult to tackle otherwise. Specifically we show how having access to a low-cost method allows one to screen a vast chemical space, as well as to probe the effects of conformational degrees of freedom and sequence isomerism. Finally, we discuss both the opportunities of computational screening in the search for polymer photocatalysts, as well as the biggest challenges.

scholarly journals Evaluation of Pre-Trained Convolutional Neural Network Models for Object Recognition

2018 ◽  
Vol 7 (3.15) ◽  
pp. 95 ◽  
Author(s):  
M Zabir ◽  
N Fazira ◽  
Zaidah Ibrahim ◽  
Nurbaity Sabri
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Large Scale ◽  
Visual Recognition ◽  
Error Function ◽  
Network Models ◽  
Neural Network Models ◽  
California Institute Of Technology ◽  
Institute Of Technology ◽  
Similar Accuracy

This paper aims to evaluate the accuracy performance of pre-trained Convolutional Neural Network (CNN) models, namely AlexNet and GoogLeNet accompanied by one custom CNN. AlexNet and GoogLeNet have been proven for their good capabilities as these network models had entered ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and produce relatively good results. The evaluation results in this research are based on the accuracy, loss and time taken of the training and validation processes. The dataset used is Caltech101 by California Institute of Technology (Caltech) that contains 101 object categories. The result reveals that custom CNN architecture produces 91.05% accuracy whereas AlexNet and GoogLeNet achieve similar accuracy which is 99.65%. GoogLeNet consistency arrives at an early training stage and provides minimum error function compared to the other two models. 

scholarly journals Deep Autoencoder Neural Networks for Short-Term Traffic Congestion Prediction of Transportation Networks

Sensors ◽  
2019 ◽  
Vol 19 (10) ◽  
pp. 2229 ◽  
Author(s):  
Sen Zhang ◽  
Yong Yao ◽  
Jie Hu ◽  
Yong Zhao ◽  
Shaobo Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Intelligent Transportation Systems ◽  
Traffic Congestion ◽  
Large Scale ◽  
Transportation Networks ◽  
Transportation Network ◽  
Transportation Systems ◽  
Neural Network Models ◽  
Computation Efficiency ◽  
Congestion Prediction

Traffic congestion prediction is critical for implementing intelligent transportation systems for improving the efficiency and capacity of transportation networks. However, despite its importance, traffic congestion prediction is severely less investigated compared to traffic flow prediction, which is partially due to the severe lack of large-scale high-quality traffic congestion data and advanced algorithms. This paper proposes an accessible and general workflow to acquire large-scale traffic congestion data and to create traffic congestion datasets based on image analysis. With this workflow we create a dataset named Seattle Area Traffic Congestion Status (SATCS) based on traffic congestion map snapshots from a publicly available online traffic service provider Washington State Department of Transportation. We then propose a deep autoencoder-based neural network model with symmetrical layers for the encoder and the decoder to learn temporal correlations of a transportation network and predicting traffic congestion. Our experimental results on the SATCS dataset show that the proposed DCPN model can efficiently and effectively learn temporal relationships of congestion levels of the transportation network for traffic congestion forecasting. Our method outperforms two other state-of-the-art neural network models in prediction performance, generalization capability, and computation efficiency.

Sign in / Sign up

Export Citation Format

Share Document