scholarly journals Neural networks-based variationally enhanced sampling

2019 ◽  
Vol 116 (36) ◽  
pp. 17641-17647 ◽  
Author(s):  
Luigi Bonati ◽  
Yue-Yu Zhang ◽  
Michele Parrinello
Keyword(s):  
Neural Networks ◽  
Free Energy ◽  
Atomistic Simulation ◽  
Sampling Method ◽  
Machine Learning Techniques ◽  
Enhanced Sampling ◽  
Collective Variables ◽  
Learning Techniques ◽  
Minimization Technique ◽  
Energy Surfaces

Sampling complex free-energy surfaces is one of the main challenges of modern atomistic simulation methods. The presence of kinetic bottlenecks in such surfaces often renders a direct approach useless. A popular strategy is to identify a small number of key collective variables and to introduce a bias potential that is able to favor their fluctuations in order to accelerate sampling. Here, we propose to use machine-learning techniques in conjunction with the recent variationally enhanced sampling method [O. Valsson, M. Parrinello, Phys. Rev. Lett. 113, 090601 (2014)] in order to determine such potential. This is achieved by expressing the bias as a neural network. The parameters are determined in a variational learning scheme aimed at minimizing an appropriate functional. This required the development of a more efficient minimization technique. The expressivity of neural networks allows representing rapidly varying free-energy surfaces, removes boundary effects artifacts, and allows several collective variables to be handled.

scholarly journals Free Energy Surfaces and Barriers for Vacancy Diffusion on Al(100), Al(110), Al(111) Reconstructed Surfaces

Nanomaterials ◽  
2021 ◽  
Vol 12 (1) ◽  
pp. 76
Author(s):  
Junais Habeeb Mokkath ◽  
Mufasila Mumthaz Muhammed ◽  
Ali J. Chamkha
Keyword(s):  
Free Energy ◽  
Energy Barrier ◽  
Computational Study ◽  
Metastable States ◽  
Vacancy Diffusion ◽  
Enhanced Sampling ◽  
Free Energy Surface ◽  
Collective Variables ◽  
Surface Vacancy ◽  
Energy Surfaces

Metadynamics is a popular enhanced sampling method based on the recurrent application of a history-dependent adaptive bias potential that is a function of a selected number of appropriately chosen collective variables. In this work, using metadynamics simulations, we performed a computational study for the diffusion of vacancies on three different Al surfaces [reconstructed Al(100), Al(110), and Al(111) surfaces]. We explored the free energy landscape of diffusion and estimated the barriers associated with this process on each surface. It is found that the surfaces are unique regarding vacancy diffusion. More specically, the reconstructed Al(110) surface presents four metastable states on the free energy surface having sizable and connected passage-ways with an energy barrier of height 0.55 eV. On the other hand, the reconstructed Al(100)/Al(111) surfaces exhibit two/three metastable states, respectively, with an energy barrier of height 0.33 eV. The findings in this study can help to understand surface vacancy diffusion in technologically relevant Al surfaces.

A Method of Environmental Sound Classification Based on Residual Networks and Data Augmentation

2021 ◽  
pp. 2150018
Author(s):  
Jinfang Zeng ◽  
Youming Li ◽  
Yu Zhang ◽  
Da Chen
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
Machine Learning Techniques ◽  
Deep Convolutional Neural Networks ◽  
Mel Frequency Cepstral Coefficients ◽  
Environmental Sound ◽  
Sound Classification ◽  
Learning Techniques ◽  
Proposed Model ◽  
Audio Data

Environmental sound classification (ESC) is a challenging problem due to the complexity of sounds. To date, a variety of signal processing and machine learning techniques have been applied to ESC task, including matrix factorization, dictionary learning, wavelet filterbanks and deep neural networks. It is observed that features extracted from deeper networks tend to achieve higher performance than those extracted from shallow networks. However, in ESC task, only the deep convolutional neural networks (CNNs) which contain several layers are used and the residual networks are ignored, which lead to degradation in the performance. Meanwhile, a possible explanation for the limited exploration of CNNs and the difficulty to improve on simpler models is the relative scarcity of labeled data for ESC. In this paper, a residual network called EnvResNet for the ESC task is proposed. In addition, we propose to use audio data augmentation to overcome the problem of data scarcity. The experiments will be performed on the ESC-50 database. Combined with data augmentation, the proposed model outperforms baseline implementations relying on mel-frequency cepstral coefficients and achieves results comparable to other state-of-the-art approaches in terms of classification accuracy.

scholarly journals Exploiting node metadata to predict interactions in large networks using graph embedding and neural networks

2021 ◽  
Author(s):  
Rogini Runghen ◽  
Daniel B Stouffer ◽  
Giulio Valentino Dalla Riva
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Link Prediction ◽  
Graph Embedding ◽  
Feature Space ◽  
Machine Learning Techniques ◽  
Large Networks ◽  
Data Set ◽  
Learning Techniques ◽  
Low Dimensional

Collecting network interaction data is difficult. Non-exhaustive sampling and complex hidden processes often result in an incomplete data set. Thus, identifying potentially present but unobserved interactions is crucial both in understanding the structure of large scale data, and in predicting how previously unseen elements will interact. Recent studies in network analysis have shown that accounting for metadata (such as node attributes) can improve both our understanding of how nodes interact with one another, and the accuracy of link prediction. However, the dimension of the object we need to learn to predict interactions in a network grows quickly with the number of nodes. Therefore, it becomes computationally and conceptually challenging for large networks. Here, we present a new predictive procedure combining a graph embedding method with machine learning techniques to predict interactions on the base of nodes' metadata. Graph embedding methods project the nodes of a network onto a---low dimensional---latent feature space. The position of the nodes in the latent feature space can then be used to predict interactions between nodes. Learning a mapping of the nodes' metadata to their position in a latent feature space corresponds to a classic---and low dimensional---machine learning problem. In our current study we used the Random Dot Product Graph model to estimate the embedding of an observed network, and we tested different neural networks architectures to predict the position of nodes in the latent feature space. Flexible machine learning techniques to map the nodes onto their latent positions allow to account for multivariate and possibly complex nodes' metadata. To illustrate the utility of the proposed procedure, we apply it to a large dataset of tourist visits to destinations across New Zealand. We found that our procedure accurately predicts interactions for both existing nodes and nodes newly added to the network, while being computationally feasible even for very large networks. Overall, our study highlights that by exploiting the properties of a well understood statistical model for complex networks and combining it with standard machine learning techniques, we can simplify the link prediction problem when incorporating multivariate node metadata. Our procedure can be immediately applied to different types of networks, and to a wide variety of data from different systems. As such, both from a network science and data science perspective, our work offers a flexible and generalisable procedure for link prediction.

scholarly journals Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques

2020 ◽  
Author(s):  
Georgios Kantidakis ◽  
Hein Putter ◽  
Carlo Lancia ◽  
Jacob de Boer ◽  
Andries E Braat ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Liver Transplantation ◽  
Prediction Models ◽  
Machine Learning Techniques ◽  
Brier Score ◽  
Survival Prediction ◽  
Cox Models ◽  
Learning Techniques ◽  
Random Survival Forest

Abstract Background: Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians.Methods: In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques.Results: Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years.Conclusion: In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables.

International Diversified Portfolio Optimization With Artificial Neural Networks

2019 ◽  
pp. 201-223
Author(s):  
Mehmet Fatih Bayramoglu ◽  
Cagatay Basarir
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Machine Learning Techniques ◽  
Portfolio Performance ◽  
Foreign Companies ◽  
Developed Markets ◽  
Learning Techniques ◽  
Artificial Neural ◽  
Diversified Portfolio

Investing in developed markets offers investors the opportunity to diversify internationally by investing in foreign firms. In other words, it provides the possibility of reducing systematic risk. For this reason, investors are very interested in developed markets. However, developed are more efficient than emerging markets, so the risk and return can be low in these markets. For this reason, developed market investors often use machine learning techniques to increase their gains while reducing their risks. In this chapter, artificial neural networks which is one of the machine learning techniques have been tested to improve internationally diversified portfolio performance. Also, the results of ANNs were compared with the performances of traditional portfolios and the benchmark portfolio. The portfolios are derived from the data of 16 foreign companies quoted on NYSE by ANNs, and they are invested for 30 trading days. According to the results, portfolio derived by ANNs gained 10.30% return, while traditional portfolios gained 5.98% return.

Application of Machine Learning Techniques in the Study of the Relevance of Environmental Factors in Prediction of Tropospheric Ozone

2010 ◽  
pp. 278-292
Author(s):  
Juan Gómez-Sanchis ◽  
Emilio Soria-Olivas ◽  
Marcelino Martinez-Sober ◽  
Jose Blasco ◽  
Juan Guerrero ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Ozone Concentration ◽  
Tropospheric Ozone ◽  
Prediction Accuracy ◽  
Linear Models ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
Atmospheric Phenomena ◽  
Atmospheric Concentrations

This work presents a new approach for one of the main problems in the analysis of atmospheric phenomena, the prediction of atmospheric concentrations of different elements. The proposed methodology is more efficient than other classical approaches and is used in this work to predict tropospheric ozone concentration. The relevance of this problem stems from the fact that excessive ozone concentrations may cause several problems related to public health. Previous research by the authors of this work has shown that the classical approach to this problem (linear models) does not achieve satisfactory results in tropospheric ozone concentration prediction. The authors’ approach is based on Machine Learning (ML) techniques, which include algorithms related to neural networks, fuzzy systems and advanced statistical techniques for data processing. In this work, the authors focus on one of the main ML techniques, namely, neural networks. These models demonstrate their suitability for this problem both in terms of prediction accuracy and information extraction.

A Review of Machine Learning Techniques for Anomaly Detection in Static Graphs

2020 ◽  
pp. 146-162
Author(s):  
Hesham M. Al-Ammal
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Anomaly Detection ◽  
Real Life ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Learning Methods ◽  
Data Set ◽  
Learning Techniques ◽  
Vector Machines

Detection of anomalies in a given data set is a vital step in several applications in cybersecurity; including intrusion detection, fraud, and social network analysis. Many of these techniques detect anomalies by examining graph-based data. Analyzing graphs makes it possible to capture relationships, communities, as well as anomalies. The advantage of using graphs is that many real-life situations can be easily modeled by a graph that captures their structure and inter-dependencies. Although anomaly detection in graphs dates back to the 1990s, recent advances in research utilized machine learning methods for anomaly detection over graphs. This chapter will concentrate on static graphs (both labeled and unlabeled), and the chapter summarizes some of these recent studies in machine learning for anomaly detection in graphs. This includes methods such as support vector machines, neural networks, generative neural networks, and deep learning methods. The chapter will reflect the success and challenges of using these methods in the context of graph-based anomaly detection.

Locally Recurrent Neural Networks and Their Applications

2010 ◽  
pp. 195-222
Author(s):  
Todor D. Ganchev
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Computational Models ◽  
Machine Learning Techniques ◽  
Spatial Correlations ◽  
Temporal Correlations ◽  
Learning Techniques ◽  
Locally Recurrent ◽  
The One ◽  
And Control

In this chapter we review various computational models of locally recurrent neurons and deliberate the architecture of some archetypal locally recurrent neural networks (LRNNs) that are based on them. Generalizations of these structures are discussed as well. Furthermore, we point at a number of realworld applications of LRNNs that have been reported in past and recent publications. These applications involve classification or prediction of temporal sequences, discovering and modeling of spatial and temporal correlations, process identification and control, etc. Validation experiments reported in these developments provide evidence that locally recurrent architectures are capable of identifying and exploiting temporal and spatial correlations (i.e., the context in which events occur), which is the main reason for their advantageous performance when compared with the one of their non-recurrent counterparts or other reasonable machine learning techniques.

Educational Software Based on Matlab GUIs for Neural Networks Courses

2016 ◽  
pp. 333-358 ◽  
Author(s):  
Pablo Díaz-Moreno ◽  
Juan José Carrasco ◽  
Emilio Soria-Olivas ◽  
José M. Martínez-Martínez ◽  
Pablo Escandell-Montero ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Educational Software ◽  
Machine Learning Techniques ◽  
Teaching Tool ◽  
Neural Models ◽  
Learning Techniques ◽  
The World ◽  
Programming Skills

Neural Networks (NN) are one of the most used machine learning techniques in different areas of knowledge. This has led to the emergence of a large number of courses of Neural Networks around the world and in areas where the users of this technique do not have a lot of programming skills. Current software that implements these elements, such as Matlab®, has a number of important limitations in teaching field. In some cases, the implementation of a MLP requires a thorough knowledge of the software and of the instructions that train and validate these systems. In other cases, the architecture of the model is fixed and they do not allow an automatic sweep of the parameters that determine the architecture of the network. This chapter presents a teaching tool for the its use in courses about neural models that solves some of the above-mentioned limitations. This tool is based on Matlab® software.

scholarly journals Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Georgios Kantidakis ◽  
Hein Putter ◽  
Carlo Lancia ◽  
Jacob de Boer ◽  
Andries E. Braat ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Factors ◽  
Neural Networks ◽  
Liver Transplantation ◽  
Prediction Models ◽  
Machine Learning Techniques ◽  
Brier Score ◽  
Cox Models ◽  
Learning Techniques ◽  
Random Survival Forest

Abstract Background Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. Methods In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. Results Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. Conclusion In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables. Trial registration Retrospective data were provided by the Scientific Registry of Transplant Recipients under Data Use Agreement number 9477 for analysis of risk factors after liver transplantation.

Sign in / Sign up

Export Citation Format

Share Document