Searching Configurations in Uncertainty Space: Active Learning of High-Dimensional Neural Network Reactive Potentials

2021 ◽  
Author(s):  
Qidong Lin ◽  
Liang Zhang ◽  
Yaolong Zhang ◽  
Bin Jiang
Keyword(s):  
Neural Network ◽  
Active Learning ◽  
High Dimensional ◽  
Uncertainty Space

scholarly journals Active Learning with Bayesian UNet for Efficient Semantic Image Segmentation

2021 ◽  
Vol 7 (2) ◽  
pp. 37
Author(s):  
Isah Charles Saidu ◽  
Lehel Csató
Keyword(s):  
Neural Network ◽  
Image Segmentation ◽  
Active Learning ◽  
Convolutional Neural Network ◽  
Medical Image ◽  
Segmentation Method ◽  
Semantic Image Segmentation ◽  
Batch Normalization ◽  
Set Up ◽  
Image Datasets

We present a sample-efficient image segmentation method using active learning, we call it Active Bayesian UNet, or AB-UNet. This is a convolutional neural network using batch normalization and max-pool dropout. The Bayesian setup is achieved by exploiting the probabilistic extension of the dropout mechanism, leading to the possibility to use the uncertainty inherently present in the system. We set up our experiments on various medical image datasets and highlight that with a smaller annotation effort our AB-UNet leads to stable training and better generalization. Added to this, we can efficiently choose from an unlabelled dataset.

scholarly journals Methodology for Interactive Labeling of Patched Asphalt Pavement Images Based on U-Net Convolutional Neural Network

2022 ◽  
Vol 14 (2) ◽  
pp. 861
Author(s):  
Han-Cheng Dan ◽  
Hao-Fan Zeng ◽  
Zhi-Heng Zhu ◽  
Ge-Wen Bai ◽  
Wei Cao
Keyword(s):  
Neural Network ◽  
Active Learning ◽  
Convolutional Neural Network ◽  
Sample Size ◽  
Pavement Distress ◽  
Image Labeling ◽  
Gray Area ◽  
Calculation Results ◽  
Labeling Method ◽  
And Training

Image recognition based on deep learning generally demands a huge sample size for training, for which the image labeling becomes inevitably laborious and time-consuming. In the case of evaluating the pavement quality condition, many pavement distress patching images would need manual screening and labeling, meanwhile the subjectivity of the labeling personnel would greatly affect the accuracy of image labeling. In this study, in order for an accurate and efficient recognition of the pavement patching images, an interactive labeling method is proposed based on the U-Net convolutional neural network, using active learning combined with reverse and correction labeling. According to the calculation results in this paper, the sample size required by the interactive labeling is about half of the traditional labeling method for the same recognition precision. Meanwhile, the accuracy of interactive labeling method based on the mean intersection over union (mean_IOU) index is 6% higher than that of the traditional method using the same sample size and training epochs. In addition, the accuracy analysis of the noise and boundary of the prediction results shows that this method eliminates 92% of the noise in the predictions (the proportion of noise is reduced from 13.85% to 1.06%), and the image definition is improved by 14.1% in terms of the boundary gray area ratio. The interactive labeling is considered as a significantly valuable approach, as it reduces the sample size in each epoch of active learning, greatly alleviates the demand for manpower, and improves learning efficiency and accuracy.

scholarly journals A Comparison of Machine Learning Methods in a High-Dimensional Classification Problem

2014 ◽  
Vol 5 (3) ◽  
pp. 82-96 ◽  
Author(s):  
Marijana Zekić-Sušac ◽  
Sanja Pfeifer ◽  
Nataša Šarlija
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Classification Accuracy ◽  
Classification Problem ◽  
High Dimensional ◽  
Nearest Neighbour ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Dimensional Classification ◽  
Artificial Neural

Abstract Background: Large-dimensional data modelling often relies on variable reduction methods in the pre-processing and in the post-processing stage. However, such a reduction usually provides less information and yields a lower accuracy of the model. Objectives: The aim of this paper is to assess the high-dimensional classification problem of recognizing entrepreneurial intentions of students by machine learning methods. Methods/Approach: Four methods were tested: artificial neural networks, CART classification trees, support vector machines, and k-nearest neighbour on the same dataset in order to compare their efficiency in the sense of classification accuracy. The performance of each method was compared on ten subsamples in a 10-fold cross-validation procedure in order to assess computing sensitivity and specificity of each model. Results: The artificial neural network model based on multilayer perceptron yielded a higher classification rate than the models produced by other methods. The pairwise t-test showed a statistical significance between the artificial neural network and the k-nearest neighbour model, while the difference among other methods was not statistically significant. Conclusions: Tested machine learning methods are able to learn fast and achieve high classification accuracy. However, further advancement can be assured by testing a few additional methodological refinements in machine learning methods.

Combining Self-supervised Learning and Active Learning for Disfluency Detection

2022 ◽  
Vol 21 (3) ◽  
pp. 1-25
Author(s):  
Shaolei Wang ◽  
Zhongyuan Wang ◽  
Wanxiang Che ◽  
Sendong Zhao ◽  
Ting Liu
Keyword(s):  
Neural Network ◽  
Active Learning ◽  
Supervised Learning ◽  
Large Scale ◽  
Training Data ◽  
Fine Tuning ◽  
Training Dataset ◽  
Performance Gap ◽  
Annotation Costs ◽  
Trained Neural Network

Spoken language is fundamentally different from the written language in that it contains frequent disfluencies or parts of an utterance that are corrected by the speaker. Disfluency detection (removing these disfluencies) is desirable to clean the input for use in downstream NLP tasks. Most existing approaches to disfluency detection heavily rely on human-annotated data, which is scarce and expensive to obtain in practice. To tackle the training data bottleneck, in this work, we investigate methods for combining self-supervised learning and active learning for disfluency detection. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled data and propose two self-supervised pre-training tasks: (i) a tagging task to detect the added noisy words and (ii) sentence classification to distinguish original sentences from grammatically incorrect sentences. We then combine these two tasks to jointly pre-train a neural network. The pre-trained neural network is then fine-tuned using human-annotated disfluency detection training data. The self-supervised learning method can capture task-special knowledge for disfluency detection and achieve better performance when fine-tuning on a small annotated dataset compared to other supervised methods. However, limited in that the pseudo training data are generated based on simple heuristics and cannot fully cover all the disfluency patterns, there is still a performance gap compared to the supervised models trained on the full training dataset. We further explore how to bridge the performance gap by integrating active learning during the fine-tuning process. Active learning strives to reduce annotation costs by choosing the most critical examples to label and can address the weakness of self-supervised learning with a small annotated dataset. We show that by combining self-supervised learning with active learning, our model is able to match state-of-the-art performance with just about 10% of the original training data on both the commonly used English Switchboard test set and a set of in-house annotated Chinese data.

scholarly journals Using principal component analysis for neural network high-dimensional potential energy surface

2020 ◽  
Vol 152 (23) ◽  
pp. 234103
Author(s):  
Bastien Casier ◽  
Stéphane Carniato ◽  
Tsveta Miteva ◽  
Nathalie Capron ◽  
Nicolas Sisourat
Keyword(s):  
Neural Network ◽  
Principal Component Analysis ◽  
Potential Energy ◽  
Potential Energy Surface ◽  
Energy Surface ◽  
Principal Component ◽  
Component Analysis ◽  
High Dimensional
Sign in / Sign up

Export Citation Format

Share Document