Solving inverse problems in stochastic models using deep neural networks and adversarial training

2021 ◽  
Vol 384 ◽  
pp. 113976
Author(s):  
Kailai Xu ◽  
Eric Darve
Keyword(s):  
Neural Networks ◽  
Inverse Problems ◽  
Stochastic Models ◽  
Deep Neural Networks ◽  
Adversarial Training

scholarly journals Diversity Adversarial Training against Adversarial Attack on Deep Neural Networks

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 428
Author(s):  
Hyun Kwon ◽  
Jun Lee
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Diversity Training ◽  
Original Data ◽  
Training Method ◽  
Learning Framework ◽  
Adversarial Examples ◽  
Adversarial Training ◽  
Adversarial Attack ◽  
Accuracy Rates

This paper presents research focusing on visualization and pattern recognition based on computer science. Although deep neural networks demonstrate satisfactory performance regarding image and voice recognition, as well as pattern analysis and intrusion detection, they exhibit inferior performance towards adversarial examples. Noise introduction, to some degree, to the original data could lead adversarial examples to be misclassified by deep neural networks, even though they can still be deemed as normal by humans. In this paper, a robust diversity adversarial training method against adversarial attacks was demonstrated. In this approach, the target model is more robust to unknown adversarial examples, as it trains various adversarial samples. During the experiment, Tensorflow was employed as our deep learning framework, while MNIST and Fashion-MNIST were used as experimental datasets. Results revealed that the diversity training method has lowered the attack success rate by an average of 27.2 and 24.3% for various adversarial examples, while maintaining the 98.7 and 91.5% accuracy rates regarding the original data of MNIST and Fashion-MNIST.

scholarly journals Semisupervised Learning for Seismic Monitoring Applications

2020 ◽  
Vol 92 (1) ◽  
pp. 388-395
Author(s):  
Lisa Linville ◽  
Dylan Anderson ◽  
Joshua Michalenko ◽  
Jennifer Galasso ◽  
Timothy Draelos
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Seismic Event ◽  
Model Performance ◽  
Seismic Monitoring ◽  
Semisupervised Learning ◽  
Unlabeled Data ◽  
Hybrid Technique ◽  
Source Type ◽  
Adversarial Training

Abstract The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. In geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.

scholarly journals Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss

2019 ◽  
Author(s):  
Pengcheng Li ◽  
Jinfeng Yi ◽  
Bowen Zhou ◽  
Lijun Zhang
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Current Model ◽  
Empirical Studies ◽  
Metric Learning ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Classification Boundary ◽  
Adversarial Training ◽  
Triplet Loss

Recent studies have highlighted that deep neural networks (DNNs) are vulnerable to adversarial examples. In this paper, we improve the robustness of DNNs by utilizing techniques of Distance Metric Learning. Specifically, we incorporate Triplet Loss, one of the most popular Distance Metric Learning methods, into the framework of adversarial training. Our proposed algorithm, Adversarial Training with Triplet Loss (AT2L), substitutes the adversarial example against the current model for the anchor of triplet loss to effectively smooth the classification boundary. Furthermore, we propose an ensemble version of AT2L, which aggregates different attack methods and model structures for better defense effects. Our empirical studies verify that the proposed approach can significantly improve the robustness of DNNs without sacrificing accuracy. Finally, we demonstrate that our specially designed triplet loss can also be used as a regularization term to enhance other defense methods.

scholarly journals Privacy Issues Regarding the Application of DNNs to Activity-Recognition using Wearables and Its Countermeasures by Use of Adversarial Training

2017 ◽  
Author(s):  
Yusuke Iwasawa ◽  
Kotaro Nakayama ◽  
Ikuko Yairi ◽  
Yutaka Matsuo
Keyword(s):  
Neural Networks ◽  
Activity Recognition ◽  
Information Disclosure ◽  
Deep Neural Networks ◽  
Recognition Performance ◽  
Empirical Validation ◽  
Privacy Concerns ◽  
Proposed Model ◽  
Significant Performance ◽  
Adversarial Training

Deep neural networks have been successfully applied to activity recognition with wearables in terms of recognition performance. However, the black-box nature of neural networks could lead to privacy concerns. Namely, generally it is hard to expect what neural networks learn from data, and so they possibly learn features that highly discriminate user-information unintentionally, which increases the risk of information-disclosure. In this study, we analyzed the features learned by conventional deep neural networks when applied to data of wearables to confirm this phenomenon.Based on the results of our analysis, we propose the use of an adversarial training framework to suppress the risk of sensitive/unintended information disclosure. Our proposed model considers both an adversarial user classifier and a regular activity-classifier during training, which allows the model to learn representations that help the classifier to distinguish the activities but which, at the same time, prevents it from accessing user-discriminative information. This paper provides an empirical validation of the privacy issue and efficacy of the proposed method using three activity recognition tasks based on data of wearables. The empirical validation shows that our proposed method suppresses the concerns without any significant performance degradation, compared to conventional deep nets on all three tasks.

scholarly journals Big in Japan: Regularizing Networks for Solving Inverse Problems

2019 ◽  
Vol 62 (3) ◽  
pp. 445-455
Author(s):  
Johannes Schwab ◽  
Stephan Antholzer ◽  
Markus Haltmeier
Keyword(s):  
Neural Networks ◽  
Inverse Problems ◽  
Regularization Method ◽  
Deep Neural Networks ◽  
Convergence Rates ◽  
Null Space ◽  
Space Network ◽  
Special Cases ◽  
Data Problem ◽  
Sparse Data Problem

Abstract Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep regularizing neural networks (RegNets) of the form $$\mathbf {B}_\alpha + \mathbf {N}_{\theta (\alpha )} \mathbf {B}_\alpha $$Bα+Nθ(α)Bα, where $$\mathbf {B}_\alpha $$Bα is a classical regularization and the network $$\mathbf {N}_{\theta (\alpha )} \mathbf {B}_\alpha $$Nθ(α)Bα is trained to recover the missing part $${\text {Id}}_X - \mathbf {B}_\alpha $$IdX-Bα not found by the classical regularization. We show that these regularizing networks yield a convergent regularization method for solving inverse problems. Additionally, we derive convergence rates (quantitative error estimates) assuming a sufficient decay of the associated distance function. We demonstrate that our results recover existing convergence and convergence rates results for filter-based regularization methods as well as the recently introduced null space network as special cases. Numerical results are presented for a tomographic sparse data problem, which clearly demonstrate that the proposed RegNets improve classical regularization as well as the null space network.

scholarly journals Solving ill-posed inverse problems using iterative deep neural networks

2017 ◽  
Vol 33 (12) ◽  
pp. 124007 ◽  
Author(s):  
Jonas Adler ◽  
Ozan Öktem
Keyword(s):  
Neural Networks ◽  
Inverse Problems ◽  
Deep Neural Networks ◽  
Ill Posed

scholarly journals Deep Neural Networks for Inverse Problems with Pseudodifferential Operators: An Application to Limited-Angle Tomography

2021 ◽  
Vol 14 (2) ◽  
pp. 470-505
Author(s):  
Tatiana A. Bubba ◽  
Mathilde Galinier ◽  
Matti Lassas ◽  
Marco Prato ◽  
Luca Ratti ◽  
...  
Keyword(s):  
Neural Networks ◽  
Inverse Problems ◽  
Deep Neural Networks ◽  
Pseudodifferential Operators ◽  
Limited Angle

scholarly journals Invariant Representations through Adversarial Forgetting

2020 ◽  
Vol 34 (04) ◽  
pp. 4272-4279
Author(s):  
Ayush Jaiswal ◽  
Daniel Moyer ◽  
Greg Ver Steeg ◽  
Wael AbdAlmageed ◽  
Premkumar Natarajan
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Empirical Results ◽  
Information Bottleneck ◽  
Novel Approach ◽  
Adversarial Training ◽  
Invariant Representations ◽  
Art Performance ◽  
Forgetting Mechanism

We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism. We show that the forgetting mechanism serves as an information-bottleneck, which is manipulated by the adversarial training to learn invariance to unwanted factors. Empirical results show that the proposed framework achieves state-of-the-art performance at learning invariance in both nuisance and bias settings on a diverse collection of datasets and tasks.

Robustness of Compressed Deep Neural Networks with Adversarial Training

2021 ◽  
Author(s):  
Zhang Yunchun ◽  
Li Chengjie ◽  
Wang Wangwang ◽  
Zhong Yuting ◽  
Zhang Xin ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Adversarial Training

Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods

2018 ◽  
Vol 35 (1) ◽  
pp. 20-36 ◽  
Author(s):  
Alice Lucas ◽  
Michael Iliadis ◽  
Rafael Molina ◽  
Aggelos K. Katsaggelos
Keyword(s):  
Neural Networks ◽  
Inverse Problems ◽  
Analytical Methods ◽  
Deep Neural Networks
Sign in / Sign up

Export Citation Format

Share Document