scholarly journals Efficient Neural Network Verification via Layer-based Semidefinite Relaxations and Linear Cuts

2021 ◽  
Author(s):  
Ben Batten ◽  
Panagiotis Kouvaros ◽  
Alessio Lomuscio ◽  
Yang Zheng
Keyword(s):  
Neural Network ◽  
Chordal Graph ◽  
Computationally Efficient ◽  
Semidefinite Relaxations ◽  
Relaxation Methods ◽  
Sdp Relaxation ◽  
Graph Decompositions ◽  
Order Of Magnitude ◽  
Network Verification ◽  
Local Robustness

We introduce an efficient and tight layer-based semidefinite relaxation for verifying local robustness of neural networks. The improved tightness is the result of the combination between semidefinite relaxations and linear cuts. We obtain a computationally efficient method by decomposing the semidefinite formulation into layerwise constraints. By leveraging on chordal graph decompositions, we show that the formulation here presented is provably tighter than current approaches. Experiments on a set of benchmark networks show that the approach here proposed enables the verification of more instances compared to other relaxation methods. The results also demonstrate that the SDP relaxation here proposed is one order of magnitude faster than previous SDP methods.

scholarly journals Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

2020 ◽  
Vol 34 (04) ◽  
pp. 3858-3865
Author(s):  
Huijie Feng ◽  
Chunpeng Wu ◽  
Guoyang Chen ◽  
Weifeng Zhang ◽  
Yang Ning
Keyword(s):  
Neural Network ◽  
High Probability ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Computationally Efficient ◽  
Base Classifier ◽  
Training Scheme ◽  
Adversarial Training ◽  
Gaussian Perturbation ◽  
Probabilistic Robustness

Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against ℓ2 norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient and can be implemented in parallel with other empirical defense methods. We discuss how to implement it under both standard (non-adversarial) and adversarial training scheme. At the same time, we also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability. Our extensive experimentation demonstrates the effectiveness of the proposed training and certification approaches on CIFAR-10 and ImageNet datasets.

Enhancing Vibration-Based Structural Health Monitoring via Edge Computing: A Tiny Machine Learning Perspective

2021 ◽  
Author(s):  
Federica Zonzini ◽  
Francesca Romano ◽  
Antonio Carbone ◽  
Matteo Zauli ◽  
Luca De Marchi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Structural Health Monitoring ◽  
Health Monitoring ◽  
Network Architecture ◽  
Binary Classification ◽  
Low Cost ◽  
Classification Problems ◽  
Computationally Efficient ◽  
Structural Health

Abstract Despite the outstanding improvements achieved by artificial intelligence in the Structural Health Monitoring (SHM) field, some challenges need to be coped with. Among them, the necessity to reduce the complexity of the models and the data-to-user latency time which are still affecting state-of-the-art solutions. This is due to the continuous forwarding of a huge amount of data to centralized servers, where the inference process is usually executed in a bulky manner. Conversely, the emerging field of Tiny Machine Learning (TinyML), promoted by the recent advancements by the electronic and information engineering community, made sensor-near data inference a tangible, low-cost and computationally efficient alternative. In line with this observation, this work explored the embodiment of the One Class Classifier Neural Network, i.e., a neural network architecture solving binary classification problems for vibration-based SHM scenarios, into a resource-constrained device. To this end, OCCNN has been ported on the Arduino Nano 33 BLE Sense platform and validated with experimental data from the Z24 bridge use case, reaching an average accuracy and precision of 95% and 94%, respectively.

scholarly journals Semi-Supervised Faster RCNN-Based Person Detection and Load Classification for Far Field Video Surveillance

2019 ◽  
Vol 1 (3) ◽  
pp. 756-767 ◽  
Author(s):  
Haoran Wei ◽  
Nasser Kehtarnavaz
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Video Data ◽  
Far Field ◽  
Neural Network Classifier ◽  
Computationally Efficient ◽  
Person Detection ◽  
Video Cameras ◽  
Processing Steps

This paper presents a semi-supervised faster region-based convolutional neural network (SF-RCNN) approach to detect persons and to classify the load carried by them in video data captured from distances several miles away via high-power lens video cameras. For detection, a set of computationally efficient image processing steps are considered to identify moving areas that may contain a person. These areas are then passed onto a faster RCNN classifier whose convolutional layers consist of ResNet50 transfer learning. Frame labels are obtained in a semi-supervised manner for the training of the faster RCNN classifier. For load classification, another convolutional neural network classifier whose convolutional layers consist of GoogleNet transfer learning is used to distinguish a person carrying a bundle from a person carrying a long arm. Despite the challenges associated with the video dataset examined in terms of the low resolution of persons, the presence of heat haze, and the shaking of the camera, it is shown that the developed approach outperforms the faster RCNN approach.

Spike: A GPU Optimised Spiking Neural Network Simulator

2018 ◽  
Author(s):  
Nasir Ahmad ◽  
James B. Isbister ◽  
Toby St. Clere Smithe ◽  
Simon M. Stringer
Keyword(s):  
Neural Network ◽  
High Performance ◽  
Internal Variables ◽  
Spiking Neural Network ◽  
Network Simulator ◽  
Synaptic Inputs ◽  
Active Synapse ◽  
Order Of Magnitude ◽  
Gpu Parallelism ◽  
Simulation Time

ABSTRACTSpiking Neural Network (SNN) simulations require internal variables – such as the membrane voltages of individual neurons and their synaptic inputs – to be updated on a sub-millisecond resolution. As a result, a single second of simulation time requires many thousands of update calculations per neuron. Furthermore, increases in the scale of SNN models have, accordingly, led to manyfold increases in the runtime of SNN simulations. Existing solutions to this problem of scale include high performance CPU based simulators capable of multithreaded execution (“CPU parallelism”). More recent GPU based simulators have emerged, which aim to utilise GPU parallelism for SNN execution. We have identified several key speedups, which give GPU based simulators up to an order of magnitude performance increase over CPU based simulators on several benchmarks. We present the Spike simulator with three key optimisations: timestep grouping, active synapse grouping, and delay insensitivity. Combined, these optimisations massively increase the speed of executing a SNN simulation and produce a simulator which is, on a single machine, faster than currently available simulators.

Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling

2019 ◽  
Vol 141 (12) ◽  
Author(s):  
Dehao Liu ◽  
Yan Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Numerical Solutions ◽  
Low Cost ◽  
Training Data ◽  
Physical Constraints ◽  
Physical Knowledge ◽  
Materials Modeling ◽  
Order Of Magnitude ◽  
The Difference

Abstract Training machine learning tools such as neural networks require the availability of sizable data, which can be difficult for engineering and scientific applications where experiments or simulations are expensive. In this work, a novel multi-fidelity physics-constrained neural network is proposed to reduce the required amount of training data, where physical knowledge is applied to constrain neural networks, and multi-fidelity networks are constructed to improve training efficiency. A low-cost low-fidelity physics-constrained neural network is used as the baseline model, whereas a limited amount of data from a high-fidelity physics-constrained neural network is used to train a second neural network to predict the difference between the two models. The proposed framework is demonstrated with two-dimensional heat transfer, phase transition, and dendritic growth problems, which are fundamental in materials modeling. Physics is described by partial differential equations. With the same set of training data, the prediction error of physics-constrained neural network can be one order of magnitude lower than that of the classical artificial neural network without physical constraints. The accuracy of the prediction is comparable to those from direct numerical solutions of equations.

Sign in / Sign up

Export Citation Format

Share Document