scholarly journals Contrastive Model Invertion for Data-Free Knolwedge Distillation

2021 ◽  
Author(s):  
Gongfan Fang ◽  
Jie Song ◽  
Xinchao Wang ◽  
Chengchao Shen ◽  
Xingen Wang ◽  
...  
Keyword(s):  
The State ◽  
Learning Objective ◽  
Training Data ◽  
Superior Performance ◽  
Model Inversion ◽  
Inversion Methods ◽  
The Arts ◽  
Limited Effectiveness ◽  
Knowledge Distillation ◽  
Main Observation

Model inversion, whose goal is to recover training data from a pre-trained model, has been recently proved feasible. However, existing inversion methods usually suffer from the mode collapse problem, where the synthesized instances are highly similar to each other and thus show limited effectiveness for downstream tasks, such as knowledge distillation. In this paper, we propose Contrastive Model Inversion (CMI), where the data diversity is explicitly modeled as an optimizable objective, to alleviate the mode collapse issue. Our main observation is that, under the constraint of the same amount of data, higher data diversity usually indicates stronger instance discrimination. To this end, we introduce in CMI a contrastive learning objective that encourages the synthesizing instances to be distinguishable from the already synthesized ones in previous batches. Experiments of pre-trained models on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CMI not only generates more visually plausible instances than the state of the arts, but also achieves significantly superior performance when the generated data are used for knowledge distillation. Code is available at https://github.com/zju-vipa/DataFree.

scholarly journals HCR-Net: A Hybrid of Classification and Regression Network for Object Pose Estimation

2018 ◽  
Author(s):  
Zairan Wang ◽  
Weiming Li ◽  
Yueying Kao ◽  
Dongqing Zou ◽  
Qiang Wang ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Data Augmentation ◽  
The State ◽  
Training Data ◽  
Classification Methods ◽  
Regression Problem ◽  
Regression Methods ◽  
The Arts ◽  
Object Pose Estimation ◽  
Imbalanced Training Data

Object pose estimation from a single image is a fundamental and challenging problem in computer vision and robotics. Generally, current methods treat pose estimation as a classification or a regression problem. However, regression based methods usually suffer from the issue of imbalanced training data, while classification methods are difficult to discriminate nearby poses. In this paper, a hybrid CNN model, which we call it HCR-Net that integrates both a classification network and a regression network, is proposed to deal with these issues. Our model is inspired by that regression methods can get better accuracy on homogeneously distributed datasets while classification methods are more effective for coarse quantization of the poses even if the dataset is not well balanced. The classification methods and the regression methods essentially complement each other. Thus we integrate both them into a neural network in a hybrid fashion and train it end-to-end with two novel loss functions. As a result, our method surpass the state-of-the-art methods, even with imbalanced training data and  much less data augmentation. The experimental results on the challenging Pascal3D+ database demonstrate that our method outperforms the state-of-the-arts significantly, achieving improvements on ACC and AVP metrics up to 4% and 6%, respectively.

Accurate and Transferable Multitask Prediction of Chemical Properties with an Atoms-in-Molecule Neural Network

2018 ◽  
Author(s):  
Roman Zubatyuk ◽  
Justin S. Smith ◽  
Jerzy Leszczynski ◽  
Olexandr Isayev
Keyword(s):  
Neural Network ◽  
Molecular System ◽  
Computational Cost ◽  
Chemical Properties ◽  
The State ◽  
Molecular Properties ◽  
Training Data ◽  
Dft Methods ◽  
Benchmark Datasets ◽  
Quantum Phenomena

<p>Atomic and molecular properties could be evaluated from the fundamental Schrodinger’s equation and therefore represent different modalities of the same quantum phenomena. Here we present AIMNet, a modular and chemically inspired deep neural network potential. We used AIMNet with multitarget training to learn multiple modalities of the state of the atom in a molecular system. The resulting model shows on several benchmark datasets the state-of-the-art accuracy, comparable to the results of orders of magnitude more expensive DFT methods. It can simultaneously predict several atomic and molecular properties without an increase in computational cost. With AIMNet we show a new dimension of transferability: the ability to learn new targets utilizing multimodal information from previous training. The model can learn implicit solvation energy (like SMD) utilizing only a fraction of original training data, and archive MAD error of 1.1 kcal/mol compared to experimental solvation free energies in MNSol database.</p>

scholarly journals SAR Target Recognition via Meta-Learning and Amortized Variational Inference

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5966
Author(s):  
Ke Wang ◽  
Gong Zhang
Keyword(s):  
Target Recognition ◽  
Probability Distributions ◽  
Automatic Target Recognition ◽  
Variational Inference ◽  
Training Data ◽  
Superior Performance ◽  
Small Data ◽  
Meta Learning ◽  
Radar Automatic Target Recognition ◽  
Global Parameters

The challenge of small data has emerged in synthetic aperture radar automatic target recognition (SAR-ATR) problems. Most SAR-ATR methods are data-driven and require a lot of training data that are expensive to collect. To address this challenge, we propose a recognition model that incorporates meta-learning and amortized variational inference (AVI). Specifically, the model consists of global parameters and task-specific parameters. The global parameters, trained by meta-learning, construct a common feature extractor shared between all recognition tasks. The task-specific parameters, modeled by probability distributions, can adapt to new tasks with a small amount of training data. To reduce the computation and storage cost, the task-specific parameters are inferred by AVI implemented with set-to-set functions. Extensive experiments were conducted on a real SAR dataset to evaluate the effectiveness of the model. The results of the proposed approach compared with those of the latest SAR-ATR methods show the superior performance of our model, especially on recognition tasks with limited data.

Structural Optimization of Cantilever Mechanical Elements

1986 ◽  
Vol 108 (4) ◽  
pp. 427-433 ◽  
Author(s):  
Eugene I. Rivin
Keyword(s):  
Structural Optimization ◽  
Natural Frequencies ◽  
Superior Performance ◽  
Vibration Absorbers ◽  
Stability Margins ◽  
Dynamic Vibration ◽  
Robot Arms ◽  
Limited Effectiveness ◽  
Mass Ratios ◽  
Dynamic Vibration Absorbers

Naturally limited stiffness of cantilever elements due to lack of constraint from other structural components, together with low structural damping, causes intensive and slow-decaying transient vibrations as well as low stability margins for self-excited vibrations. In cases of dimensional limitations (e.g., boring bars), such common antivibration means as dynamic vibration absorbers have limited effectiveness due to low mass ratios. This paper describes novel concepts of structural optimization of cantilever components by using combinations of rigid and light materials for their design. Two examples are given: tool holders (boring bars) and robot arms. Optimized boring bars demonstrate substantially increased natural frequencies, together with the possibility of greatly enhanced mass ratios for dynamic vibration absorbers. Machining tests with combination boring bars have been performed in comparison with conventional boring bars showing superior performance of the former. Computer optimization of combination-type robot arms has shown a potential of 10–60 percent reduction in tip-of-arm deflection, together with a commensurate reduction of driving torque for a given acceleration, and a higher natural frequencies (i.e., shorter transients). Optimization has been performed for various ratios of bending and joint compliance and various payloads.

scholarly journals Adjacency-Hash-Table Based Public Auditing for Data Integrity in Mobile Cloud Computing

2018 ◽  
Vol 2018 ◽  
pp. 1-12
Author(s):  
Wenqi Chen ◽  
Hui Tian ◽  
Chin-Chen Chang ◽  
Fulin Nan ◽  
Jing Lu
Keyword(s):  
Cloud Computing ◽  
High Speed ◽  
Security Analysis ◽  
Hash Table ◽  
The State ◽  
Public Auditing ◽  
The Arts ◽  
High Speed Data ◽  
And Performance ◽  
Data Updating

Cloud storage, one of the core services of cloud computing, provides an effective way to solve the problems of storage and management caused by high-speed data growth. Thus, a growing number of organizations and individuals tend to store their data in the cloud. However, due to the separation of data ownership and management, it is difficult for users to check the integrity of data in the traditional way. Therefore, many researchers focus on developing several protocols, which can remotely check the integrity of data in the cloud. In this paper, we propose a novel public auditing protocol based on the adjacency-hash table, where dynamic auditing and data updating are more efficient than those of the state of the arts. Moreover, with such an authentication structure, computation and communication costs can be reduced effectively. The security analysis and performance evaluation based on comprehensive experiments demonstrate that our protocol can achieve all the desired properties and outperform the state-of-the-art ones in computing overheads for updating and verification.

scholarly journals Algorithms that remember: model inversion attacks and data protection law

2018 ◽  
Vol 376 (2133) ◽  
pp. 20180083 ◽  
Author(s):  
Michael Veale ◽  
Reuben Binns ◽  
Lilian Edwards
Keyword(s):  
Data Protection ◽  
Personal Data ◽  
Training Data ◽  
Theme Issue ◽  
Future Directions ◽  
Model Inversion ◽  
General Data Protection Regulation ◽  
General Data ◽  
Inference Attacks ◽  
Use Of Models

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around ‘model inversion’ and ‘membership inference’ attacks, which indicates that the process of turning training data into machine-learned systems is not one way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation. This article is part of the theme issue ‘Governing artificial intelligence: ethical, legal, and technical opportunities and challenges’.

Sign in / Sign up

Export Citation Format

Share Document