scholarly journals Few Shot Network Compression via Cross Distillation

2020 ◽  
Vol 34 (04) ◽  
pp. 3203-3210
Author(s):  
Haoli Bai ◽  
Jiaxiang Wu ◽  
Irwin King ◽  
Michael Lyu
Keyword(s):  
Training Data ◽  
Fine Tuning ◽  
Privacy And Security ◽  
Estimation Errors ◽  
Security Issues ◽  
Model Compression ◽  
Network Output ◽  
Benchmark Datasets ◽  
And Performance ◽  
Network Compression

Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensure accuracy, which could be challenged by privacy and security issues. As a compromise between privacy and performance, in this paper we investigate few shot network compression: given few samples per class, how can we effectively compress the network with negligible performance drop? The core challenge of few shot network compression lies in high estimation errors from the original network during inference, since the compressed network can easily over-fits on the few training instances. The estimation errors could propagate and accumulate layer-wisely and finally deteriorate the network output. To address the problem, we propose cross distillation, a novel layer-wise knowledge distillation approach. By interweaving hidden layers of teacher and student network, layer-wisely accumulated estimation errors can be effectively reduced. The proposed method offers a general framework compatible with prevalent network compression techniques such as pruning. Extensive experiments n benchmark datasets demonstrate that cross distillation can significantly improve the student network's accuracy when only a few training instances are available.

scholarly journals An Experimental Analysis of Model Compression Techniques for Object Detection

2020 ◽  
Author(s):  
Andrey De Aguiar Salvi ◽  
Rodrigo Coelho Barros
Keyword(s):  
Object Detection ◽  
Experimental Analysis ◽  
State Of The Art ◽  
Neural Architecture ◽  
Model Compression ◽  
Processing Power ◽  
Benchmark Datasets ◽  
The Difference ◽  
And Performance ◽  
Consumption Constraints

Recent research on Convolutional Neural Networks focuses on how to create models with a reduced number of parameters and a smaller storage size while keeping the model’s ability to perform its task, allowing the use of the best CNN for automating tasks in limited devices, with reduced processing power, memory, or energy consumption constraints. There are many different approaches in the literature: removing parameters, reduction of the floating-point precision, creating smaller models that mimic larger models, neural architecture search (NAS), etc. With all those possibilities, it is challenging to say which approach provides a better trade-off between model reduction and performance, due to the difference between the approaches, their respective models, the benchmark datasets, or variations in training details. Therefore, this article contributes to the literature by comparing three state-of-the-art model compression approaches to reduce a well-known convolutional approach for object detection, namely YOLOv3. Our experimental analysis shows that it is possible to create a reduced version of YOLOv3 with 90% fewer parameters and still outperform the original model by pruning parameters. We also create models that require only 0.43% of the original model’s inference effort.

scholarly journals Cooperative Pruning in Cross-Domain Deep Neural Network Compression

2019 ◽  
Author(s):  
Shangyu Chen ◽  
Wenya Wang ◽  
Sinno Jialin Pan
Keyword(s):  
Training Data ◽  
Limited Data ◽  
Source Information ◽  
Target Domain ◽  
Source Domain ◽  
Model Compression ◽  
Cross Domain ◽  
Network Compression ◽  
Computational Ability ◽  
And Storage

The advancement of deep models poses great challenges to real-world deployment because of the limited computational ability and storage space on edge devices. To solve this problem, existing works have made progress to compress deep models by pruning or quantization. However, most existing methods rely on a large amount of training data and a pre-trained model in the same domain. When only limited in-domain training data is available, these methods fail to perform well. This prompts the idea of transferring knowledge from a resource-rich source domain to a target domain with limited data to perform model compression. In this paper, we propose a method to perform cross-domain pruning by cooperatively training in both domains: taking advantage of data and a pre-trained model from the source domain to assist pruning in the target domain. Specifically, source and target pruned models are trained simultaneously and interactively, with source information transferred through the construction of a cooperative pruning mask. Our method significantly improves pruning quality in the target domain, and shed light to model compression in the cross-domain setting.

scholarly journals Blockchain-Based Authentication with Optional Privacy Preservation for Internet of Vehicles

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Jinxin Zhang ◽  
Meng Wu
Keyword(s):  
Privacy Preservation ◽  
Safety Management ◽  
Rapid Development ◽  
Mobile Internet ◽  
Privacy And Security ◽  
Internet Of Vehicles ◽  
Security Issues ◽  
Intelligent Technology ◽  
And Performance ◽  
High Level

With the rapid development of the mobile internet and intelligent technology of in-vehicle equipment, the Internet of Vehicles (IoV), centered on intelligent connected cars, has gradually entered people’s lives. However, these technologies also bring serious privacy risks and security issues in terms of data transmission and storage. In this article, we propose a blockchain-based authentication system to provide vehicle safety management. The privacy and security attributes of various vehicle authentication transactions are based on high-level cryptographic primitives, realizing temporary and formal authentication methods. At the same time, a fair blockchain consensus mechanism Auction of block generation Rights (AoR) is proposed. To demonstrate the feasibility and scalability of the proposed scheme, security and performance analyses are presented. The relevant experimental results show that the scheme can provide superior decentralized management for IoV.

scholarly journals RFID security protocols simulation and prototyping

2021 ◽  
Author(s):  
Xiaohui Yu
Keyword(s):  
Radio Frequency Identification ◽  
Security Protocol ◽  
Security And Privacy ◽  
Privacy And Security ◽  
Rfid Security ◽  
Security Issues ◽  
Frequency Identification ◽  
Symmetric Key ◽  
And Performance ◽  
Attack Models

As Radio Frequency Identification (RFID) technology achieves commercial success, its privacy and security issues are becoming a barrier to limit its potential for future start of the art applications. In this report, we present an investigation of the past and current research related to RFID security algorithms and protocols for product authentication. We also present a novel RFID security protocol based on eXtended Tiny Encryption Algorithm (XTEA). Analysis of the security and privacy level of our proposed protocol is performed using SystemC based modeling and different attack models are simulated to show that the protocol is robust and safe against application, protoypes of these attack models are implemented on FPGA platform. We also compare our proposed protocol technique with similar protocols presented in the near past that also use symmetric key algorithms to verify and demostrate main advantages of our protocol in terms of security and performance.

scholarly journals RZcoin: Ethereum-Based Decentralized Payment with Optional Privacy Service

Entropy ◽  
2020 ◽  
Vol 22 (7) ◽  
pp. 712
Author(s):  
Hong Zhao ◽  
Xue Bai ◽  
Shihui Zheng ◽  
Licheng Wang
Keyword(s):  
Privacy And Security ◽  
Security Issues ◽  
Large Size ◽  
Smart Contract ◽  
Redundancy Problem ◽  
Schnorr Signature ◽  
Equivalent Test ◽  
And Performance ◽  
Turing Complete ◽  
Further Development

As the blockchain 2.0 platform, Ethereum’s turing complete programming language and smart contract components make it play an important role in the commercialization of blockchain. With the further development of blockchain applications, the privacy and security issues of Ethereum have gradually emerged. To solve this problem, we proposed a blockchain privacy protection model called RZcash in the previous work. It implements the dynamically updateable and verifiable hiding of the asset information in Ethereum, namely the account balance and transaction amount. However, RZcash does not pay attention to the key redundancy problem that may be caused by the creation of secret accounts. In addition, the large size of proofs gives it high communication costs. In response to these problems, we further improve RZcash. For the key redundancy problem, we construct a new signature scheme based on the ciphertext equivalent test commitment. Moreover, we use the Schnorr signature and bulletproof to improve the corresponding proof scheme in RZcash, thereby reducing the size of proof. Based on these improvements, we propose a decentralized payment system, called RZcoin, based on Ethereum. Finally, we implement the algorithm model of RZcoin and evaluate its security and performance. The results show that RZcoin has higher security and Lower communication cost than RZcash.

scholarly journals Dual-objective fine-tuning of BERT for entity matching

2021 ◽  
Vol 14 (10) ◽  
pp. 1913-1921
Author(s):  
Ralph Peeters ◽  
Christian Bizer
Keyword(s):  
Classification Problem ◽  
Training Data ◽  
Fine Tuning ◽  
Entity Matching ◽  
Domain Specific ◽  
Benchmark Datasets ◽  
Word Classes ◽  
Multi Class Classification ◽  
Training Pair ◽  
Relevant Word

An increasing number of data providers have adopted shared numbering schemes such as GTIN, ISBN, DUNS, or ORCID numbers for identifying entities in the respective domain. This means for data integration that shared identifiers are often available for a subset of the entity descriptions to be integrated while such identifiers are not available for others. The challenge in these settings is to learn a matcher for entity descriptions without identifiers using the entity descriptions containing identifiers as training data. The task can be approached by learning a binary classifier which distinguishes pairs of entity descriptions for the same real-world entity from descriptions of different entities. The task can also be modeled as a multi-class classification problem by learning classifiers for identifying descriptions of individual entities. We present a dual-objective training method for BERT, called JointBERT, which combines binary matching and multi-class classification, forcing the model to predict the entity identifier for each entity description in a training pair in addition to the match/non-match decision. Our evaluation across five entity matching benchmark datasets shows that dual-objective training can increase the matching performance for seen products by 1% to 5% F1 compared to single-objective Transformer-based methods, given that enough training data is available for both objectives. In order to gain a deeper understanding of the strengths and weaknesses of the proposed method, we compare JointBERT to several other BERT-based matching methods as well as baseline systems along a set of specific matching challenges. This evaluation shows that JointBERT, given enough training data for both objectives, outperforms the other methods on tasks involving seen products, while it underperforms for unseen products. Using a combination of LIME explanations and domain-specific word classes, we analyze the matching decisions of the different deep learning models and conclude that BERT-based models are better at focusing on relevant word classes compared to RNN-based models.

scholarly journals RFID security protocols simulation and prototyping

2021 ◽  
Author(s):  
Xiaohui Yu
Keyword(s):  
Radio Frequency Identification ◽  
Security Protocol ◽  
Security And Privacy ◽  
Privacy And Security ◽  
Rfid Security ◽  
Security Issues ◽  
Frequency Identification ◽  
Symmetric Key ◽  
And Performance ◽  
Attack Models

As Radio Frequency Identification (RFID) technology achieves commercial success, its privacy and security issues are becoming a barrier to limit its potential for future start of the art applications. In this report, we present an investigation of the past and current research related to RFID security algorithms and protocols for product authentication. We also present a novel RFID security protocol based on eXtended Tiny Encryption Algorithm (XTEA). Analysis of the security and privacy level of our proposed protocol is performed using SystemC based modeling and different attack models are simulated to show that the protocol is robust and safe against application, protoypes of these attack models are implemented on FPGA platform. We also compare our proposed protocol technique with similar protocols presented in the near past that also use symmetric key algorithms to verify and demostrate main advantages of our protocol in terms of security and performance.

Accurate and Transferable Multitask Prediction of Chemical Properties with an Atoms-in-Molecule Neural Network

2018 ◽  
Author(s):  
Roman Zubatyuk ◽  
Justin S. Smith ◽  
Jerzy Leszczynski ◽  
Olexandr Isayev
Keyword(s):  
Neural Network ◽  
Molecular System ◽  
Computational Cost ◽  
Chemical Properties ◽  
The State ◽  
Molecular Properties ◽  
Training Data ◽  
Dft Methods ◽  
Benchmark Datasets ◽  
Quantum Phenomena

<p>Atomic and molecular properties could be evaluated from the fundamental Schrodinger’s equation and therefore represent different modalities of the same quantum phenomena. Here we present AIMNet, a modular and chemically inspired deep neural network potential. We used AIMNet with multitarget training to learn multiple modalities of the state of the atom in a molecular system. The resulting model shows on several benchmark datasets the state-of-the-art accuracy, comparable to the results of orders of magnitude more expensive DFT methods. It can simultaneously predict several atomic and molecular properties without an increase in computational cost. With AIMNet we show a new dimension of transferability: the ability to learn new targets utilizing multimodal information from previous training. The model can learn implicit solvation energy (like SMD) utilizing only a fraction of original training data, and archive MAD error of 1.1 kcal/mol compared to experimental solvation free energies in MNSol database.</p>

scholarly journals Experiments with Neural Networks in the Identification and Control of a Magnetic Levitation System Using a Low-Cost Platform

2021 ◽  
Vol 11 (6) ◽  
pp. 2535
Author(s):  
Bruno E. Silva ◽  
Ramiro S. Barbosa
Keyword(s):  
Neural Networks ◽  
Neural Control ◽  
Magnetic Levitation ◽  
Low Cost ◽  
Training Data ◽  
Neural Controllers ◽  
Levitation System ◽  
Internal Model Controller ◽  
Magnetic Levitation System ◽  
And Performance

In this article, we designed and implemented neural controllers to control a nonlinear and unstable magnetic levitation system composed of an electromagnet and a magnetic disk. The objective was to evaluate the implementation and performance of neural control algorithms in a low-cost hardware. In a first phase, we designed two classical controllers with the objective to provide the training data for the neural controllers. After, we identified several neural models of the levitation system using Nonlinear AutoRegressive eXogenous (NARX)-type neural networks that were used to emulate the forward dynamics of the system. Finally, we designed and implemented three neural control structures: the inverse controller, the internal model controller, and the model reference controller for the control of the levitation system. The neural controllers were tested on a low-cost Arduino control platform through MATLAB/Simulink. The experimental results proved the good performance of the neural controllers.

Sign in / Sign up

Export Citation Format

Share Document