Nonlinear Chen-Lee Chaotic System Based Deep Convolutional Generative Adversarial Nets for Chatter Diagnosis

Chatter has a direct effect on the precision and life of machine tools and its detection is a crucial issue in all metal machining processes. Traditional methods focus on how to extract discriminative features to help identify chatter. Nowadays, deep learning models have shown an extraordinary ability to extract data features which are their necessary fuel. In this study deep learning models have been substituted for more traditional methods. Chatter data are rare and valuable because the collecting process is extremely difficult. To solve this practical problem an innovative training strategy has been proposed that is combined with a modified convolutional neural network and deep convolutional generative adversarial nets. This improves chatter detection and classification. Convolutional neural networks can be effective chatter classifiers, and adversarial networks can act as generators that produce more data. The convolutional neural networks were trained using original data as well as by forged data produced by the generator. Original training data were collected and preprocessed by the Chen-Lee chaotic system. The adversarial training process used these data to create the generator and the generator could produce enough data to compensate for the lack of training data. The experimental results were compared with without a data generator and data augmentation. The proposed method had an accuracy of 95.3% on leave-one-out cross-validation over ten runs and surpassed other methods and models. The forged data were also compared with original training data as well as data produced by augmentation. The distribution shows that forged data had similar quality and characteristics to the original data. The proposed training strategy provides a high-quality deep learning chatter detection model.

Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study

Generative adversarial networks (GANs) nowadays are capable of producing images of incredible realism. Two concerns raised are whether the state-of-the-art GAN’s learned distribution still suffers from mode collapse and what to do if so. Existing diversity tests of samples from GANs are usually conducted qualitatively on a small scale and/or depend on the access to original training data as well as the trained model parameters. This article explores GAN intra-mode collapse and calibrates that in a novel black-box setting: access to neither training data nor the trained model parameters is assumed. The new setting is practically demanded yet rarely explored and significantly more challenging. As a first stab, we devise a set of statistical tools based on sampling that can visualize, quantify, and rectify intra-mode collapse . We demonstrate the effectiveness of our proposed diagnosis and calibration techniques, via extensive simulations and experiments, on unconditional GAN image generation (e.g., face and vehicle). Our study reveals that the intra-mode collapse is still a prevailing problem in state-of-the-art GANs and the mode collapse is diagnosable and calibratable in black-box settings. Our codes are available at https://github.com/VITA-Group/BlackBoxGANCollapse .

Target Recognition of Synthetic Aperture Radar Images by Updated Classifiers

For the problem of reliable decision in synthetic aperture radar (SAR) target recognition, a method based on updated classifiers is proposed. The convolutional neural network (CNN) and support vector machine (SVM) are used as basic classifiers to classify samples with unknown target labels. The two decisions are fused and the reliability of the fused decision is evaluated. The classified test samples with high reliabilities are added to the original training samples to update the classifiers. The updated classifiers have stronger classification abilities and the fused result of the two classifiers can obtain a more reliable decision. The proposed method is tested and verified based on the moving and stationary target acquisition and recognition (MSTAR) dataset. The experimental results verify the effectiveness and robustness of the proposed method.

No Fine-Tuning, No Cry: Robust SVD for Compressing Deep Networks

A common technique for compressing a neural network is to compute the k-rank ℓ2 approximation Ak of the matrix A∈Rn×d via SVD that corresponds to a fully connected layer (or embedding layer). Here, d is the number of input neurons in the layer, n is the number in the next one, and Ak is stored in O((n+d)k) memory instead of O(nd). Then, a fine-tuning step is used to improve this initial compression. However, end users may not have the required computation resources, time, or budget to run this fine-tuning stage. Furthermore, the original training set may not be available. In this paper, we provide an algorithm for compressing neural networks using a similar initial compression time (to common techniques) but without the fine-tuning step. The main idea is replacing the k-rank ℓ2 approximation with ℓp, for p∈[1,2], which is known to be less sensitive to outliers but much harder to compute. Our main technical result is a practical and provable approximation algorithm to compute it for any p≥1, based on modern techniques in computational geometry. Extensive experimental results on the GLUE benchmark for compressing the networks BERT, DistilBERT, XLNet, and RoBERTa confirm this theoretical advantage.

AutoReCon: Neural Architecture Search-based Reconstruction for Data-free Compression

Data-free compression raises a new challenge because the original training dataset for a pre-trained model to be compressed is not available due to privacy or transmission issues. Thus, a common approach is to compute a reconstructed training dataset before compression. The current reconstruction methods compute the reconstructed training dataset with a generator by exploiting information from the pre-trained model. However, current reconstruction methods focus on extracting more information from the pre-trained model but do not leverage network engineering. This work is the first to consider network engineering as an approach to design the reconstruction method. Specifically, we propose the AutoReCon method, which is a neural architecture search-based reconstruction method. In the proposed AutoReCon method, the generator architecture is designed automatically given the pre-trained model for reconstruction. Experimental results show that using generators discovered by the AutoRecon method always improve the performance of data-free compression.

Protect privacy of deep classification networks by exploiting their generative power

Research showed that deep learning models are vulnerable to membership inference attacks, which aim to determine if an example is in the training set of the model. We propose a new framework to defend against this sort of attack. Our key insight is that if we retrain the original classifier with a new dataset that is independent of the original training set while their elements are sampled from the same distribution, the retrained classifier will leak no information that cannot be inferred from the distribution about the original training set. Our framework consists of three phases. First, we transferred the original classifier to a Joint Energy-based Model (JEM) to exploit the model’s implicit generative power. Then, we sampled from the JEM to create a new dataset. Finally, we used the new dataset to retrain or fine-tune the original classifier. We empirically studied different transfer learning schemes for the JEM and fine-tuning/retraining strategies for the classifier against shadow-model attacks. Our evaluation shows that our framework can suppress the attacker’s membership advantage to a negligible level while keeping the classifier’s accuracy acceptable. We compared it with other state-of-the-art defenses considering adaptive attackers and showed our defense is effective even under the worst-case scenario. Besides, we also found that combining other defenses with our framework often achieves better robustness. Our code will be made available at https://github.com/ChenJiyu/meminf-defense.git.

Nondominant hand computer mouse training and the bilateral transfer effect to the dominant hand

This study explored the effects of training computer mouse use in the nondominant hand on clicking performance of the dominant and nondominant hands. Computer mouse use is a daily operation in the workplace and requires minute hand and wrist movements developed and refined through practice and training for many years. Our study had eleven right-handed computer mouse users train their nondominant hand for 15 min a day, five days per week, for six weeks. This study found improved performance with the computer mouse in the dominant hand following nondominant hand training because of the bilateral transfer effect of training. Additionally, our study showed that the nondominant hand is capable of learning the complex movements that our dominant hand has trained for many years. Last, our research showed that nondominant hand performance decreases when the skill is not trained for over a year, but the performance is significantly higher than that prior to the original training and can be rapidly relearned. Overall, training the nondominant hand on the computer mouse will allow for improved performance in industry while allowing safer, sustainable, and more achievable work in a multitude of economies.

Deep Learning-Based Computer-Aided Diagnosis System for Gastroscopy Image Classification Using Synthetic Data

Gastric cancer has a high mortality rate worldwide, but it can be prevented with early detection through regular gastroscopy. Herein, we propose a deep learning-based computer-aided diagnosis (CADx) system applying data augmentation to help doctors classify gastroscopy images as normal or abnormal. To improve the performance of deep learning, a large amount of training data are required. However, the collection of medical data, owing to their nature, is highly expensive and time consuming. Therefore, data were generated through deep convolutional generative adversarial networks (DCGAN), and 25 augmentation policies optimized for the CIFAR-10 dataset were implemented through AutoAugment to augment the data. Accordingly, a gastroscopy image was augmented, only high-quality images were selected through an image quality-measurement method, and gastroscopy images were classified as normal or abnormal through the Xception network. We compared the performances of the original training dataset, which did not improve, the dataset generated through the DCGAN, the dataset augmented through the augmentation policies of CIFAR-10, and the dataset combining the two methods. The dataset combining the two methods delivered the best performance in terms of accuracy (0.851) and achieved an improvement of 0.06 over the original training dataset. We confirmed that augmenting data through the DCGAN and CIFAR-10 augmentation policies is most suitable for the classification model for normal and abnormal gastric endoscopy images. The proposed method not only solves the medical-data problem but also improves the accuracy of gastric disease diagnosis.

Neural machine translation of low-resource languages using SMT phrase pair injection

Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

A Dictionary Learning Algorithm Based on Dictionary Reconstruction and Its Application in Face Recognition

In recent years, dictionary learning has received more and more attention in the study of face recognition. However, most dictionary learning algorithms directly use the original training samples to learn the dictionary, ignoring noise existing in the training samples. For example, there are differences between different images of the same subject due to changes in illumination, expression, etc. To address the above problems, this paper proposes the dictionary relearning algorithm (DRLA) based on locality constraint and label embedding, which can effectively reduce the influence of noise on the dictionary learning algorithm. In our proposed dictionary learning algorithm, first, the initial dictionary and coding coefficient matrix are directly obtained from the training samples, and then the original training samples are reconstructed by the product of the initial dictionary and coding coefficient matrix. Finally, the dictionary learning algorithm is reapplied to obtain a new dictionary and coding coefficient matrix, and the newly obtained dictionary and coding coefficient matrix are used for subsequent image classification. The dictionary reconstruction method can partially eliminate noise in the original training samples. Therefore, the proposed algorithm can obtain more robust classification results. The experimental results demonstrate that the proposed algorithm performs better in recognition accuracy than some state-of-the-art algorithms.