scholarly journals Improving Generalization via Attribute Selection on Out-of-the-Box Data

2020 ◽  
Vol 32 (2) ◽  
pp. 485-514
Author(s):  
Xiaofeng Xu ◽  
Ivor W. Tsang ◽  
Chuancai Liu
Keyword(s):  
Test Data ◽  
State Of The Art ◽  
Attribute Selection ◽  
Generalization Error ◽  
Generalization Capability ◽  
Training Stage ◽  
Unseen Data ◽  
Negative Impacts ◽  
Unseen Objects ◽  
Pseudo Data

Zero-shot learning (ZSL) aims to recognize unseen objects (test classes) given some other seen objects (training classes) by sharing information of attributes between different objects. Attributes are artificially annotated for objects and treated equally in recent ZSL tasks. However, some inferior attributes with poor predictability or poor discriminability may have negative impacts on the ZSL system performance. This letter first derives a generalization error bound for ZSL tasks. Our theoretical analysis verifies that selecting the subset of key attributes can improve the generalization performance of the original ZSL model, which uses all the attributes. Unfortunately, previous attribute selection methods have been conducted based on the seen data, and their selected attributes have poor generalization capability to the unseen data, which is unavailable in the training stage of ZSL tasks. Inspired by learning from pseudo-relevance feedback, this letter introduces out-of-the-box data—pseudo-data generated by an attribute-guided generative model—to mimic the unseen data. We then present an iterative attribute selection (IAS) strategy that iteratively selects key attributes based on the out-of-the-box data. Since the distribution of the generated out-of-the-box data is similar to that of the test data, the key attributes selected by IAS can be effectively generalized to test data. Extensive experiments demonstrate that IAS can significantly improve existing attribute-based ZSL methods and achieve state-of-the-art performance.

scholarly journals Learning Compact Model for Large-Scale Multi-Label Data

2019 ◽  
Vol 33 ◽  
pp. 5385-5392 ◽  
Author(s):  
Tong Wei ◽  
Yu-Feng Li
Keyword(s):  
Large Scale ◽  
Optimization Problem ◽  
Performance Metrics ◽  
State Of The Art ◽  
Optimization Method ◽  
Model Parameters ◽  
Generalization Capability ◽  
Constrained Optimization Problem ◽  
Unseen Data ◽  
Model Size

Large-scale multi-label learning (LMLL) aims to annotate relevant labels from a large number of candidates for unseen data. Due to the high dimensionality in both feature and label spaces in LMLL, the storage overheads of LMLL models are often costly. This paper proposes a POP (joint label and feature Parameter OPtimization) method. It tries to filter out redundant model parameters to facilitate compact models. Our key insights are as follows. First, we investigate labels that have little impact on the commonly used LMLL performance metrics and only preserve a small number of dominant parameters for these labels. Second, for the remaining influential labels, we reduce spurious feature parameters that have little contribution to the generalization capability of models, and preserve parameters for only discriminative features. The overall problem is formulated as a constrained optimization problem pursuing minimal model size. In order to solve the resultant difficult optimization, we show that a relaxation of the optimization can be efficiently solved using binary search and greedy strategies. Experiments verify that the proposed method clearly reduces the model size compared to state-of-the-art LMLL approaches, in addition, achieves highly competitive performance.

scholarly journals New polyp image classification technique using transfer learning of network-in-network structure in endoscopic images

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Young Jae Kim ◽  
Jang Pyo Bae ◽  
Jun-Won Chung ◽  
Dong Kyun Park ◽  
Kwang Gi Kim ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Transfer Learning ◽  
Test Data ◽  
State Of The Art ◽  
Early Stage ◽  
Statistical Significance ◽  
Recall Rate ◽  
Training Data ◽  
Fine Tuning ◽  
Accuracy Evaluation

AbstractWhile colorectal cancer is known to occur in the gastrointestinal tract. It is the third most common form of cancer of 27 major types of cancer in South Korea and worldwide. Colorectal polyps are known to increase the potential of developing colorectal cancer. Detected polyps need to be resected to reduce the risk of developing cancer. This research improved the performance of polyp classification through the fine-tuning of Network-in-Network (NIN) after applying a pre-trained model of the ImageNet database. Random shuffling is performed 20 times on 1000 colonoscopy images. Each set of data are divided into 800 images of training data and 200 images of test data. An accuracy evaluation is performed on 200 images of test data in 20 experiments. Three compared methods were constructed from AlexNet by transferring the weights trained by three different state-of-the-art databases. A normal AlexNet based method without transfer learning was also compared. The accuracy of the proposed method was higher in statistical significance than the accuracy of four other state-of-the-art methods, and showed an 18.9% improvement over the normal AlexNet based method. The area under the curve was approximately 0.930 ± 0.020, and the recall rate was 0.929 ± 0.029. An automatic algorithm can assist endoscopists in identifying polyps that are adenomatous by considering a high recall rate and accuracy. This system can enable the timely resection of polyps at an early stage.

Performance Evaluation of Convolutional Neural Network Using Synthetic Medical Data Augmentation Generated by GAN

2021 ◽  
Author(s):  
Ramesh Adhikari ◽  
Suresh Pokharel
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Data Augmentation ◽  
Medical Diagnostics ◽  
Generative Adversarial Networks ◽  
Generalization Capability ◽  
X Ray ◽  
Original Dataset ◽  
Unseen Data

Data augmentation is widely used in image processing and pattern recognition problems in order to increase the richness in diversity of available data. It is commonly used to improve the classification accuracy of images when the available datasets are limited. Deep learning approaches have demonstrated an immense breakthrough in medical diagnostics over the last decade. A significant amount of datasets are needed for the effective training of deep neural networks. The appropriate use of data augmentation techniques prevents the model from over-fitting and thus increases the generalization capability of the network while testing afterward on unseen data. However, it remains a huge challenge to obtain such a large dataset from rare diseases in the medical field. This study presents the synthetic data augmentation technique using Generative Adversarial Networks to evaluate the generalization capability of neural networks using existing data more effectively. In this research, the convolutional neural network (CNN) model is used to classify the X-ray images of the human chest in both normal and pneumonia conditions; then, the synthetic images of the X-ray from the available dataset are generated by using the deep convolutional generative adversarial network (DCGAN) model. Finally, the CNN model is trained again with the original dataset and augmented data generated using the DCGAN model. The classification performance of the CNN model is improved by 3.2% when the augmented data were used along with the originally available dataset.

Comparison of Wind Turbine Performance Prediction and Measurement

1982 ◽  
Vol 104 (2) ◽  
pp. 84-88 ◽  
Author(s):  
J. L. Tangler
Keyword(s):  
Test Data ◽  
Wind Turbines ◽  
Performance Prediction ◽  
State Of The Art ◽  
Correlation Study ◽  
Performance Models ◽  
Rocky Flats ◽  
Turbine Performance ◽  
Horizontal Axis Wind Turbines ◽  
Development Center

The purpose of this work was to evaluate the state-of-the-art of performance prediction for small horizontal-axis wind turbines. This effort was undertaken since few of the existing performance methods used to predict rotor power output have been validated with reliable test data. The program involved evaluating several existing performance models from four contractors by comparing their predictions for two wind turbines with actual test data. Test data were acquired by Rocky Flats Test and Development Center and furnished to the contractors after submission of their prediction reports. The results of the correlation study will help identify areas in which existing rotor performance models are inadequate and, where possible, the reasons for the models shortcomings. In addition, several problems associated with obtaining accurate test data will be discussed.

Flat Minima

1997 ◽  
Vol 9 (1) ◽  
pp. 1-42 ◽  
Author(s):  
Sepp Hochreiter ◽  
Jürgen Schmidhuber
Keyword(s):  
Neural Networks ◽  
Error Function ◽  
Low Complexity ◽  
Generalization Error ◽  
Input Output ◽  
Generalization Capability ◽  
Training Set ◽  
Weight Decay ◽  
Optimal Brain Surgeon ◽  
And Training

We present a new algorithm for finding low-complexity neural networks with high generalization capability. The algorithm searches for a “flat” minimum of the error function. A flat minimum is a large connected region in weight space where the error remains approximately constant. An MDL-based, Bayesian argument suggests that flat minima correspond to “simple” networks and low expected overfitting. The argument is based on a Gibbs algorithm variant and a novel way of splitting generalization error into underfitting and overfitting error. Unlike many previous approaches, ours does not require gaussian assumptions and does not depend on a “good” weight prior. Instead we have a prior over input output functions, thus taking into account net architecture and training set. Although our algorithm requires the computation of second-order derivatives, it has backpropagation's order of complexity. Automatically, it effectively prunes units, weights, and input lines. Various experiments with feedforward and recurrent nets are described. In an application to stock market prediction, flat minimum search outperforms conventional backprop, weight decay, and “optimal brain surgeon/optimal brain damage.”

A Comprehensive Taxonomy of Dynamic Texture Representation

2023 ◽  
Vol 55 (1) ◽  
pp. 1-39
Author(s):  
Thanh Tuan Nguyen ◽  
Thanh Phuong Nguyen
Keyword(s):  
Large Scale ◽  
Environmental Changes ◽  
State Of The Art ◽  
The State ◽  
Future Research ◽  
Research Activities ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Negative Impacts ◽  
Made In

Representing dynamic textures (DTs) plays an important role in many real implementations in the computer vision community. Due to the turbulent and non-directional motions of DTs along with the negative impacts of different factors (e.g., environmental changes, noise, illumination, etc.), efficiently analyzing DTs has raised considerable challenges for the state-of-the-art approaches. For 20 years, many different techniques have been introduced to handle the above well-known issues for enhancing the performance. Those methods have shown valuable contributions, but the problems have been incompletely dealt with, particularly recognizing DTs on large-scale datasets. In this article, we present a comprehensive taxonomy of DT representation in order to purposefully give a thorough overview of the existing methods along with overall evaluations of their obtained performances. Accordingly, we arrange the methods into six canonical categories. Each of them is then taken in a brief presentation of its principal methodology stream and various related variants. The effectiveness levels of the state-of-the-art methods are then investigated and thoroughly discussed with respect to quantitative and qualitative evaluations in classifying DTs on benchmark datasets. Finally, we point out several potential applications and the remaining challenges that should be addressed in further directions. In comparison with two existing shallow DT surveys (i.e., the first one is out of date as it was made in 2005, while the newer one (published in 2016) is an inadequate overview), we believe that our proposed comprehensive taxonomy not only provides a better view of DT representation for the target readers but also stimulates future research activities.

The Art and Science of Rotary Wing Data Correlation

1976 ◽  
Vol 21 (3) ◽  
pp. 2-12
Author(s):  
Jan M. Drees
Keyword(s):  
Wind Tunnel ◽  
Test Data ◽  
State Of The Art ◽  
Helicopter Rotor ◽  
Small Scale ◽  
Test Results ◽  
Free Flight ◽  
Full Scale Tests ◽  
Rotary Wing ◽  
Theoretical Analyses

This paper presents an overview of the correlation of helicopter rotor performance and loads data from various tests and analyses. Information is included from U.S. Army‐sponsored tests conducted by Bell Helicopter Company for free‐flight full‐scale tests in the NASA‐Ames 40 × 80 wind tunnel, one‐fifth scale tests in the NASA‐Langley Transonic Dynamics Tunnel, and small‐scale tests of a rotor in air. These test data are compared with each other, where appropriate, and with calculated results. Typical examples illustrate the state of the art for correlation and indicate anomalies encountered. It is concluded that a procedure using theoretical analyses to aid in interpretation and evaluation of test results is essential to developing a science of correlation.

Compact dictionary pair learning and refining based on principal components analysis

2019 ◽  
Vol 17 (05) ◽  
pp. 1950033
Author(s):  
Yibin Yu ◽  
Min Yang ◽  
Yulan Zhang ◽  
Shifang Yuan
Keyword(s):  
Principal Components Analysis ◽  
Principal Components ◽  
State Of The Art ◽  
Great Success ◽  
Computation Complexity ◽  
Training Stage ◽  
Learned Dictionary ◽  
Sample Data ◽  
Components Analysis ◽  
Row Space

Although traditional dictionary learning (DL) methods have made great success in pattern recognition and machine learning, it is extremely time-consuming, especially in the training stage. The projective dictionary pair learning (DPL) learned the synthesis dictionary and the analysis dictionary jointly to achieve a fast and accurate classifier. However, the dictionary pair is initialized as random matrices without using any data samples information, it required many iterations to ensure convergence. In this paper, we propose a novel compact DPL and refining method based on the observation that the eigenvalue curve of sample data covariance matrix usually decrease very fast, which means we can compact the synthesis dictionary and analysis dictionary. For each class of the data samples, we utilize the principal components analysis (PCA) to retain global important information and compact the row space of a synthesis dictionary and the column space of an analysis dictionary in the first stage. We further refine the learned dictionary pair to achieve a more accurate classifier during compact dictionary pair refining, which combines the orthogonality of PCA with the redundancy of DL. We solve this refining problem in closed-form completely, naturally reducing the computation complexity significantly. Experimental results on the Extended YaleB database and AR database show that the proposed method achieves competitive accuracy and low computational complexity compared with other state-of-the-art methods.

scholarly journals Advanced Electric Vehicle Fast-Charging Technologies

Energies ◽  
2019 ◽  
Vol 12 (10) ◽  
pp. 1839 ◽  
Author(s):  
Ryan Collin ◽  
Yu Miao ◽  
Alex Yokochi ◽  
Prasad Enjeti ◽  
Annette von Jouanne
Keyword(s):  
Electric Vehicle ◽  
State Of The Art ◽  
Power Electronic ◽  
Heat Management ◽  
Battery Charging ◽  
Fast Charging ◽  
Battery Systems ◽  
Converter Topologies ◽  
Negative Impacts ◽  
Power Electronic Converter

Negative impacts from the dominant use of petroleum-based transportation have propelled the globe towards electrified transportation. With this thrust, many technological challenges are being encountered and addressed, one of which is the development and availability of fast-charging technologies. To compete with petroleum-based transportation, electric vehicle (EV) battery charging times need to decrease to the 5–10 min range. This paper provides a review of EV fast-charging technologies and the impacts on the battery systems, including heat management and associated limitations. In addition, the paper presents promising new approaches and opportunities for power electronic converter topologies and systems level research to advance the state-of-the-art in fast-charging.

scholarly journals Simple primitive recognition via hierarchical face clustering

2020 ◽  
Vol 6 (4) ◽  
pp. 431-443
Author(s):  
Xiaolong Yang ◽  
Xiaohong Jia
Keyword(s):  
Hierarchical Clustering ◽  
Test Data ◽  
Efficient Algorithm ◽  
Clustering Algorithm ◽  
State Of The Art ◽  
Experimental Results ◽  
Face Clustering ◽  
Wide Range ◽  
Hierarchical Clustering Algorithm ◽  
Bottom To Top

AbstractWe present a simple yet efficient algorithm for recognizing simple quadric primitives (plane, sphere, cylinder, cone) from triangular meshes. Our approach is an improved version of a previous hierarchical clustering algorithm, which performs pairwise clustering of triangle patches from bottom to top. The key contributions of our approach include a strategy for priority and fidelity consideration of the detected primitives, and a scheme for boundary smoothness between adjacent clusters. Experimental results demonstrate that the proposed method produces qualitatively and quantitatively better results than representative state-of-the-art methods on a wide range of test data.

Sign in / Sign up

Export Citation Format

Share Document