scholarly journals Layered Feature Representation for Differentiable Architecture Search

2021 ◽  
Author(s):  
Jie Hao ◽  
William Zhu
Keyword(s):  
Search Costs ◽  
Feature Representation ◽  
Adaptive Network ◽  
Feature Representations ◽  
Neural Architecture ◽  
Target Network ◽  
Deep Layers ◽  
A Cell ◽  
Different Depths ◽  
Multiple Cells

Abstract Differentiable architecture search (DARTS) approach has made great progress in reducing the com- putational costs of neural architecture search. DARTS tries to discover an optimal architecture module called cell from a predefined super network. However, the obtained cell is then repeatedly and simply stacked to build a target network, failing to extract layered fea- tures hidden in different network depths. Therefore, this target network cannot meet the requirements of prac- tical applications. To address this problem, we propose an effective approach called Layered Feature Repre- sentation for Differentiable Architecture Search (LFR- DARTS). Specifically, we iteratively search for multiple cells with different architectures from shallow to deep layers of the super network. For each iteration, we optimize the architecture of a cell by gradient descent and prune out weak connections from this cell. After obtain- ing the optimal architecture of this cell, we deepen the super network by increasing the number of this cell, so as to create an adaptive network context to search for a deeper-adaptive cell in the next iteration. Thus, our LFR-DARTS can discover the architecture of each cell at a specific and adaptive network depth, which embeds the ability of layered feature representations into each cell to sufficiently extract layered features in different depths. Extensive experiments show that our algorithm achieves an advanced performance on the datasets of CIFAR10, fashionMNIST and ImageNet while at low search costs.

scholarly journals Meta Learning for Few-Shot One-Class Classification

AI ◽  
2021 ◽  
Vol 2 (2) ◽  
pp. 195-208
Author(s):  
Gabriel Dahia ◽  
Maurício Pamplona Segundo
Keyword(s):  
Feature Representation ◽  
Support Vector ◽  
Support Vector Data Description ◽  
Target Class ◽  
Feature Representations ◽  
Shot Classification ◽  
Training Stage ◽  
Comparable Performance ◽  
Meta Learning ◽  
One Class Classification

We propose a method that can perform one-class classification given only a small number of examples from the target class and none from the others. We formulate the learning of meaningful features for one-class classification as a meta-learning problem in which the meta-training stage repeatedly simulates one-class classification, using the classification loss of the chosen algorithm to learn a feature representation. To learn these representations, we require only multiclass data from similar tasks. We show how the Support Vector Data Description method can be used with our method, and also propose a simpler variant based on Prototypical Networks that obtains comparable performance, indicating that learning feature representations directly from data may be more important than which one-class algorithm we choose. We validate our approach by adapting few-shot classification datasets to the few-shot one-class classification scenario, obtaining similar results to the state-of-the-art of traditional one-class classification, and that improves upon that of one-class classification baselines employed in the few-shot setting.

A Comprehensive Survey of Neural Architecture Search

2021 ◽  
Vol 54 (4) ◽  
pp. 1-34
Author(s):  
Pengzhen Ren ◽  
Yun Xiao ◽  
Xiaojun Chang ◽  
Po-yao Huang ◽  
Zhihui Li ◽  
...  
Keyword(s):  
Research Work ◽  
Search Space ◽  
Feature Representation ◽  
Future Research ◽  
Related Research ◽  
Neural Architecture ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Intuitive Idea ◽  
New Perspective

Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers’ prior knowledge and experience. And due to the limitations of humans’ inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search ( NAS ) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.

scholarly journals Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

2019 ◽  
Vol 33 ◽  
pp. 8594-8601 ◽  
Author(s):  
Guanbin Li ◽  
Xin Zhu ◽  
Yirui Zeng ◽  
Qing Wang ◽  
Liang Lin
Keyword(s):  
Neural Network ◽  
Facial Expressions ◽  
Mutual Exclusion ◽  
Representation Learning ◽  
Feature Representation ◽  
Semantic Relationship ◽  
Action Unit ◽  
Facial Action ◽  
Feature Representations ◽  
Regional Feature

Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.

scholarly journals A Novel Point Cloud Encoding Method Based on Local Information for 3D Classification and Segmentation

Sensors ◽  
2020 ◽  
Vol 20 (9) ◽  
pp. 2501 ◽  
Author(s):  
Yanan Song ◽  
Liang Gao ◽  
Xinyu Li ◽  
Weiming Shen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Local Information ◽  
Local Region ◽  
Feature Representation ◽  
Learning Network ◽  
Feature Representations ◽  
Encoding Method ◽  
Deep Learning Network

Deep learning is robust to the perturbation of a point cloud, which is an important data form in the Internet of Things. However, it cannot effectively capture the local information of the point cloud and recognize the fine-grained features of an object. Different levels of features in the deep learning network are integrated to obtain local information, but this strategy increases network complexity. This paper proposes an effective point cloud encoding method that facilitates the deep learning network to utilize the local information. An axis-aligned cube is used to search for a local region that represents the local information. All of the points in the local region are available to construct the feature representation of each point. These feature representations are then input to a deep learning network. Two well-known datasets, ModelNet40 shape classification benchmark and Stanford 3D Indoor Semantics Dataset, are used to test the performance of the proposed method. Compared with other methods with complicated structures, the proposed method with only a simple deep learning network, can achieve a higher accuracy in 3D object classification and semantic segmentation.

scholarly journals Deep Learning for Generic Object Detection: A Survey

2019 ◽  
Vol 128 (2) ◽  
pp. 261-318 ◽  
Author(s):  
Li Liu ◽  
Wanli Ouyang ◽  
Xiaogang Wang ◽  
Paul Fieguth ◽  
Jie Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Rapid Evolution ◽  
Feature Representation ◽  
Future Research ◽  
Context Modeling ◽  
Feature Representations ◽  
Learning Techniques ◽  
Comprehensive Survey ◽  
Object Feature

Abstract Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

scholarly journals A Systematic Evaluation of Interneuron Morphology Representations for Cell Type Discrimination

2020 ◽  
Vol 18 (4) ◽  
pp. 591-609 ◽  
Author(s):  
Sophie Laturnus ◽  
Dmitry Kobak ◽  
Philipp Berens
Keyword(s):  
Cell Types ◽  
Bipolar Cells ◽  
Machine Learning Algorithms ◽  
Feature Representation ◽  
Inhibitory Neurons ◽  
Systematic Evaluation ◽  
Two Dimensional ◽  
Feature Representations ◽  
Density Maps ◽  
The Difference

Abstract Quantitative analysis of neuronal morphologies usually begins with choosing a particular feature representation in order to make individual morphologies amenable to standard statistics tools and machine learning algorithms. Many different feature representations have been suggested in the literature, ranging from density maps to intersection profiles, but they have never been compared side by side. Here we performed a systematic comparison of various representations, measuring how well they were able to capture the difference between known morphological cell types. For our benchmarking effort, we used several curated data sets consisting of mouse retinal bipolar cells and cortical inhibitory neurons. We found that the best performing feature representations were two-dimensional density maps, two-dimensional persistence images and morphometric statistics, which continued to perform well even when neurons were only partially traced. Combining these feature representations together led to further performance increases suggesting that they captured non-redundant information. The same representations performed well in an unsupervised setting, implying that they can be suitable for dimensionality reduction or clustering.

scholarly journals High-Resolution Neural Network for Driver Visual Attention Prediction

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2030 ◽  
Author(s):  
Byeongkeun Kang ◽  
Yeejin Lee
Keyword(s):  
Neural Network ◽  
High Resolution ◽  
Visual Attention ◽  
Visual Information ◽  
Prediction Accuracy ◽  
Critical Role ◽  
Image Resolution ◽  
Feature Representation ◽  
Safe Driving ◽  
Feature Representations

Driving is a task that puts heavy demands on visual information, thereby the human visual system plays a critical role in making proper decisions for safe driving. Understanding a driver’s visual attention and relevant behavior information is a challenging but essential task in advanced driver-assistance systems (ADAS) and efficient autonomous vehicles (AV). Specifically, robust prediction of a driver’s attention from images could be a crucial key to assist intelligent vehicle systems where a self-driving car is required to move safely interacting with the surrounding environment. Thus, in this paper, we investigate a human driver’s visual behavior in terms of computer vision to estimate the driver’s attention locations in images. First, we show that feature representations at high resolution improves visual attention prediction accuracy and localization performance when being fused with features at low-resolution. To demonstrate this, we employ a deep convolutional neural network framework that learns and extracts feature representations at multiple resolutions. In particular, the network maintains the feature representation with the highest resolution at the original image resolution. Second, attention prediction tends to be biased toward centers of images when neural networks are trained using typical visual attention datasets. To avoid overfitting to the center-biased solution, the network is trained using diverse regions of images. Finally, the experimental results verify that our proposed framework improves the prediction accuracy of a driver’s attention locations.

scholarly journals Predicting Cross-Species Infection of Swine Influenza Virus with Representation Learning of Amino Acid Features

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Zheng Kou ◽  
Junjie Li ◽  
Xinyue Fan ◽  
Saeed Kosari ◽  
Xiaoli Qiang
Keyword(s):  
Amino Acid ◽  
High Performance ◽  
Swine Influenza Virus ◽  
Swine Influenza ◽  
Representation Learning ◽  
Influenza Viruses ◽  
Feature Representation ◽  
Specific Information ◽  
Feature Representations ◽  
Probabilistic Information

Swine influenza viruses (SIVs) can unforeseeably cross the species barriers and directly infect humans, which pose huge challenges for public health and trigger pandemic risk at irregular intervals. Computational tools are needed to predict infection phenotype and early pandemic risk of SIVs. For this purpose, we propose a feature representation algorithm to predict cross-species infection of SIVs. We built a high-quality dataset of 1902 viruses. A feature representation learning scheme was applied to learn feature representations from 64 well-trained random forest models with multiple feature descriptors of mutant amino acid in the viral proteins, including compositional information, position-specific information, and physicochemical properties. Class and probabilistic information were integrated into the feature representations, and redundant features were removed by feature space optimization. High performance was achieved using 20 informative features and 22 probabilistic information. The proposed method will facilitate SIV characterization of transmission phenotype.

scholarly journals Assessing Regression-Based Sentiment Analysis Techniques in Financial Texts

2019 ◽  
Author(s):  
Taynan Ferreira ◽  
Francisco Paiva ◽  
Roberto Silva ◽  
Angel Paula ◽  
Anna Costa ◽  
...  
Keyword(s):  
Sentiment Analysis ◽  
Feature Representation ◽  
Support Vector ◽  
Data Set ◽  
Feature Representations ◽  
Textual Data ◽  
Enormous Amount ◽  
Financial Domain ◽  
Classification Tasks ◽  
The Impact

Sentiment analysis (SA) is increasing its importance due to the enormous amount of opinionated textual data available today. Most of the researches have investigated different models, feature representation and hyperparameters in SA classification tasks. However, few studies were conducted to evaluate the impact of these features on regression SA tasks. In this paper, we conduct such assessment on a financial domain data set by investigating different feature representations and hyperparameters in two important models -- Support Vector Regression (SVR) and Convolution Neural Networks (CNN). We conclude presenting the most relevant feature representations and hyperparameters and how they impact outcomes on a regression SA task.

scholarly journals Large-Scale Whale-Call Classification by Transfer Learning on Multi-Scale Waveforms and Time-Frequency Features

2019 ◽  
Vol 9 (5) ◽  
pp. 1020 ◽  
Author(s):  
Lilun Zhang ◽  
Dezhi Wang ◽  
Changchun Bao ◽  
Yongxian Wang ◽  
Kele Xu
Keyword(s):  
Transfer Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Feature Representation ◽  
Biological Research ◽  
Time Frequency ◽  
Feature Representations ◽  
Multi Scale ◽  
Data Driven Approach

Whale vocal calls contain valuable information and abundant characteristics that are important for classification of whale sub-populations and related biological research. In this study, an effective data-driven approach based on pre-trained Convolutional Neural Networks (CNN) using multi-scale waveforms and time-frequency feature representations is developed in order to perform the classification of whale calls from a large open-source dataset recorded by sensors carried by whales. Specifically, the classification is carried out through a transfer learning approach by using pre-trained state-of-the-art CNN models in the field of computer vision. 1D raw waveforms and 2D log-mel features of the whale-call data are respectively used as the input of CNN models. For raw waveform input, windows are applied to capture multiple sketches of a whale-call clip at different time scales and stack the features from different sketches for classification. When using the log-mel features, the delta and delta-delta features are also calculated to produce a 3-channel feature representation for analysis. In the training, a 4-fold cross-validation technique is employed to reduce the overfitting effect, while the Mix-up technique is also applied to implement data augmentation in order to further improve the system performance. The results show that the proposed method can improve the accuracies by more than 20% in percentage for the classification into 16 whale pods compared with the baseline method using groups of 2D shape descriptors of spectrograms and the Fisher discriminant scores on the same dataset. Moreover, it is shown that classifications based on log-mel features have higher accuracies than those based directly on raw waveforms. The phylogeny graph is also produced to significantly illustrate the relationships among the whale sub-populations.

Sign in / Sign up

Export Citation Format

Share Document