scholarly journals Optimality of Spectrum Pursuit for Column Subset Selection Problem: Theoretical Guarantees and Applications in Deep Learning

2020 ◽  
Author(s):  
Mohsen Joneidi ◽  
Saeed Vahidian ◽  
Ashkan Esmaeili ◽  
Siavash Khodadadeh
Keyword(s):  
Deep Learning ◽  
Upper Bound ◽  
Linear Complexity ◽  
Subset Selection ◽  
Selection Problem ◽  
Theoretical Methods ◽  
Novel Technique ◽  
Original Dataset ◽  
Column Subset Selection ◽  
Minimum Number

We propose a novel technique for finding representatives from a large, unsupervised dataset. The approach is based on the concept of self-rank, defined as the minimum number of samples needed to reconstruct all samples with an accuracy proportional to the rank-$K$ approximation. Our proposed algorithm enjoys linear complexity w.r.t. the size of original dataset and simultaneously it provides an adaptive upper bound for approximation ratio. These favorable characteristics result in filling a historical gap between practical and theoretical methods in finding representatives.<br>

scholarly journals Optimality of Spectrum Pursuit for Column Subset Selection Problem: Theoretical Guarantees and Applications in Deep Learning

2020 ◽  
Author(s):  
Mohsen Joneidi ◽  
Saeed Vahidian ◽  
Ashkan Esmaeili ◽  
Siavash Khodadadeh
Keyword(s):  
Deep Learning ◽  
Upper Bound ◽  
Linear Complexity ◽  
Subset Selection ◽  
Selection Problem ◽  
Theoretical Methods ◽  
Novel Technique ◽  
Original Dataset ◽  
Column Subset Selection ◽  
Minimum Number

We propose a novel technique for finding representatives from a large, unsupervised dataset. The approach is based on the concept of self-rank, defined as the minimum number of samples needed to reconstruct all samples with an accuracy proportional to the rank-$K$ approximation. Our proposed algorithm enjoys linear complexity w.r.t. the size of original dataset and simultaneously it provides an adaptive upper bound for approximation ratio. These favorable characteristics result in filling a historical gap between practical and theoretical methods in finding representatives.<br>

Select to better learn: Fast and accurate deep learning using data selection from nonlinear manifolds

2020 ◽  
Author(s):  
Mohsen Joneidi ◽  
Saeed Vahidian ◽  
Ashkan Esmaeili ◽  
Weijia Wang ◽  
Nazanin Rahnavard ◽  
...  
Keyword(s):  
Deep Learning ◽  
Small Subset ◽  
Original Dataset ◽  
Wide Range ◽  
Spectral Components ◽  
Column Subset Selection ◽  
Important Open Problem ◽  
Data Points ◽  
Using Data ◽  
Nonlinear Manifolds

Finding a small subset of data whose linear combination spans other data points, also called column subset selection problem (CSSP), is an important open problem in computer science with many applications in computer vision and deep learning. There are some studies that solve CSSP in a polynomial time complexity w.r.t. the size of the original dataset. A simple and efficient selection algorithm with a linear complexity order, referred to as spectrum pursuit (SP), is proposed that pursuits spectral components of the dataset using available sample points. The proposed non-greedy algorithm aims to iteratively find K data samples whose span is close to that of the first K spectral components of entire data. SP has no parameter to be fine tuned and this desirable property makes it problem-independent. The simplicity of SP enables us to extend the underlying linear model to more complex models such as nonlinear manifolds and graph-based models. The nonlinear extension of SP is introduced as kernel-SP (KSP). The superiority of the proposed algorithms is demonstrated in a wide range of applications.

scholarly journals Select to better learn: Fast and accurate deep learning using data selection from nonlinear manifolds

2020 ◽  
Author(s):  
Mohsen Joneidi ◽  
Saeed Vahidian ◽  
Ashkan Esmaeili ◽  
Weijia Wang ◽  
Nazanin Rahnavard ◽  
...  
Keyword(s):  
Deep Learning ◽  
Small Subset ◽  
Original Dataset ◽  
Wide Range ◽  
Spectral Components ◽  
Column Subset Selection ◽  
Important Open Problem ◽  
Data Points ◽  
Using Data ◽  
Nonlinear Manifolds

Finding a small subset of data whose linear combination spans other data points, also called column subset selection problem (CSSP), is an important open problem in computer science with many applications in computer vision and deep learning. There are some studies that solve CSSP in a polynomial time complexity w.r.t. the size of the original dataset. A simple and efficient selection algorithm with a linear complexity order, referred to as spectrum pursuit (SP), is proposed that pursuits spectral components of the dataset using available sample points. The proposed non-greedy algorithm aims to iteratively find K data samples whose span is close to that of the first K spectral components of entire data. SP has no parameter to be fine tuned and this desirable property makes it problem-independent. The simplicity of SP enables us to extend the underlying linear model to more complex models such as nonlinear manifolds and graph-based models. The nonlinear extension of SP is introduced as kernel-SP (KSP). The superiority of the proposed algorithms is demonstrated in a wide range of applications.

scholarly journals Structured (De)composable Representations Trained with Neural Networks

Computers ◽  
2020 ◽  
Vol 9 (4) ◽  
pp. 79
Author(s):  
Graham Spinks ◽  
Marie-Francine Moens
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Contextual Information ◽  
Learning To Learn ◽  
Class Label ◽  
Impact Performance ◽  
Novel Technique ◽  
Language Data ◽  
Concept Classes ◽  
Generic Representation

This paper proposes a novel technique for representing templates and instances of concept classes. A template representation refers to the generic representation that captures the characteristics of an entire class. The proposed technique uses end-to-end deep learning to learn structured and composable representations from input images and discrete labels. The obtained representations are based on distance estimates between the distributions given by the class label and those given by contextual information, which are modeled as environments. We prove that the representations have a clear structure allowing decomposing the representation into factors that represent classes and environments. We evaluate our novel technique on classification and retrieval tasks involving different modalities (visual and language data). In various experiments, we show how the representations can be compressed and how different hyperparameters impact performance.

scholarly journals Evolving Deep Learning Convolutional Neural Networks for Early COVID-19 Detection in Chest X-ray Images

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 1002
Author(s):  
Mohammad Khishe ◽  
Fabio Caraffini ◽  
Stefan Kuhn
Keyword(s):  
Deep Learning ◽  
Early Detection ◽  
Iterative Process ◽  
High Accuracy ◽  
X Ray ◽  
Starting Point ◽  
Reliable Model ◽  
Minimum Number ◽  
Chest X Ray ◽  
Deep Learning Model

This article proposes a framework that automatically designs classifiers for the early detection of COVID-19 from chest X-ray images. To do this, our approach repeatedly makes use of a heuristic for optimisation to efficiently find the best combination of the hyperparameters of a convolutional deep learning model. The framework starts with optimising a basic convolutional neural network which represents the starting point for the evolution process. Subsequently, at most two additional convolutional layers are added, at a time, to the previous convolutional structure as a result of a further optimisation phase. Each performed phase maximises the the accuracy of the system, thus requiring training and assessment of the new model, which gets gradually deeper, with relevant COVID-19 chest X-ray images. This iterative process ends when no improvement, in terms of accuracy, is recorded. Hence, the proposed method evolves the most performing network with the minimum number of convolutional layers. In this light, we simultaneously achieve high accuracy while minimising the presence of redundant layers to guarantee a fast but reliable model. Our results show that the proposed implementation of such a framework achieves accuracy up to 99.11%, thus being particularly suitable for the early detection of COVID-19.

Sign in / Sign up

Export Citation Format

Share Document