scholarly journals Self-supervised Learning for Reading Activity Classification

2021 ◽  
Vol 5 (3) ◽  
pp. 1-22
Author(s):  
Md. Rabiul Islam ◽  
Shuji Sakamoto ◽  
Yoshihiro Yamada ◽  
Andrew W. Vargo ◽  
Motoi Iwata ◽  
...  
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Training Data ◽  
Classification Task ◽  
Support Vector ◽  
Multiple Choice Questions ◽  
Confidence Estimation ◽  
Vector Machines ◽  
Automatic Reading ◽  
Reading Activity

Reading analysis can relay information about user's confidence and habits and can be used to construct useful feedback. A lack of labeled data inhibits the effective application of fully-supervised Deep Learning (DL) for automatic reading analysis. We propose a Self-supervised Learning (SSL) method for reading analysis. Previously, SSL has been effective in physical human activity recognition (HAR) tasks, but it has not been applied to cognitive HAR tasks like reading. We first evaluate the proposed method on a four-class classification task on reading detection using electrooculography datasets, followed by an evaluation of a two-class classification task of confidence estimation on multiple-choice questions using eye-tracking datasets. Fully-supervised DL and support vector machines (SVMs) are used as comparisons for the proposed SSL method. The results show that the proposed SSL method is superior to the fully-supervised DL and SVM for both tasks, especially when training data is scarce. This result indicates the proposed method is the superior choice for reading analysis tasks. These results are important for informing the design of automatic reading analysis platforms.

Employing different loss functions for the classification of images via supervised learning

2014 ◽  
Vol 12 (2) ◽  
Author(s):  
Radu Boţ ◽  
André Heinrich ◽  
Gert Wanka
Keyword(s):  
Supervised Learning ◽  
Strong Duality ◽  
Training Data ◽  
Loss Functions ◽  
Classification Task ◽  
Support Vector ◽  
Real World Data ◽  
Vector Machines ◽  
Regularization Problem ◽  
Classification Of Images

AbstractSupervised learning methods are powerful techniques to learn a function from a given set of labeled data, the so-called training data. In this paper the support vector machines approach is applied to an image classification task. Starting with the corresponding Tikhonov regularization problem, reformulated as a convex optimization problem, we introduce a conjugate dual problem to it and prove that, whenever strong duality holds, the function to be learned can be expressed via the dual optimal solutions. Corresponding dual problems are then derived for different loss functions. The theoretical results are applied by numerically solving a classification task using high dimensional real-world data in order to obtain optimal classifiers. The results demonstrate the excellent performance of support vector classification for this particular problem.

scholarly journals Identifying indicator species in ecological habitats using Deep Optimal Feature Learning

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0256782
Author(s):  
Yiting Tsai ◽  
Susan A. Baldwin ◽  
Bhushan Gopaluni
Keyword(s):  
Deep Learning ◽  
Indicator Species ◽  
Prediction Accuracy ◽  
Species Abundance ◽  
Keystone Species ◽  
Feature Learning ◽  
Training Data ◽  
Support Vector ◽  
Vector Machines ◽  
Domain Expertise

Much of the current research on supervised modelling is focused on maximizing outcome prediction accuracy. However, in engineering disciplines, an arguably more important goal is that of feature extraction, the identification of relevant features associated with the various outcomes. For instance, in microbial communities, the identification of keystone species can often lead to improved prediction of future behavioral shifts. This paper proposes a novel feature extractor based on Deep Learning, which is largely agnostic to underlying assumptions regarding the training data. Starting from a collection of microbial species abundance counts, the Deep Learning model first trains itself to classify the selected distinct habitats. It then identifies indicator species associated with the habitats. The results are then compared and contrasted with those obtained by traditional statistical techniques. The indicator species are similar when compared at top taxonomic levels such as Domain and Phylum, despite visible differences in lower levels such as Class and Order. More importantly, when our estimated indicators are used to predict final habitat labels using simpler models (such as Support Vector Machines and traditional Artificial Neural Networks), the prediction accuracy is improved. Overall, this study serves as a preliminary step that bridges modern, black-box Machine Learning models with traditional, domain expertise-rich techniques.

scholarly journals A Neural Basis for the Implementation of Deep Learning and Arti cial Intelligence

2018 ◽  
Vol 7 (4.36) ◽  
pp. 444 ◽  
Author(s):  
Alan F. Smeaton ◽  
. .
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Training Data ◽  
Computer Architectures ◽  
Support Vector ◽  
Neural Basis ◽  
Mathematical Basis ◽  
Unseen Data ◽  
Vector Machines ◽  
Conventional Computer

One of the mathematical cornerstones of modern data ana-lytics is machine learning whereby we automatically learn subtle patterns which may be hidden in training data, we associate those patterns with outcomes and we apply these patterns to new and unseen data and make predictions about as yet unseen outcomes. This form of data analytics al-lows us to bring value to the huge volumes of data that is collected from people, from the environment, from commerce, from online activities, from scienti c experiments, from many other sources. The mathematical basis for this form of machine learning has led to tools like Support Vector Machines which have shown moderate e ectiveness and good e ciency in their implementation. Recently, however, these have been usurped by the emergence of deep learning based on convolutional neural networks. In this presentation we will examine the basis for why such deep net-works are remarkably successful and accurate, their similarity to ways in which the human brain is organised, and the challenges of implementing such deep networks on conventional computer architectures.  

scholarly journals Laplacian networks: bounding indicator function smoothness for neural networks robustness

2021 ◽  
Vol 10 ◽  
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Supervised Learning ◽  
Indicator Function ◽  
Training Data ◽  
Theoretical Justification ◽  
The Past ◽  
Noisy Examples

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

Deep Learning Representation from Electroencephalography of Early-Stage Creutzfeldt-Jakob Disease and Features for Differentiation from Rapidly Progressive Dementia

2016 ◽  
Vol 27 (02) ◽  
pp. 1650039 ◽  
Author(s):  
Francesco Carlo Morabito ◽  
Maurizio Campolo ◽  
Nadia Mammone ◽  
Mario Versaci ◽  
Silvana Franceschetti ◽  
...  
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Early Stage ◽  
Processing System ◽  
Fine Tuning ◽  
Permutation Entropy ◽  
Support Vector ◽  
Progressive Dementia ◽  
Time Frequency ◽  
Jakob Disease

A novel technique of quantitative EEG for differentiating patients with early-stage Creutzfeldt–Jakob disease (CJD) from other forms of rapidly progressive dementia (RPD) is proposed. The discrimination is based on the extraction of suitable features from the time-frequency representation of the EEG signals through continuous wavelet transform (CWT). An average measure of complexity of the EEG signal obtained by permutation entropy (PE) is also included. The dimensionality of the feature space is reduced through a multilayer processing system based on the recently emerged deep learning (DL) concept. The DL processor includes a stacked auto-encoder, trained by unsupervised learning techniques, and a classifier whose parameters are determined in a supervised way by associating the known category labels to the reduced vector of high-level features generated by the previous processing blocks. The supervised learning step is carried out by using either support vector machines (SVM) or multilayer neural networks (MLP-NN). A subset of EEG from patients suffering from Alzheimer’s Disease (AD) and healthy controls (HC) is considered for differentiating CJD patients. When fine-tuning the parameters of the global processing system by a supervised learning procedure, the proposed system is able to achieve an average accuracy of 89%, an average sensitivity of 92%, and an average specificity of 89% in differentiating CJD from RPD. Similar results are obtained for CJD versus AD and CJD versus HC.

scholarly journals Evaluating Grayware Characteristics and Risks

2011 ◽  
Vol 2011 ◽  
pp. 1-28 ◽  
Author(s):  
Zhongqiang Chen ◽  
Zhanyan Liang ◽  
Yuan Zhang ◽  
Zhongrong Chen
Keyword(s):  
Information Gain ◽  
Feature Space ◽  
Training Data ◽  
Support Vector ◽  
Learning Models ◽  
Generalization Capability ◽  
Self Organizing Maps ◽  
Defense Strategies ◽  
Security Applications ◽  
Vector Machines

Grayware encyclopedias collect known species to provide information for incident analysis, however, the lack of categorization and generalization capability renders them ineffective in the development of defense strategies against clustered strains. A grayware categorization framework is therefore proposed here to not only classify grayware according to diverse taxonomic features but also facilitate evaluations on grayware risk to cyberspace. Armed with Support Vector Machines, the framework builds learning models based on training data extracted automatically from grayware encyclopedias and visualizes categorization results with Self-Organizing Maps. The features used in learning models are selected with information gain and the high dimensionality of feature space is reduced by word stemming and stopword removal process. The grayware categorizations on diversified features reveal that grayware typically attempts to improve its penetration rate by resorting to multiple installation mechanisms and reduced code footprints. The framework also shows that grayware evades detection by attacking victims' security applications and resists being removed by enhancing its clotting capability with infected hosts. Our analysis further points out that species in categoriesSpywareandAdwarecontinue to dominate the grayware landscape and impose extremely critical threats to the Internet ecosystem.

scholarly journals State of the Art Survey of Deep Learning and Machine Learning Models for Smart Cities and Urban Sustainability

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Ramin Keivani ◽  
Sina Faizollahzadeh Ardabili ◽  
Farshid Aram
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
State Of The Art ◽  
Smart Cities ◽  
Model Development ◽  
Urban Sustainability ◽  
Urban Transport ◽  
Support Vector ◽  
Neuro Fuzzy ◽  
Vector Machines

Deep learning (DL) and machine learning (ML) methods have recently contributed to the advancement of models in the various aspects of prediction, planning, and uncertainty analysis of smart cities and urban development. This paper presents the state of the art of DL and ML methods used in this realm. Through a novel taxonomy, the advances in model development and new application domains in urban sustainability and smart cities are presented. Findings reveal that five DL and ML methods have been most applied to address the different aspects of smart cities. These are artificial neural networks; support vector machines; decision trees; ensembles, Bayesians, hybrids, and neuro-fuzzy; and deep learning. It is also disclosed that energy, health, and urban transport are the main domains of smart cities that DL and ML methods contributed in to address their problems.

scholarly journals Optimization of RBF-SVM hyperparameters using genetic algorithm for face recognit

2021 ◽  
Vol 39 (4) ◽  
pp. 1190-1197
Author(s):  
Y. Ibrahim ◽  
E. Okafor ◽  
B. Yahaya
Keyword(s):  
Genetic Algorithm ◽  
Deep Learning ◽  
Face Recognition ◽  
Regularization Parameter ◽  
Local Binary Patterns ◽  
Classification Performance ◽  
Support Vector ◽  
Vector Machines ◽  
Linear Svm ◽  
Learning Architectures

Manual grid-search tuning of machine learning hyperparameters is very time-consuming. Hence, to curb this problem, we propose the use of a genetic algorithm (GA) for the selection of optimal radial-basis-function based support vector machine (RBF-SVM) hyperparameters; regularization parameter C and cost-factor γ. The resulting optimal parameters were used during the training of face recognition models. To train the models, we independently extracted features from the ORL face image dataset using local binary patterns (handcrafted) and deep learning architectures (pretrained variants of VGGNet). The resulting features were passed as input to either linear-SVM or optimized RBF-SVM. The results show that the models from optimized RBFSVM combined with deep learning or hand-crafted features yielded performances that surpass models obtained from Linear-SVM combined with the aforementioned features in most of the data splits. The study demonstrated that it is profitable to optimize the hyperparameters of an SVM to obtain the best classification performance. Keywords: Face Recognition, Feature Extraction, Local Binary Patterns, Transfer Learning, Genetic Algorithm and Support Vector  Machines.

Sign in / Sign up

Export Citation Format

Share Document