The Effects of Feature Optimization on High-Dimensional Essay Data

Current machine learning (ML) based automated essay scoring (AES) systems have employed various and vast numbers of features, which have been proven to be useful, in improving the performance of the AES. However, the high-dimensional feature space is not properly represented, due to the large volume of features extracted from the limited training data. As a result, this problem gives rise to poor performance and increased training time for the system. In this paper, we experiment and analyze the effects of feature optimization, including normalization, discretization, and feature selection techniques for different ML algorithms, while taking into consideration the size of the feature space and the performance of the AES. Accordingly, we show that the appropriate feature optimization techniques can reduce the dimensions of features, thus, contributing to the efficient training and performance improvement of AES.

Download Full-text

Deep Ensemble Reinforcement Learning with Multiple Deep Deterministic Policy Gradient Algorithm

Mathematical Problems in Engineering ◽

10.1155/2020/4275623 ◽

2020 ◽

Vol 2020 ◽

pp. 1-12 ◽

Cited By ~ 1

Author(s):

Junta Wu ◽

Huiyun Li

Keyword(s):

Reinforcement Learning ◽

Autonomous Vehicles ◽

Training Data ◽

Gradient Algorithm ◽

Distributed Data ◽

Robot Arm ◽

Training Time ◽

Continuous Space ◽

Policy Gradient ◽

And Performance

Deep deterministic policy gradient algorithm operating over continuous space of actions has attracted great attention for reinforcement learning. However, the exploration strategy through dynamic programming within the Bayesian belief state space is rather inefficient even for simple systems. Another problem is the sequential and iterative training data with autonomous vehicles subject to the law of causality, which is against the i.i.d. (independent identically distributed) data assumption of the training samples. This usually results in failure of the standard bootstrap when learning an optimal policy. In this paper, we propose a framework of m-out-of-n bootstrapped and aggregated multiple deep deterministic policy gradient to accelerate the training process and increase the performance. Experiment results on the 2D robot arm game show that the reward gained by the aggregated policy is 10%–50% better than those gained by subpolicies. Experiment results on the open racing car simulator (TORCS) demonstrate that the new algorithm can learn successful control policies with less training time by 56.7%. Analysis on convergence is also given from the perspective of probability and statistics. These results verify that the proposed method outperforms the existing algorithms in both efficiency and performance.

Download Full-text

An Introduction to Kernel Methods

Encyclopedia of Data Warehousing and Mining, Second Edition ◽

10.4018/978-1-60566-010-3.ch170 ◽

2011 ◽

pp. 1097-1101

Author(s):

Gustavo Camps-Valls ◽

Manel Martínez-Ramón ◽

José Luis Rojo-Álvarez

Keyword(s):

Kernel Methods ◽

Adaptive Systems ◽

A Priori ◽

Feature Space ◽

Training Data ◽

High Dimensional ◽

Data Sets ◽

Practical Performance ◽

Active Research ◽

Innovative Techniques

Machine learning has experienced a great advance in the eighties and nineties due to the active research in artificial neural networks and adaptive systems. These tools have demonstrated good results in many real applications, since neither a priori knowledge about the distribution of the available data nor the relationships among the independent variables should be necessarily assumed. Overfitting due to reduced training data sets is controlled by means of a regularized functional which minimizes the complexity of the machine. Working with high dimensional input spaces is no longer a problem thanks to the use of kernel methods. Such methods also provide us with new ways to interpret the classification or estimation results. Kernel methods are emerging and innovative techniques that are based on first mapping the data from the original input feature space to a kernel feature space of higher dimensionality, and then solving a linear problem in that space. These methods allow us to geometrically design (and interpret) learning algorithms in the kernel space (which is nonlinearly related to the input space), thus combining statistics and geometry in an effective way. This theoretical elegance is also matched by their practical performance.

Download Full-text

Fourier-transform-based attribution priors improve the interpretability and stability of deep learning models for genomics

10.1101/2020.06.11.147272 ◽

2020 ◽

Cited By ~ 2

Author(s):

Alex M. Tseng ◽

Avanti Shrikumar ◽

Anshul Kundaje

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Dna Sequences ◽

Regulatory Elements ◽

Training Data ◽

Sequence Motifs ◽

Learning Models ◽

Training Time ◽

And Performance ◽

The Fourier Transform

AbstractDeep learning models can accurately map genomic DNA sequences to associated functional molecular readouts such as protein–DNA binding data. Base-resolution importance (i.e. “attribution”) scores inferred from these models can highlight predictive sequence motifs and syntax. Unfortunately, these models are prone to overfitting and are sensitive to random initializations, often resulting in noisy and irreproducible attributions that obfuscate underlying motifs. To address these shortcomings, we propose a novel attribution prior, where the Fourier transform of input-level attribution scores are computed at training-time, and high-frequency components of the Fourier spectrum are penalized. We evaluate different model architectures with and without attribution priors trained on genome-wide binary or continuous molecular profiles. We show that our attribution prior dramatically improves models’ stability, interpretability, and performance on held-out data, especially when training data is severely limited. Our attribution prior also allows models to identify biologically meaningful sequence motifs more sensitively and precisely within individual regulatory elements. The prior is agnostic to the model architecture or predicted experimental assay, yet provides similar gains across all experiments. This work represents an important advancement in improving the reliability of deep learning models for deciphering the regulatory code of the genome.

Download Full-text

A Fast Logdet Divergence Based Metric Learning Algorithm for Large Data Sets Classification

Abstract and Applied Analysis ◽

10.1155/2014/463981 ◽

2014 ◽

Vol 2014 ◽

pp. 1-9 ◽

Cited By ~ 1

Author(s):

Jiangyuan Mei ◽

Jian Hou ◽

Jicheng Chen ◽

Hamid Reza Karimi

Keyword(s):

Learning Algorithm ◽

Metric Learning ◽

Large Data ◽

Feature Space ◽

Industrial Applications ◽

Large Data Sets ◽

Training Data ◽

High Dimensional ◽

Data Sets ◽

Benchmark Data

Large data sets classification is widely used in many industrial applications. It is a challenging task to classify large data sets efficiently, accurately, and robustly, as large data sets always contain numerous instances with high dimensional feature space. In order to deal with this problem, in this paper we present an online Logdet divergence based metric learning (LDML) model by making use of the powerfulness of metric learning. We firstly generate a Mahalanobis matrix via learning the training data with LDML model. Meanwhile, we propose a compressed representation for high dimensional Mahalanobis matrix to reduce the computation complexity in each iteration. The final Mahalanobis matrix obtained this way measures the distances between instances accurately and serves as the basis of classifiers, for example, thek-nearest neighbors classifier. Experiments on benchmark data sets demonstrate that the proposed algorithm compares favorably with the state-of-the-art methods.

Download Full-text

Optimization techniques on fuzzy inference systems to detect Xanthomonas campestris disease

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v11i4.pp3510-3518 ◽

2021 ◽

Vol 11 (4) ◽

pp. 3510

Author(s):

Julio Barón Velandia ◽

Camilo Enrique Rocha Calderón ◽

Daniel David Leal Lara

Keyword(s):

Xanthomonas Campestris ◽

Fuzzy Inference ◽

Model Performance ◽

Optimization Techniques ◽

Training Data ◽

Fuzzy Inference Systems ◽

The Status ◽

And Performance ◽

Inference Systems ◽

The Impact

This paper shows the outcomes for four optimization models based on fuzzy inference systems, intervened using Quasi-Newton and genetic algorithms, to early assess bean plants’ leaves for Xanthomonas campestris disease. The assessment on the status of the plant (sane or ill) is defined through the intensity of the color in the RGB scale for the data-sets and images to analyze the implementation of the models. The best model performance is 99.68% when compared with the training data and a 94% effectiveness rate on the detection of Xanthomonas campestris in a bean leave image. Therefore, these results would allow farmers to take early measures to reduce the impact of the disease on the look and performance of green bean crops.

Download Full-text

COMPARE

Proceedings of the VLDB Endowment ◽

10.14778/3476249.3476291 ◽

2021 ◽

Vol 14 (11) ◽

pp. 2419-2431

Author(s):

Tarique Siddiqui ◽

Surajit Chaudhuri ◽

Vivek Narasayya

Keyword(s):

Relational Databases ◽

Poor Performance ◽

Database Systems ◽

Optimization Techniques ◽

Sql Server ◽

High Dimensional ◽

Extensive Evaluation ◽

Real World Datasets ◽

High Dimensional Datasets ◽

Database Engine

Data analysis often involves comparing subsets of data across many dimensions for finding unusual trends and patterns. While the comparison between subsets of data can be expressed using SQL, they tend to be complex to write, and suffer from poor performance over large and high-dimensional datasets. In this paper, we propose a new logical operator COMPARE for relational databases that concisely captures the enumeration and comparison between subsets of data and greatly simplifies the expressing of a large class of comparative queries. We extend the database engine with optimization techniques that exploit the semantics of COMPARE to significantly improve the performance of such queries. We have implemented these extensions inside Microsoft SQL Server, a commercial DBMS engine. Our extensive evaluation on synthetic and real-world datasets shows that COMPARE results in a significant speedup over existing approaches, including physical plans generated by today's database systems, user-defined functions (UDFs), as well as middleware solutions that compare subsets outside the databases.

Download Full-text

Siamese Reconstruction Network: Accurate Image Reconstruction from Human Brain Activity by Learning to Compare

Applied Sciences ◽

10.3390/app9224749 ◽

2019 ◽

Vol 9 (22) ◽

pp. 4749

Author(s):

Lingyun Jiang ◽

Kai Qiao ◽

Linyuan Wang ◽

Chi Zhang ◽

Jian Chen ◽

...

Keyword(s):

Deep Learning ◽

Human Brain ◽

Brain Activity ◽

Feature Space ◽

Training Data ◽

Reconstruction Method ◽

Learning Method ◽

Training Samples ◽

Visual Reconstruction ◽

Relationship Of

Decoding human brain activities, especially reconstructing human visual stimuli via functional magnetic resonance imaging (fMRI), has gained increasing attention in recent years. However, the high dimensionality and small quantity of fMRI data impose restrictions on satisfactory reconstruction, especially for the reconstruction method with deep learning requiring huge amounts of labelled samples. When compared with the deep learning method, humans can recognize a new image because our human visual system is naturally capable of extracting features from any object and comparing them. Inspired by this visual mechanism, we introduced the mechanism of comparison into deep learning method to realize better visual reconstruction by making full use of each sample and the relationship of the sample pair by learning to compare. In this way, we proposed a Siamese reconstruction network (SRN) method. By using the SRN, we improved upon the satisfying results on two fMRI recording datasets, providing 72.5% accuracy on the digit dataset and 44.6% accuracy on the character dataset. Essentially, this manner can increase the training data about from n samples to 2n sample pairs, which takes full advantage of the limited quantity of training samples. The SRN learns to converge sample pairs of the same class or disperse sample pairs of different class in feature space.

Download Full-text

Improved SinGAN Integrated with an Attentional Mechanism for Remote Sensing Image Classification

Remote Sensing ◽

10.3390/rs13091713 ◽

2021 ◽

Vol 13 (9) ◽

pp. 1713

Author(s):

Songwei Gu ◽

Rui Zhang ◽

Hongxia Luo ◽

Mengyao Li ◽

Huamei Feng ◽

...

Keyword(s):

Remote Sensing ◽

Real Life ◽

Attention Mechanism ◽

Training Data ◽

Generative Adversarial Networks ◽

Natural Image ◽

Remote Sensing Images ◽

Training Time ◽

Adversarial Networks ◽

Remote Sensing Image Classification

Deep learning is an important research method in the remote sensing field. However, samples of remote sensing images are relatively few in real life, and those with markers are scarce. Many neural networks represented by Generative Adversarial Networks (GANs) can learn from real samples to generate pseudosamples, rather than traditional methods that often require more time and man-power to obtain samples. However, the generated pseudosamples often have poor realism and cannot be reliably used as the basis for various analyses and applications in the field of remote sensing. To address the abovementioned problems, a pseudolabeled sample generation method is proposed in this work and applied to scene classification of remote sensing images. The improved unconditional generative model that can be learned from a single natural image (Improved SinGAN) with an attention mechanism can effectively generate enough pseudolabeled samples from a single remote sensing scene image sample. Pseudosamples generated by the improved SinGAN model have stronger realism and relatively less training time, and the extracted features are easily recognized in the classification network. The improved SinGAN can better identify sub-jects from images with complex ground scenes compared with the original network. This mechanism solves the problem of geographic errors of generated pseudosamples. This study incorporated the generated pseudosamples into training data for the classification experiment. The result showed that the SinGAN model with the integration of the attention mechanism can better guarantee feature extraction of the training data. Thus, the quality of the generated samples is improved and the classification accuracy and stability of the classification network are also enhanced.

Download Full-text

Experiments with Neural Networks in the Identification and Control of a Magnetic Levitation System Using a Low-Cost Platform

Applied Sciences ◽

10.3390/app11062535 ◽

2021 ◽

Vol 11 (6) ◽

pp. 2535

Author(s):

Bruno E. Silva ◽

Ramiro S. Barbosa

Keyword(s):

Neural Networks ◽

Neural Control ◽

Magnetic Levitation ◽

Low Cost ◽

Training Data ◽

Neural Controllers ◽

Levitation System ◽

Internal Model Controller ◽

Magnetic Levitation System ◽

And Performance

In this article, we designed and implemented neural controllers to control a nonlinear and unstable magnetic levitation system composed of an electromagnet and a magnetic disk. The objective was to evaluate the implementation and performance of neural control algorithms in a low-cost hardware. In a first phase, we designed two classical controllers with the objective to provide the training data for the neural controllers. After, we identified several neural models of the levitation system using Nonlinear AutoRegressive eXogenous (NARX)-type neural networks that were used to emulate the forward dynamics of the system. Finally, we designed and implemented three neural control structures: the inverse controller, the internal model controller, and the model reference controller for the control of the levitation system. The neural controllers were tested on a low-cost Arduino control platform through MATLAB/Simulink. The experimental results proved the good performance of the neural controllers.

Download Full-text

MULFE: Multi-Label Learning via Label-Specific Feature Space Ensemble

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3451392 ◽

2021 ◽

Vol 16 (1) ◽

pp. 1-24

Author(s):

Yaojin Lin ◽

Qinghua Hu ◽

Jinghua Liu ◽

Xingquan Zhu ◽

Xindong Wu

Keyword(s):

Empirical Studies ◽

Feature Space ◽

Training Data ◽

Data Sets ◽

Learning Framework ◽

Feature Spaces ◽

Public Data ◽

Margin Distribution ◽

Label Correlations ◽

Label Correlation

In multi-label learning, label correlations commonly exist in the data. Such correlation not only provides useful information, but also imposes significant challenges for multi-label learning. Recently, label-specific feature embedding has been proposed to explore label-specific features from the training data, and uses feature highly customized to the multi-label set for learning. While such feature embedding methods have demonstrated good performance, the creation of the feature embedding space is only based on a single label, without considering label correlations in the data. In this article, we propose to combine multiple label-specific feature spaces, using label correlation, for multi-label learning. The proposed algorithm, mu lti- l abel-specific f eature space e nsemble (MULFE), takes consideration label-specific features, label correlation, and weighted ensemble principle to form a learning framework. By conducting clustering analysis on each label’s negative and positive instances, MULFE first creates features customized to each label. After that, MULFE utilizes the label correlation to optimize the margin distribution of the base classifiers which are induced by the related label-specific feature spaces. By combining multiple label-specific features, label correlation based weighting, and ensemble learning, MULFE achieves maximum margin multi-label classification goal through the underlying optimization framework. Empirical studies on 10 public data sets manifest the effectiveness of MULFE.

Download Full-text