Automated feature extraction in deep learning models: A boon or a bane?

Deep learning models have strong abilities in learning features and they have been successfully applied in hyperspectral images (HSIs). However, the training of most deep learning models requires labeled samples and the collection of labeled samples are labor-consuming in HSI. In addition, single-level features from a single layer are usually considered, which may result in the loss of some important information. Using multiple networks to obtain multi-level features is a solution, but at the cost of longer training time and computational complexity. To solve these problems, a novel unsupervised multi-level feature extraction framework that is based on a three dimensional convolutional autoencoder (3D-CAE) is proposed in this paper. The designed 3D-CAE is stacked by fully 3D convolutional layers and 3D deconvolutional layers, which allows for the spectral-spatial information of targets to be mined simultaneously. Besides, the 3D-CAE can be trained in an unsupervised way without involving labeled samples. Moreover, the multi-level features are directly obtained from the encoded layers with different scales and resolutions, which is more efficient than using multiple networks to get them. The effectiveness of the proposed multi-level features is verified on two hyperspectral data sets. The results demonstrate that the proposed method has great promise in unsupervised feature learning and can help us to further improve the hyperspectral classification when compared with single-level features.

Download Full-text

Challenges of Sarcasm Detection for Social Network : A Literature Review

JUITA Jurnal Informatika ◽

10.30595/juita.v8i2.8709 ◽

2020 ◽

Vol 8 (2) ◽

pp. 169

Author(s):

Afiyati Afiyati ◽

Azhari Azhari ◽

Anny Kartika Sari ◽

Abdul Karim

Keyword(s):

Social Media ◽

Feature Extraction ◽

Deep Learning ◽

Contextual Information ◽

Unstructured Data ◽

Learning Models ◽

Critical Aspect ◽

Hybrid Approaches ◽

Data Problem ◽

The Future

Nowadays, sarcasm recognition and detection simplified with various domains knowledge, among others, computer science, social science, psychology, mathematics, and many more. This article aims to explain trends in sentiment analysis especially sarcasm detection in the last ten years and its direction in the future. We review journals with the title’s keyword “sarcasm” and published from the year 2008 until 2018. The articles were classified based on the most frequently discussed topics among others: the dataset, pre-processing, annotations, approaches, features, context, and methods used. The significant increase in the number of articles on “sarcasm” in recent years indicates that research in this area still has enormous opportunities. The research about “sarcasm” also became very interesting because only a few researchers offer solutions for unstructured language. Some hybrid approaches using classification and feature extraction are used to identify the sarcasm sentence using deep learning models. This article will provide a further explanation of the most widely used algorithms for sarcasm detection with object social media. At the end of this article also shown that the critical aspect of research on sarcasm sentence that could be done in the future is dataset usage with various languages that cover unstructured data problem with contextual information will effectively detect sarcasm sentence and will improve the existing performance.

Download Full-text

AUTOMATIC DETECTION OF CARDIOVASCULAR DISEASE USING DEEP KERNEL EXTREME LEARNING MACHINE

Biomedical Engineering Applications Basis and Communications ◽

10.4015/s1016237218500382 ◽

2018 ◽

Vol 30 (06) ◽

pp. 1850038

Author(s):

Dongping Li

Keyword(s):

Neural Network ◽

Cardiovascular Disease ◽

Feature Extraction ◽

Deep Learning ◽

Extreme Learning Machine ◽

Learning Process ◽

Support Vector ◽

Learning Models ◽

Kernel Extreme Learning Machine ◽

Learning Machine

The electrocardiogram (ECG) is a principal signal employed to automatically diagnose cardiovascular disease in shallow and deep learning models. However, ECG feature extraction is required and this may reduce diagnosis accuracy in traditional shallow learning models, while backward propagation (BP) algorithm used by the traditional deep learning models has the disadvantages of local minimization and slow convergence rate. To solve these problems, a new deep learning algorithm called deep kernel extreme learning machine (DKELM) is proposed by combining the extreme learning machine auto-encoder (ELM-AE) and kernel ELM (KELM). In the new DKELM architecture with [Formula: see text] hidden layers, ELM-AEs are employed by the front [Formula: see text] hidden layers for feature extraction in the unsupervised learning process, which can effectively extract abstract features from the original ECG signal. To overcome the “dimension disaster” problem, the kernel function is introduced into ELM to act as classifier by the [Formula: see text]th hidden layer in the supervised learning process. The experiments demonstrate that DKELM outperforms the BP neural network, support vector machine (SVM), extreme learning machine (ELM), deep auto-encoder (DAE), deep belief network (DBN) in classification accuracy. Though the accuracy of convolutional neural network (CNN) is almost the same as DKELM, the computing time of CNN is much longer than DKELM.

Download Full-text

Semi-Supervised Generative Adversarial Nets with Multiple Generators for SAR Image Recognition

Sensors ◽

10.3390/s18082706 ◽

2018 ◽

Vol 18 (8) ◽

pp. 2706 ◽

Cited By ~ 9

Author(s):

Fei Gao ◽

Fei Ma ◽

Jun Wang ◽

Jinping Sun ◽

Erfu Yang ◽

...

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

Recognition Performance ◽

Sar Image ◽

Learning Models ◽

Sar Images ◽

Stationary Target ◽

Optical Images ◽

Supervised Methods ◽

The Stability

As an important model of deep learning, semi-supervised learning models are based on Generative Adversarial Nets (GANs) and have achieved a competitive performance on standard optical images. However, the training of GANs becomes unstable when they are applied to SAR images, which reduces the feature extraction capability of the discriminator in GANs. This paper presents a new semi-supervised GANs with Multiple generators and a classifier (MCGAN). This model improves the stability of training for SAR images by employing multiple generators. A multi-classifier is introduced to the new GANs to utilize the labeled images during the training of the GANs, which shares the low level layers with the discriminator. Then, the layers of the trained discriminator and the classifier construct the recognition network for SAR images after having been finely tuned using a small number of the labeled images. Experiments on the Moving and Stationary Target Acquisition and Recognition (MSTAR) databases show that the proposed recognition network achieves a better and more stable recognition performance than several traditional semi-supervised methods as well as other GANs-based semi-supervised methods.

Download Full-text

ANALISIS SENTIMEN TERHADAP GAME GENSHIN IMPACT MENGGUNAKAN BERT

Rabit : Jurnal Teknologi dan Sistem Informasi Univrab ◽

10.36341/rabit.v6i2.1765 ◽

2021 ◽

Vol 6 (2) ◽

pp. 122-129

Author(s):

Ryo Kusnadi ◽

Yusuf Yusuf ◽

Andriantony Andriantony ◽

Richard Ardian Yaputra ◽

Melna Caintan

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

Learning Models ◽

Google Play

Dengan pesatnya peningkatan jasa internet di jaringan sosial, ada banyaknya informasi dalam jumlah besar terus-menerus dihasilkan secara langsung di saat yang sama. Akhir-akhir ini, analisis sentimen dengan menggunakan ulasan dan pesan telah menjadi topik penelitian yang populer dibicarakan di bidang Natural Langauage Processing. Selama bertahun-tahun, permainan online telah menjadi suatu aktivitas yang tidak bisa dipisahkan dari Sebagian besar orang, terlebih karena gangguan ekonomi yang disebabkan oleh virus Covid-19. Genshin Impact adalah salah satu permainan terkenal yang dikembangkan oleh miHoYo. Penelitian ini berfokus pada analisis sentimen dengan tujuan mengetahui apakah ulasan terpercaya yang dikumpulkan dari Google Play Store memiliki sentimen netral, baik atau sentimen buruk sehingga dapat membantu pengembangan permainan kedepannya. Diperlukan proses klasifikasi analisis sentimen otomatis untuk mengurangi kesalahan yang disebabkan oleh sumber daya manusia. Meskipun demikian, sangat jarang ditemukan studi yang membahas feature extraction dan deep learning models yang sesuai dengan kasus ini, terutama dalam bisnis permainan. Tahap proses penelitian ini adalah pengekstraksian data melalui Google Play Store, dan menggunakan Bidirectional Encoder Representations from Transformers (BERT) sebagai model kecerdasan buatan.

Download Full-text

Automating Morphological Profiling with Generic Deep Convolutional Networks

10.1101/085118 ◽

2016 ◽

Cited By ~ 19

Author(s):

Nick Pawlowski ◽

Juan C Caicedo ◽

Shantanu Singh ◽

Anne E Carpenter ◽

Amos Storkey

Keyword(s):

Computer Vision ◽

Feature Extraction ◽

Deep Learning ◽

Processing Speed ◽

State Of The Art ◽

Learning Models ◽

Convolutional Networks ◽

Automated Processing ◽

Classical Computer ◽

Microscopy Images

AbstractMorphological profiling aims to create signatures of genes, chemicals and diseases from microscopy images. Current approaches use classical computer vision-based segmentation and feature extraction. Deep learning models achieve state-of-the-art performance in many computer vision tasks such as classification and segmentation. We propose to transfer activation features of generic deep convolutional networks to extract features for morphological profiling. Our approach surpasses currently used methods in terms of accuracy and processing speed. Furthermore, it enables fully automated processing of microscopy images without need for single cell identification.

Download Full-text

Smart teaching mode based on particle swarm image recognition and human-computer interaction deep learning

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189048 ◽

2020 ◽

Vol 39 (4) ◽

pp. 5699-5711

Author(s):

Shirong Long ◽

Xuekong Zhao

Keyword(s):

Feature Extraction ◽

Particle Swarm Optimization ◽

Deep Learning ◽

Real Time ◽

Image Recognition ◽

Particle Swarm ◽

Learning Technology ◽

Search Performance ◽

Swarm Optimization ◽

Teaching Mode

The smart teaching mode overcomes the shortcomings of traditional teaching online and offline, but there are certain deficiencies in the real-time feature extraction of teachers and students. In view of this, this study uses the particle swarm image recognition and deep learning technology to process the intelligent classroom video teaching image and extracts the classroom task features in real time and sends them to the teacher. In order to overcome the shortcomings of the premature convergence of the standard particle swarm optimization algorithm, an improved strategy for multiple particle swarm optimization algorithms is proposed. In order to improve the premature problem in the search performance algorithm of PSO algorithm, this paper combines the algorithm with the useful attributes of other algorithms to improve the particle diversity in the algorithm, enhance the global search ability of the particle, and achieve effective feature extraction. The research indicates that the method proposed in this paper has certain practical effects and can provide theoretical reference for subsequent related research.

Download Full-text

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

10.26434/chemrxiv.12562121 ◽

2020 ◽

Author(s):

Dean Sumner ◽

Jiazhen He ◽

Amol Thakkar ◽

Ola Engkvist ◽

Esben Jannik Bjerrum

Keyword(s):

Neural Networks ◽

Pattern Recognition ◽

Deep Learning ◽

Recurrent Neural Networks ◽

Data Augmentation ◽

State Of The Art ◽

Sequence Similarity ◽

Learning Models ◽

Underlying Network

SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as attentional gain – an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.

Download Full-text

Improving the Accuracy of Protein-Ligand Binding Affinity Prediction by Deep Learning Models: Benchmark and Model

10.26434/chemrxiv.9866912 ◽

2019 ◽

Author(s):

Mohammad Rezaei ◽

Yanjun Li ◽

Xiaolin Li ◽

Chenglong Li

Keyword(s):

Deep Learning ◽

Drug Design ◽

Binding Affinity ◽

Benchmark Dataset ◽

Rational Drug Design ◽

Learning Models ◽

Structure Based Drug Design ◽

Binding Affinity Prediction ◽

Affinity Prediction ◽

Rational Drug

Introduction: The ability to discriminate among ligands binding to the same protein target in terms of their relative binding affinity lies at the heart of structure-based drug design. Any improvement in the accuracy and reliability of binding affinity prediction methods decreases the discrepancy between experimental and computational results. Objectives: The primary objectives were to find the most relevant features affecting binding affinity prediction, least use of manual feature engineering, and improving the reliability of binding affinity prediction using efficient deep learning models by tuning the model hyperparameters. Methods: The binding site of target proteins was represented as a grid box around their bound ligand. Both binary and distance-dependent occupancies were examined for how an atom affects its neighbor voxels in this grid. A combination of different features including ANOLEA, ligand elements, and Arpeggio atom types were used to represent the input. An efficient convolutional neural network (CNN) architecture, DeepAtom, was developed, trained and tested on the PDBbind v2016 dataset. Additionally an extended benchmark dataset was compiled to train and evaluate the models. Results: The best DeepAtom model showed an improved accuracy in the binding affinity prediction on PDBbind core subset (Pearson’s R=0.83) and is better than the recent state-of-the-art models in this field. In addition when the DeepAtom model was trained on our proposed benchmark dataset, it yields higher correlation compared to the baseline which confirms the value of our model. Conclusions: The promising results for the predicted binding affinities is expected to pave the way for embedding deep learning models in virtual screening and rational drug design fields.

Download Full-text

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

10.31232/osf.io/4pxq2 ◽

2020 ◽

Author(s):

Saeed Nosratabadi ◽

Amir Mosavi ◽

Puhong Duan ◽

Pedram Ghamisi ◽

Ferdinand Filip ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Data Science ◽

State Of The Art ◽

Science Methods ◽

Learning Models ◽

Diverse Range ◽

Hybrid Machine ◽

Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

Download Full-text