scholarly journals Survey on Neural Network Architectures with Deep Learning

2020 ◽  
Vol 2 (3) ◽  
pp. 186-194
Author(s):  
Smys S. ◽  
Joy Iong Zong Chen ◽  
Subarna Shakya
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Visual Recognition ◽  
Research Work ◽  
Network Architectures ◽  
Significant Performance ◽  
Recent Trends ◽  
Cost Efficient ◽  
The Cost ◽  
Learning Architectures

In the present research era, machine learning is an important and unavoidable zone where it provides better solutions to various domains. In particular deep learning is one of the cost efficient, effective supervised learning model, which can be applied to various complicated issues. Since deep learning has various illustrative features and it doesn’t depend on any limited learning methods which helps to obtain better solutions. As deep learning has significant performance and advancements it is widely used in various applications like image classification, face recognition, visual recognition, language processing, speech recognition, object detection and various science, business analysis, etc., This survey work mainly provides an insight about deep learning through an intensive analysis of deep learning architectures and its characteristics along with its limitations. Also, this research work analyses recent trends in deep learning through various literatures to explore the present evolution in deep learning models.

scholarly journals Automated Source Code Generation and Auto-Completion Using Deep Learning: Comparing and Discussing Current Language Model-Related Approaches

AI ◽  
2021 ◽  
Vol 2 (1) ◽  
pp. 1-16
Author(s):  
Juan Cruz-Benito ◽  
Sanjay Vishwakarma ◽  
Francisco Martin-Fernandez ◽  
Ismael Faro
Keyword(s):  
Deep Learning ◽  
Learning Community ◽  
Programming Languages ◽  
Language Processing ◽  
Code Generation ◽  
Language Model ◽  
Language Models ◽  
Stochastic Gradient Descent ◽  
Network Architectures ◽  
Learning Architectures

In recent years, the use of deep learning in language models has gained much attention. Some research projects claim that they can generate text that can be interpreted as human writing, enabling new possibilities in many application areas. Among the different areas related to language processing, one of the most notable in applying this type of modeling is programming languages. For years, the machine learning community has been researching this software engineering area, pursuing goals like applying different approaches to auto-complete, generate, fix, or evaluate code programmed by humans. Considering the increasing popularity of the deep learning-enabled language models approach, we found a lack of empirical papers that compare different deep learning architectures to create and use language models based on programming code. This paper compares different neural network architectures like Average Stochastic Gradient Descent (ASGD) Weight-Dropped LSTMs (AWD-LSTMs), AWD-Quasi-Recurrent Neural Networks (QRNNs), and Transformer while using transfer learning and different forms of tokenization to see how they behave in building language models using a Python dataset for code generation and filling mask tasks. Considering the results, we discuss each approach’s different strengths and weaknesses and what gaps we found to evaluate the language models or to apply them in a real programming context.

Two New Large Corpora for Vietnamese Aspect-based Sentiment Analysis at Sentence Level

2021 ◽  
Vol 20 (4) ◽  
pp. 1-22
Author(s):  
Dang Van Thin ◽  
Ngan Luu-Thuy Nguyen ◽  
Tri Minh Truong ◽  
Lac Si Le ◽  
Duy Tin Vo
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
Network Architectures ◽  
Low Resource ◽  
The Neural Network ◽  
Sentence Level ◽  
Push Forward ◽  
Polarity Classification ◽  
Learning Architectures ◽  
Single Approach

Aspect-based sentiment analysis has been studied in both research and industrial communities over recent years. For the low-resource languages, the standard benchmark corpora play an important role in the development of methods. In this article, we introduce two benchmark corpora with the largest sizes at sentence-level for two tasks: Aspect Category Detection and Aspect Polarity Classification in Vietnamese. Our corpora are annotated with high inter-annotator agreements for the restaurant and hotel domains. The release of our corpora would push forward the low-resource language processing community. In addition, we deploy and compare the effectiveness of supervised learning methods with a single and multi-task approach based on deep learning architectures. Experimental results on our corpora show that the multi-task approach based on BERT architecture outperforms the neural network architectures and the single approach. Our corpora and source code are published on this footnoted site. 1

scholarly journals Deep learning model for metagenome fragment classification using spaced k-mers feature extraction

2020 ◽  
Vol 8 (3) ◽  
pp. 234-238
Author(s):  
Nur Choiriyati ◽  
Yandra Arkeman ◽  
Wisnu Ananta Kusuma
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Deep Learning ◽  
Language Processing ◽  
Computational Time ◽  
Genus Level ◽  
Computational Resources ◽  
Learning Architectures ◽  
Deep Learning Model

An open challenge in bioinformatics is the analysis of the sequenced metagenomes from the various environments. Several studies demonstrated bacteria classification at the genus level using k-mers as feature extraction where the highest value of k gives better accuracy but it is costly in terms of computational resources and computational time. Spaced k-mers method was used to extract the feature of the sequence using 111 1111 10001 where 1 was a match and 0 was the condition that could be a match or did not match. Currently, deep learning provides the best solutions to many problems in image recognition, speech recognition, and natural language processing. In this research, two different deep learning architectures, namely Deep Neural Network (DNN) and Convolutional Neural Network (CNN), trained to approach the taxonomic classification of metagenome data and spaced k-mers method for feature extraction. The result showed the DNN classifier reached 90.89 % and the CNN classifier reached 88.89 % accuracy at the genus level taxonomy.

scholarly journals Classification of Remote Sensing Image Scenes Using Double Feature Extraction Hybrid Deep Learning Approach

2021 ◽  
Vol 3 (2) ◽  
pp. 133-149
Author(s):  
Akey Sungheetha ◽  
Rajesh Sharma R
Keyword(s):  
Remote Sensing ◽  
Feature Extraction ◽  
Deep Learning ◽  
Visual Recognition ◽  
Research Work ◽  
Recognition Rate ◽  
Remote Sensing Image ◽  
Quality Data ◽  
Learning Approach ◽  
Remote Sensing Images

Over the last decade, remote sensing technology has advanced dramatically, resulting in significant improvements on image quality, data volume, and application usage. These images have essential applications since they can help with quick and easy interpretation. Many standard detection algorithms fail to accurately categorize a scene from a remote sensing image recorded from the earth. A method that uses bilinear convolution neural networks to produce a lessweighted set of models those results in better visual recognition in remote sensing images using fine-grained techniques. This proposed hybrid method is utilized to extract scene feature information in two times from remote sensing images for improved recognition. In layman's terms, these features are defined as raw, and only have a single defined frame, so they will allow basic recognition from remote sensing images. This research work has proposed a double feature extraction hybrid deep learning approach to classify remotely sensed image scenes based on feature abstraction techniques. Also, the proposed algorithm is applied to feature values in order to convert them to feature vectors that have pure black and white values after many product operations. The next stage is pooling and normalization, which occurs after the CNN feature extraction process has changed. This research work has developed a novel hybrid framework method that has a better level of accuracy and recognition rate than any prior model.

scholarly journals Deep Learning Based Question Answering Search Engine

2021 ◽  
pp. 25-32
Author(s):  
Mrunal Malekar
Keyword(s):  
Information Retrieval ◽  
Deep Learning ◽  
Natural Language ◽  
Search Engine ◽  
Language Processing ◽  
Question Answering ◽  
Research Work ◽  
Construction Company ◽  
Exact Answer ◽  
Search For Information

Domain based Question Answering is concerned with building systems which provide answers to natural language questions that are asked specific to a domain. It comes under Information Retrieval and Natural language processing. Using Information Retrieval, one can search for the relevant documents which may contain the answer but it won’t give the exact answer for the question asked. In the presented work, a question answering search engine has been developed which first finds out the relevant documents from a huge textual document data of a construction company and then goes a step beyond to extract answer from the extracted document. The robust question answering system developed uses Elastic Search for Information Retrieval [paragraphs extraction] and Deep Learning for answering the question from the short extracted paragraph. It leverages BERT Deep Learning Model to understand the layers and representations between the question and answer. The research work also focuses on how to improve the search accuracy of the Information Retrieval based Elastic Search engine which returns the relevant documents which may contain the answer.

scholarly journals A comparative analysis on question classification task based on deep learning approaches

2021 ◽  
Vol 7 ◽  
pp. e570
Author(s):  
Muhammad Zulqarnain ◽  
Ahmed Khalaf Zager Alsaedi ◽  
Rozaida Ghazali ◽  
Muhammad Ghulam Ghouse ◽  
Wareesa Sharif ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Web Mining ◽  
Question Answering ◽  
Short Term Memory ◽  
Learning Approaches ◽  
Question Classification ◽  
Considerable Impact ◽  
Turkish Language ◽  
Learning Architectures

Question classification is one of the essential tasks for automatic question answering implementation in natural language processing (NLP). Recently, there have been several text-mining issues such as text classification, document categorization, web mining, sentiment analysis, and spam filtering that have been successfully achieved by deep learning approaches. In this study, we illustrated and investigated our work on certain deep learning approaches for question classification tasks in an extremely inflected Turkish language. In this study, we trained and tested the deep learning architectures on the questions dataset in Turkish. In addition to this, we used three main deep learning approaches (Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN)) and we also applied two different deep learning combinations of CNN-GRU and CNN-LSTM architectures. Furthermore, we applied the Word2vec technique with both skip-gram and CBOW methods for word embedding with various vector sizes on a large corpus composed of user questions. By comparing analysis, we conducted an experiment on deep learning architectures based on test and 10-cross fold validation accuracy. Experiment results were obtained to illustrate the effectiveness of various Word2vec techniques that have a considerable impact on the accuracy rate using different deep learning approaches. We attained an accuracy of 93.7% by using these techniques on the question dataset.

scholarly journals Exploring the Effectiveness of Convolutional Neural Network with Ensemble Technique

2020 ◽  
Author(s):  
Monalisha Ghosh ◽  
Goutam Sanyal
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Research Work ◽  
Research Field ◽  
Svm Classifier ◽  
Binary Particle Swarm Optimization ◽  
Ensemble Technique

Abstract ­­­­­­­­­­­­­­­­­­­­­­­­­­­ Sentiment Analysis has recently been considered as the most active research field in the natural language processing (NLP) domain. Deep Learning is a subset of the large family of Machine Learning and becoming a growing trend due to its automatic learning capability with impressive consequences across different NLP tasks. Hence, a fusion-based Machine Learning framework has been attempted by merging the Traditional Machine Learning method with Deep Learning techniques to tackle the challenge of sentiment prediction for a massive amount of unstructured review dataset. The proposed architecture aims to utilize the Convolutional Neural Network (CNN) with a backpropagation algorithm to extract embedded feature vectors from the top hidden layer. Thereafter, these vectors augmented to an optimized feature set generated from binary particle swarm optimization (BPSO) method. Finally, a traditional SVM classifier is trained with these extended features set to determine the optimal hyper-plane for separating two classes of review datasets. The evaluation of this research work has been carried out on two benchmark movie review datasets IMDB, SST2. Experimental results with comparative studies based on performance accuracy and F-score value are reported to highlight the benefits of the developed frameworks.

scholarly journals Recent advances of deep learning in psychiatric disorders

2020 ◽  
Vol 3 (3) ◽  
pp. 202-213
Author(s):  
Lu Chen ◽  
Chunchao Xia ◽  
Huaiqiang Sun
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Psychiatric Disorders ◽  
Language Processing ◽  
Visual Recognition ◽  
Learning Algorithm ◽  
Future Research ◽  
Academic World ◽  
Recent Advances ◽  
Conventional Machine

ABSTRACT Deep learning (DL) is a recently proposed subset of machine learning methods that has gained extensive attention in the academic world, breaking benchmark records in areas such as visual recognition and natural language processing. Different from conventional machine learning algorithm, DL is able to learn useful representations and features directly from raw data through hierarchical nonlinear transformations. Because of its ability to detect abstract and complex patterns, DL has been used in neuroimaging studies of psychiatric disorders, which are characterized by subtle and diffuse alterations. Here, we provide a brief review of recent advances and associated challenges in neuroimaging studies of DL applied to psychiatric disorders. The results of these studies indicate that DL could be a powerful tool in assisting the diagnosis of psychiatric diseases. We conclude our review by clarifying the main promises and challenges of DL application in psychiatric disorders, and possible directions for future research.

Part-of-Speech (POS) Tagging Using Deep Learning-Based Approaches on the Designed Khasi POS Corpus

2022 ◽  
Vol 21 (3) ◽  
pp. 1-24
Author(s):  
Sunita Warjri ◽  
Partha Pakray ◽  
Saralin A. Lyngdoh ◽  
Arnab Kumar Maji
Keyword(s):  
Deep Learning ◽  
Natural Language ◽  
Computational Linguistics ◽  
Language Processing ◽  
Research Work ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Corpus Size ◽  
Increase In Accuracy ◽  
Pos Tagger

Part-of-speech (POS) tagging is one of the research challenging fields in natural language processing (NLP). It requires good knowledge of a particular language with large amounts of data or corpora for feature engineering, which can lead to achieving a good performance of the tagger. Our main contribution in this research work is the designed Khasi POS corpus. Till date, there has been no form of any kind of Khasi corpus developed or formally developed. In the present designed Khasi POS corpus, each word is tagged manually using the designed tagset. Methods of deep learning have been used to experiment with our designed Khasi POS corpus. The POS tagger based on BiLSTM, combinations of BiLSTM with CRF, and character-based embedding with BiLSTM are presented. The main challenges of understanding and handling Natural Language toward Computational linguistics to encounter are anticipated. In the presently designed corpus, we have tried to solve the problems of ambiguities of words concerning their context usage, and also the orthography problems that arise in the designed POS corpus. The designed Khasi corpus size is around 96,100 tokens and consists of 6,616 distinct words. Initially, while running the first few sets of data of around 41,000 tokens in our experiment the taggers are found to yield considerably accurate results. When the Khasi corpus size has been increased to 96,100 tokens, we see an increase in accuracy rate and the analyses are more pertinent. As results, accuracy of 96.81% is achieved for the BiLSTM method, 96.98% for BiLSTM with CRF technique, and 95.86% for character-based with LSTM. Concerning substantial research from the NLP perspectives for Khasi, we also present some of the recently existing POS taggers and other NLP works on the Khasi language for comparative purposes.

Biological Data Classification and Analysis Using Convolutional Neural Network

2020 ◽  
Vol 10 (10) ◽  
pp. 2459-2465
Author(s):  
Iftikhar Ahmad ◽  
Muhammad Javed Iqbal ◽  
Mohammad Basheri
Keyword(s):  
Deep Learning ◽  
Early Diagnosis ◽  
Language Processing ◽  
Large Scale ◽  
Genetic Diseases ◽  
Research Work ◽  
Biological Data ◽  
Learning Ability ◽  
Learning Models ◽  
Depth Analysis

The size of data gathered from various ongoing biological and clinically studies is increasing at an exponential rate. The bio-inspired data mainly comprises of genes of DNA, protein and variety of proteomics and genetic diseases. Additionally, DNA microarray data is also available for early diagnosis and prediction of various types of cancer diseases. Interestingly, this data may store very vital information about genes, their structure and important biological function. The huge volume and constant increase in the extracted bio data has opened several challenges. Many bioinformatics and machine learning models have been developed but those fail to address key challenges presents in the efficient and accurate analysis of variety of complex biologically inspired data such as genetic diseases etc. The reliable and robust process of classifying the extracted data into different classes based on the information hidden in the sample data is also a very interesting and open problem. This research work mainly focuses to overcome major challenges in the accurate protein classification keeping in view of the success of deep learning models in natural language processing since it assumes the proteins sequences as a language. The learning ability and overall classification performance of the proposed system can be validated with deep learning classification models. The proposed system can have the superior ability to accurately classify the mentioned datasets than previous approaches and shows better results. The in-depth analysis of multifaceted biological data may also help in the early diagnosis of diseases that causes due to mutation of genes and to overcome arising challenges in the development of large-scale healthcare systems.

Sign in / Sign up

Export Citation Format

Share Document