scholarly journals Creating, Managing, and Understanding Large, Sparse, Multitask Neural Networks

2020 ◽  
Author(s):  
Harshvardhan Sikka
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Text Classification ◽  
Multitask Learning ◽  
Learning Tasks ◽  
Deep Networks ◽  
Classification Tasks

One of the popular directions in Deep Learning (DL) research has been to build larger and more complex deep networks that can perform well on several different learning tasks, commonly known as multitask learning. This work is usually done within specific domains, e.g. multitask models that perform captioning, translation, and text classification tasks. Some work has been done in building multimodal/crossmodal networks that use deep networks with a combination of different neural network primitives (Convolutional Layers, Recurrent Layers, Mixture of Expert layers, etc). This paper explores various topics and ideas that may prove relevant to large, sparse, multitask networks and explores the potential for a general approach to building and managing these networks. A framework to automatically build, update, and interpret modular LSMNs is presented in the context of current tooling and theory.

scholarly journals A comparative review on deep learning models for text classification

2020 ◽  
Vol 19 (1) ◽  
pp. 325
Author(s):  
Muhammad Zulqarnain ◽  
Rozaida Ghazali ◽  
Yana Mazwin Mohmad Hassim ◽  
Muhammad Rehan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Language Processing ◽  
Text Classification ◽  
Question Answering ◽  
Learning Models ◽  
Semantic Classification ◽  
Analysis Question ◽  
Comparative Review ◽  
Classification Tasks

<p>Text classification is a fundamental task in several areas of natural language processing (NLP), including words semantic classification, sentiment analysis, question answering, or dialog management. This paper investigates three basic architectures of deep learning models for the tasks of text classification: Deep Belief Neural (DBN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), these three main types of deep learning architectures, are largely explored to handled various classification tasks. DBN have excellent learning capabilities to extracts highly distinguishable features and good for general purpose. CNN have supposed to be better at extracting the position of various related features while RNN is modeling in sequential of long-term dependencies. This paper work shows the systematic comparison of DBN, CNN, and RNN on text classification tasks. Finally, we show the results of deep models by research experiment. The aim of this paper to provides basic guidance about the deep learning models that which models are best for the task of text classification.</p>

Identifying Emotion on Indonesian Tweets using Convolutional Neural Networks

2021 ◽  
Vol 5 (3) ◽  
pp. 584-593
Author(s):  
Naufal Hilmiaji ◽  
Kemas Muslim Lhaksmana ◽  
Mahendra Dwifebri Purbolaksono
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Performance Evaluation ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Classification Model ◽  
Learning Methods ◽  
Expected Performance

especially with the advancement of deep learning methods for text classification. Despite some effort to identify emotion on Indonesian tweets, its performance evaluation results have not achieved acceptable numbers. To solve this problem, this paper implements a classification model using a convolutional neural network (CNN), which has demonstrated expected performance in text classification. To easily compare with the previous research, this classification is performed on the same dataset, which consists of 4,403 tweets in Indonesian that were labeled using five different emotion classes: anger, fear, joy, love, and sadness. The performance evaluation results achieve the precision, recall, and F1-score at respectively 90.1%, 90.3%, and 90.2%, while the highest accuracy achieves 89.8%. These results outperform previous research that classifies the same classification on the same dataset.

scholarly journals A systematic review of text classification research based on deep learning models in Arabic language

2020 ◽  
Vol 10 (6) ◽  
pp. 6629
Author(s):  
Ahlam Wahdan ◽  
Sendeyah AL Hantoobi ◽  
Said A. Salloum ◽  
Khaled Shaalan
Keyword(s):  
Neural Network ◽  
Systematic Review ◽  
Neural Networks ◽  
Deep Learning ◽  
Text Classification ◽  
Arabic Language ◽  
Machine Learning Techniques ◽  
Learning Models ◽  
Learning Techniques

Classifying or categorizing texts is the process by which documents are classified into groups by subject, title, author, etc. This paper undertakes a systematic review of the latest research in the field of the classification of Arabic texts. Several machine learning techniques can be used for text classification, but we have focused only on the recent trend of neural network algorithms. In this paper, the concept of classifying texts and classification processes are reviewed. Deep learning techniques in classification and its type are discussed in this paper as well. Neural networks of various types, namely, RNN, CNN, FFNN, and LSTM, are identified as the subject of study. Through systematic study, 12 research papers related to the field of the classification of Arabic texts using neural networks are obtained: for each paper the methodology for each type of neural network and the accuracy ration for each type is determined. The evaluation criteria used in the algorithms of different neural network types and how they play a large role in the highly accurate classification of Arabic texts are discussed. Our results provide some findings regarding how deep learning models can be used to improve text classification research in Arabic language.

Challenges and Issues in Plant Disease Detection Using Deep Learning

2021 ◽  
pp. 56-74
Author(s):  
Priyanka Sahu ◽  
Anuradha Chug ◽  
Amit Prakash Singh ◽  
Dinesh Singh ◽  
Ravinder Pal Singh
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Plant Disease ◽  
Image Database ◽  
Plant Diseases ◽  
Disease Detection ◽  
Factors Influencing ◽  
Classification Tasks ◽  
Main Factors

Deep learning (DL) has rapidly become an essential tool for image classification tasks. This technique is now being deployed to the tasks of classifying and detecting plant diseases. The encouraging results achieved with this methodology hide many problems that are rarely addressed in related experiments. This study examines the main factors influencing the efficiency of deep neural networks for plant disease detection. The challenges discussed in the study are based on the literature as well as experiments conducted using an image database, which contains approximately 1,296 leaf images of the beans crop. A pre-trained convolutional neural network, EfficientNet B0, is used for training and testing purposes. This study gives and emphasizes on factors and challenges that may potentially affect the use of DL techniques to detect and classify plant diseases. Some solutions are also suggested that may overcome these problems.

scholarly journals A Tensor Space Model-Based Deep Neural Network for Text Classification

2021 ◽  
Vol 11 (20) ◽  
pp. 9703
Author(s):  
Han-joon Kim ◽  
Pureum Lim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Text Classification ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Classification Systems ◽  
Support Vector ◽  
Tensor Space ◽  
Space Model

Most text classification systems use machine learning algorithms; among these, naïve Bayes and support vector machine algorithms adapted to handle text data afford reasonable performance. Recently, given developments in deep learning technology, several scholars have used deep neural networks (recurrent and convolutional neural networks) to improve text classification. However, deep learning-based text classification has not greatly improved performance compared to that of conventional algorithms. This is because a textual document is essentially expressed as a vector (only), albeit with word dimensions, which compromises the inherent semantic information, even if the vector is (appropriately) transformed to add conceptual information. To solve this `loss of term senses’ problem, we develop a concept-driven deep neural network based upon our semantic tensor space model. The semantic tensor used for text representation features a dependency between the term and the concept; we use this to develop three deep neural networks for text classification. We perform experiments using three standard document corpora, and we show that our proposed methods are superior to both traditional and more recent learning methods.

A Deep Learning Model and a Dataset for Diagnosing Ophthalmology Diseases

2021 ◽  
pp. 2150036
Author(s):  
Rehab M. Duwairi ◽  
Saad A. Al-Zboon ◽  
Rami A. Al-Dwairi ◽  
Ahmad Obaidi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Macular Hole ◽  
Binary Classification ◽  
Rapid Development ◽  
Weighted Kappa ◽  
Classification Tasks ◽  
Medical Domain ◽  
Multi Class Classification

The rapid development of artificial neural network techniques, especially convolutional neural networks, encouraged the researchers to adapt such techniques in the medical domain. Specifically, to provide assist tools to help the professionals in patients’ diagnosis. The main problem faced by the researchers in the medical domain is the lack of available annotated datasets which can be used to train and evaluate large and complex deep neural networks. In this paper, to assist researchers who are interested in applying deep learning techniques to aid the ophthalmologists in diagnosing eye-related diseases, we provide an optical coherence tomography dataset with collaboration with ophthalmologists from the King Abdullah University Hospital, Irbid, Jordan. This dataset consists of 21,991 OCT images distributed over seven eye diseases in addition to normal images (no disease), namely, Choroidal Neovascularisation, Full Macular Hole (Full Thickness), Partial Macular Hole, Central Serous Retinopathy, Geographic atrophy, Macular Retinal Oedema, and Vitreomacular Traction. To the best of our knowledge, this dataset is the largest of its kind, where images belong to actual patients from Jordan and the annotation was carried out by ophthalmologists. Two classification tasks were applied to this dataset; a binary classification to distinguish between images which belong to healthy eyes (normal) and images which belong to diseased eyes (abnormal). The second classification task is a multi-class classification, where the deep neural network is trained to distinguish between the seven diseases listed above in addition to the normal case. In both classification tasks, the U-Net neural network was modified and subsequently utilised. This modification adds an additional block of layers to the original U-Net model to become capable of handling classification as the original network is used for image segmentation. The results of the binary classification were equal to 84.90% and 69.50% as accuracy and quadratic weighted kappa, respectively. The results of the multi-class classification, by contrast, were equal to 63.68% and 66.06% as accuracy and quadratic weighted kappa, respectively.

scholarly journals Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

2021 ◽  
Vol 26 (1) ◽  
pp. 200-215
Author(s):  
Muhammad Alam ◽  
Jian-Feng Wang ◽  
Cong Guangpei ◽  
LV Yunrong ◽  
Yuanfang Chen
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Natural Scene ◽  
Remote Sensing Images ◽  
Advantages And Disadvantages ◽  
Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

Searching for new physics: Using explainable AI to understand deep learned parameterizations of turbulent heat fluxes

2021 ◽  
Author(s):  
Andrew Bennett ◽  
Bart Nijssen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Coupled System ◽  
Hydrologic Model ◽  
Heat Fluxes ◽  
Turbulent Heat ◽  
Phase Lag ◽  
Geophysical Research ◽  
Turbulent Heat Fluxes

&lt;p&gt;Machine learning (ML), and particularly deep learning (DL), for geophysical research has shown dramatic successes in recent years. However, these models are primarily geared towards better predictive capabilities, and are generally treated as black box models, limiting researchers&amp;#8217; ability to interpret and understand how these predictions are made. As these models are incorporated into larger models and pushed to be used in more areas it will be important to build methods that allow us to reason about how these models operate. This will have implications for scientific discovery that will ensure that these models are robust and reliable for their respective applications. Recent work in explainable artificial intelligence (XAI) has been used to interpret and explain the behavior of machine learned models.&lt;/p&gt;&lt;p&gt;Here, we apply new tools from the field of XAI to provide physical interpretations of a system that couples a deep-learning based parameterization for turbulent heat fluxes to a process based hydrologic model. To develop this coupling we have trained a neural network to predict turbulent heat fluxes using FluxNet data from a large number of hydroclimatically diverse sites. This neural network is coupled to the SUMMA hydrologic model, taking imodel derived states as additional inputs to improve predictions. We have shown that this coupled system provides highly accurate simulations of turbulent heat fluxes at 30 minute timesteps, accurately predicts the long-term observed water balance, and reproduces other signatures such as the phase lag with shortwave radiation. Because of these features, it seems this coupled system is learning physically accurate relationships between inputs and outputs.&amp;#160;&lt;/p&gt;&lt;p&gt;We probe the relative importance of which input features are used to make predictions during wet and dry conditions to better understand what the neural network has learned. Further, we conduct controlled experiments to understand how the neural networks are able to learn to regionalize between different hydroclimates. By understanding how these neural networks make their predictions as well as how they learn to make predictions we can gain scientific insights and use them to further improve our models of the Earth system.&lt;/p&gt;

Sign in / Sign up

Export Citation Format

Share Document