LSTM Neural Network for Textual Ngrams

LSTM neural network for textual ngrams

10.7287/peerj.preprints.27377 ◽

2018 ◽

Author(s):

Shaun C. D'Souza

Keyword(s):

Neural Network ◽

Deep Learning ◽

Human Brain ◽

Cognitive Neuroscience ◽

Probabilistic Models ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Language Models ◽

Brain Functions ◽

The Web

Cognitive neuroscience is the study of how the human brain functions on tasks like decision making, language, perception and reasoning. Deep learning is a class of machine learning algorithms that use neural networks. They are designed to model the responses of neurons in the human brain. Learning can be supervised or unsupervised. Ngram token models are used extensively in language prediction. Ngrams are probabilistic models that are used in predicting the next word or token. They are a statistical model of word sequences or tokens and are called Language Models or Lms. Ngrams are essential in creating language prediction models. We are exploring a broader sandbox ecosystems enabling for AI. Specifically, around Deep learning applications on unstructured content form on the web.

Download Full-text

LSTM neural network for textual ngrams

10.7287/peerj.preprints.27377v1 ◽

2018 ◽

Author(s):

Shaun C. D'Souza

Keyword(s):

Neural Network ◽

Deep Learning ◽

Human Brain ◽

Cognitive Neuroscience ◽

Probabilistic Models ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Language Models ◽

Brain Functions ◽

The Web

Cognitive neuroscience is the study of how the human brain functions on tasks like decision making, language, perception and reasoning. Deep learning is a class of machine learning algorithms that use neural networks. They are designed to model the responses of neurons in the human brain. Learning can be supervised or unsupervised. Ngram token models are used extensively in language prediction. Ngrams are probabilistic models that are used in predicting the next word or token. They are a statistical model of word sequences or tokens and are called Language Models or Lms. Ngrams are essential in creating language prediction models. We are exploring a broader sandbox ecosystems enabling for AI. Specifically, around Deep learning applications on unstructured content form on the web.

Download Full-text

Improving Lives of Indebted Farmers Using Deep Learning: Predicting Agricultural Produce Prices Using Convolutional Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i08.7039 ◽

2020 ◽

Vol 34 (08) ◽

pp. 13294-13299

Author(s):

Hangzhi Guo ◽

Alexander Woodruff ◽

Amulya Yadav

Keyword(s):

Neural Network ◽

Deep Learning ◽

Network Architecture ◽

Prediction Models ◽

Learning Algorithm ◽

State Of The Art ◽

Machine Learning Algorithms ◽

Volume Data ◽

Farmer Suicides ◽

Spatio Temporal

Farmer suicides have become an urgent social problem which governments around the world are trying hard to solve. Most farmers are driven to suicide due to an inability to sell their produce at desired profit levels, which is caused by the widespread uncertainty/fluctuation in produce prices resulting from varying market conditions. To prevent farmer suicides, this paper takes the first step towards resolving the issue of produce price uncertainty by presenting PECAD, a deep learning algorithm for accurate prediction of future produce prices based on past pricing and volume patterns. While previous work presents machine learning algorithms for prediction of produce prices, they suffer from two limitations: (i) they do not explicitly consider the spatio-temporal dependence of future prices on past data; and as a result, (ii) they rely on classical ML prediction models which often perform poorly when applied to spatio-temporal datasets. PECAD addresses these limitations via three major contributions: (i) we gather real-world daily price and (produced) volume data of different crops over a period of 11 years from an official Indian government administered website; (ii) we pre-process this raw dataset via state-of-the-art imputation techniques to account for missing data entries; and (iii) PECAD proposes a novel wide and deep neural network architecture which consists of two separate convolutional neural network models (trained for pricing and volume data respectively). Our simulation results show that PECAD outperforms existing state-of-the-art baseline methods by achieving significantly lesser root mean squared error (RMSE) - PECAD achieves ∼25% lesser coefficient of variance than state-of-the-art baselines. Our work is done in collaboration with a non-profit agency that works on preventing farmer suicides in the Indian state of Jharkhand, and PECAD is currently being reviewed by them for potential deployment.

Download Full-text

Learners Demographics Classification on MOOCs During the COVID-19: Author Profiling via Deep Learning Based on Semantic and Syntactic Representations

Frontiers in Research Metrics and Analytics ◽

10.3389/frma.2021.673928 ◽

2021 ◽

Vol 6 ◽

Author(s):

Tahani Aljohani ◽

Alexandra I. Cristea

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Prediction Models ◽

Short Term Memory ◽

Methodological Approach ◽

High Accuracy ◽

Directional Model ◽

Textual Representations ◽

The One

Massive Open Online Courses (MOOCs) have become universal learning resources, and the COVID-19 pandemic is rendering these platforms even more necessary. In this paper, we seek to improve Learner Profiling (LP), i.e. estimating the demographic characteristics of learners in MOOC platforms. We have focused on examining models which show promise elsewhere, but were never examined in the LP area (deep learning models) based on effective textual representations. As LP characteristics, we predict here the employment status of learners. We compare sequential and parallel ensemble deep learning architectures based on Convolutional Neural Networks and Recurrent Neural Networks, obtaining an average high accuracy of 96.3% for our best method. Next, we predict the gender of learners based on syntactic knowledge from the text. We compare different tree-structured Long-Short-Term Memory models (as state-of-the-art candidates) and provide our novel version of a Bi-directional composition function for existing architectures. In addition, we evaluate 18 different combinations of word-level encoding and sentence-level encoding functions. Based on these results, we show that our Bi-directional model outperforms all other models and the highest accuracy result among our models is the one based on the combination of FeedForward Neural Network and the Stack-augmented Parser-Interpreter Neural Network (82.60% prediction accuracy). We argue that our prediction models recommended for both demographics characteristics examined in this study can achieve high accuracy. This is additionally also the first time a sound methodological approach toward improving accuracy for learner demographics classification on MOOCs was proposed.

Download Full-text

Low-order Spherical Harmonic HRTF Restoration using a Neural Network Approach

10.20944/preprints202007.0209.v1 ◽

2020 ◽

Author(s):

Benjamin Tsui ◽

William A. P. Smith ◽

Gavin Kearney

Keyword(s):

Neural Network ◽

Spherical Harmonic ◽

High Frequency ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Network Approach ◽

Spectral Difference ◽

Neural Network Approach ◽

Frequency Representation ◽

Head Related Transfer Function

Spherical harmonic (SH) interpolation is a commonly used method to spatially up-sample sparse Head Related Transfer Function (HRTF) datasets to denser HRTF datasets. However, depending on the number of sparse HRTF measurements and SH order, this process can introduce distortions in high frequency representation of the HRTFs. This paper investigates whether it is possible to restore some of the distorted high frequency HRTF components using machine learning algorithms. A combination of Convolutional Auto-Encoder (CAE) and Denoising Auto-Encoder (DAE) models is proposed to restore the high frequency distortion in SH interpolated HRTFs. Results are evaluated using both Perceptual Spectral Difference (PSD) and localisation prediction models, both of which demonstrate significant improvement after the restoration process.

Download Full-text

Convolutional Neural Network

10.4018/978-1-6684-2408-7.ch077 ◽

2022 ◽

pp. 1559-1575

Author(s):

Mário Pereira Véstias

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Artificial Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Machine Learning Algorithms ◽

Training Data ◽

Machine Learning Model ◽

Artificial Neural

Machine learning is the study of algorithms and models for computing systems to do tasks based on pattern identification and inference. When it is difficult or infeasible to develop an algorithm to do a particular task, machine learning algorithms can provide an output based on previous training data. A well-known machine learning model is deep learning. The most recent deep learning models are based on artificial neural networks (ANN). There exist several types of artificial neural networks including the feedforward neural network, the Kohonen self-organizing neural network, the recurrent neural network, the convolutional neural network, the modular neural network, among others. This article focuses on convolutional neural networks with a description of the model, the training and inference processes and its applicability. It will also give an overview of the most used CNN models and what to expect from the next generation of CNN models.

Download Full-text

Retinal blood vessels detection for diabetic retinopathy with Ridgelet transform and convolution neural network

International Journal of Wavelets Multiresolution and Information Processing ◽

10.1142/s0219691320500484 ◽

2020 ◽

Vol 18 (06) ◽

pp. 2050048

Author(s):

Nirmal Yadav

Keyword(s):

Neural Network ◽

Machine Learning ◽

Diabetic Retinopathy ◽

Deep Learning ◽

Blood Vessels ◽

Machine Learning Algorithms ◽

Convolution Neural Network ◽

Transform Method ◽

Ridgelet Transform ◽

Study Results

Applying machine learning in life sciences, especially diagnostics, has become a key area of focus for researchers. Combining machine learning with traditional algorithms provides a unique opportunity of providing better solutions for the patients. In this paper, we present study results of applying the Ridgelet Transform method on retina images to enhance the blood vessels, then using machine learning algorithms to identify cases of Diabetic Retinopathy (DR). The Ridgelet transform provides better results for line singularity of image function and, thus, helps to reduce artefacts along the edges of the image. The Ridgelet Transform method, when compared with earlier known methods of image enhancement, such as Wavelet Transform and Contourlet Transform, provided satisfactory results. The transformed image using the Ridgelet Transform method with pre-processing quantifies the amount of information in the dataset. It efficiently enhances the generation of features vectors in the convolution neural network (CNN). In this study, a sample of fundus photographs was processed, which was obtained from a publicly available dataset. In pre-processing, first, CLAHE was applied, followed by filtering and application of Ridgelet transform on the patches to improve the quality of the image. Then, this processed image was used for statistical feature detection and classified by deep learning method to detect DR images from the dataset. The successful classification ratio was 98.61%. This result concludes that the transformed image of fundus using the Ridgelet Transform enables better detection by leveraging a transform-based algorithm and the deep learning.

Download Full-text

A Comparative Study of Bitcoin Price Prediction Using Deep Learning

Mathematics ◽

10.3390/math7100898 ◽

2019 ◽

Vol 7 (10) ◽

pp. 898 ◽

Cited By ~ 12

Author(s):

Suhwan Ji ◽

Jongmin Kim ◽

Hyeonseung Im

Keyword(s):

Neural Network ◽

Deep Learning ◽

Prediction Models ◽

Short Term Memory ◽

Learning Methods ◽

The Public ◽

Price Prediction ◽

Profitability Analysis ◽

The Media ◽

Long Short Term Memory

Bitcoin has recently received a lot of attention from the media and the public due to its recent price surge and crash. Correspondingly, many researchers have investigated various factors that affect the Bitcoin price and the patterns behind its fluctuations, in particular, using various machine learning methods. In this paper, we study and compare various state-of-the-art deep learning methods such as a deep neural network (DNN), a long short-term memory (LSTM) model, a convolutional neural network, a deep residual network, and their combinations for Bitcoin price prediction. Experimental results showed that although LSTM-based prediction models slightly outperformed the other prediction models for Bitcoin price prediction (regression), DNN-based models performed the best for price ups and downs prediction (classification). In addition, a simple profitability analysis showed that classification models were more effective than regression models for algorithmic trading. Overall, the performances of the proposed deep learning-based prediction models were comparable.

Download Full-text

DeepGS: Predicting phenotypes from genotypes using Deep Learning

10.1101/241414 ◽

2017 ◽

Cited By ~ 7

Author(s):

Wenlong Ma ◽

Zhixu Qiu ◽

Jie Song ◽

Qian Cheng ◽

Chuang Ma

Keyword(s):

Neural Network ◽

Deep Learning ◽

Prediction Models ◽

Linear Regression Analysis ◽

Hidden Variables ◽

R Package ◽

Breeding Strategy ◽

Simulation Experiments ◽

Linear Relationships ◽

Genome Wide

AbstractMotivationGenomic selection (GS) is a new breeding strategy by which the phenotypes of quantitative traits are usually predicted based on genome-wide markers of genotypes using conventional statistical models. However, the GS prediction models typically make strong assumptions and perform linear regression analysis, limiting their accuracies since they do not capture the complex, non-linear relationships within genotypes, and between genotypes and phenotypes.ResultsWe present a deep learning method, named DeepGS, to predict phenotypes from genotypes. Using a deep convolutional neural network, DeepGS uses hidden variables that jointly represent features in genotypic markers when making predictions; it also employs convolution, sampling and dropout strategies to reduce the complexity of high-dimensional marker data. We used a large GS dataset to train DeepGS and compare its performance with other methods. In terms of mean normalized discounted cumulative gain value, DeepGS achieves an increase of 27.70%~246.34% over a conventional neural network in selecting top-ranked 1% individuals with high phenotypic values for the eight tested traits. Additionally, compared with the widely used method RR-BLUP, DeepGS still yields a relative improvement ranging from 1.44% to 65.24%. Through extensive simulation experiments, we also demonstrated the effectiveness and robustness of DeepGS for the absent of outlier individuals and subsets of genotypic markers. Finally, we illustrated the complementarity of DeepGS and RR-BLUP with an ensemble learning approach for further improving prediction performance.AvailabilityDeepGS is provided as an open source R package available at https://github.com/cma2015/DeepGS.

Download Full-text

Soybean Yield Preharvest Prediction Based on Bean Pods and Leaves Image Recognition Using Deep Learning Neural Network Combined With GRNN

Frontiers in Plant Science ◽

10.3389/fpls.2021.791256 ◽

2022 ◽

Vol 12 ◽

Author(s):

Wei Lu ◽

Rongting Du ◽

Pengshuai Niu ◽

Guangnan Xing ◽

Hui Luo ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Image Recognition ◽

Clustering Algorithm ◽

Prediction Models ◽

Soybean Seed ◽

Soybean Plant ◽

Single Shot ◽

Yield Prediction ◽

Soybean Yield

Soybean yield is a highly complex trait determined by multiple factors such as genotype, environment, and their interactions. The earlier the prediction during the growing season the better. Accurate soybean yield prediction is important for germplasm innovation and planting environment factor improvement. But until now, soybean yield has been determined by weight measurement manually after soybean plant harvest which is time-consuming, has high cost and low precision. This paper proposed a soybean yield in-field prediction method based on bean pods and leaves image recognition using a deep learning algorithm combined with a generalized regression neural network (GRNN). A faster region-convolutional neural network (Faster R-CNN), feature pyramid network (FPN), single shot multibox detector (SSD), and You Only Look Once (YOLOv3) were employed for bean pods recognition in which recognition precision and speed were 86.2, 89.8, 80.1, 87.4%, and 13 frames per second (FPS), 7 FPS, 24 FPS, and 39 FPS, respectively. Therefore, YOLOv3 was selected considering both recognition precision and speed. For enhancing detection performance, YOLOv3 was improved by changing IoU loss function, using the anchor frame clustering algorithm, and utilizing the partial neural network structure with which recognition precision increased to 90.3%. In order to improve soybean yield prediction precision, leaves were identified and counted, moreover, pods were further classified as single, double, treble, four, and five seeds types by improved YOLOv3 because each type seed weight varies. In addition, soybean seed number prediction models of each soybean planter were built using PLSR, BP, and GRNN with the input of different type pod numbers and leaf numbers with which prediction results were 96.24, 96.97, and 97.5%, respectively. Finally, the soybean yield of each planter was obtained by accumulating the weight of all soybean pod types and the average accuracy was up to 97.43%. The results show that it is feasible to predict the soybean yield of plants in situ with high precision by fusing the number of leaves and different type soybean pods recognized by a deep neural network combined with GRNN which can speed up germplasm innovation and planting environmental factor optimization.

Download Full-text