Automatic Correction of Real-Word Errors in Spanish Clinical Texts

Real-word errors are characterized by being actual terms in the dictionary. By providing context, real-word errors are detected. Traditional methods to detect and correct such errors are mostly based on counting the frequency of short word sequences in a corpus. Then, the probability of a word being a real-word error is computed. On the other hand, state-of-the-art approaches make use of deep learning models to learn context by extracting semantic features from text. In this work, a deep learning model were implemented for correcting real-word errors in clinical text. Specifically, a Seq2seq Neural Machine Translation Model mapped erroneous sentences to correct them. For that, different types of error were generated in correct sentences by using rules. Different Seq2seq models were trained and evaluated on two corpora: the Wikicorpus and a collection of three clinical datasets. The medicine corpus was much smaller than the Wikicorpus due to privacy issues when dealing with patient information. Moreover, GloVe and Word2Vec pretrained word embeddings were used to study their performance. Despite the medicine corpus being much smaller than the Wikicorpus, Seq2seq models trained on the medicine corpus performed better than those models trained on the Wikicorpus. Nevertheless, a larger amount of clinical text is required to improve the results.

Download Full-text

Deep Learning Methods for Classification of Certain Abnormalities in Echocardiography

Electronics ◽

10.3390/electronics10040495 ◽

2021 ◽

Vol 10 (4) ◽

pp. 495

Author(s):

Imayanmosha Wahlang ◽

Arnab Kumar Maji ◽

Goutam Saha ◽

Prasun Chakrabarti ◽

Michal Jasinski ◽

...

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Support Vector ◽

Variational Autoencoder ◽

Different Types ◽

Static Images ◽

Long Short Term Memory ◽

2D And 3D ◽

Better Than

This article experiments with deep learning methodologies in echocardiogram (echo), a promising and vigorously researched technique in the preponderance field. This paper involves two different kinds of classification in the echo. Firstly, classification into normal (absence of abnormalities) or abnormal (presence of abnormalities) has been done, using 2D echo images, 3D Doppler images, and videographic images. Secondly, based on different types of regurgitation, namely, Mitral Regurgitation (MR), Aortic Regurgitation (AR), Tricuspid Regurgitation (TR), and a combination of the three types of regurgitation are classified using videographic echo images. Two deep-learning methodologies are used for these purposes, a Recurrent Neural Network (RNN) based methodology (Long Short Term Memory (LSTM)) and an Autoencoder based methodology (Variational AutoEncoder (VAE)). The use of videographic images distinguished this work from the existing work using SVM (Support Vector Machine) and also application of deep-learning methodologies is the first of many in this particular field. It was found that deep-learning methodologies perform better than SVM methodology in normal or abnormal classification. Overall, VAE performs better in 2D and 3D Doppler images (static images) while LSTM performs better in the case of videographic images.

Download Full-text

Prediction of Merchandise Sales on E-Commerce Platforms Based on Data Mining and Deep Learning

Scientific Programming ◽

10.1155/2021/2179692 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Xiaoting Yin ◽

Xiaosha Tao

Keyword(s):

Data Mining ◽

Deep Learning ◽

Learning Algorithm ◽

Research Process ◽

Deep Learning Algorithm ◽

Sales Prediction ◽

Different Types ◽

Online Business ◽

Product Sales ◽

Deep Learning Model

Online business has grown exponentially during the last decade, and the industries are focusing on online business more than before. However, just setting up an online store and starting selling might not work. Different machine learning and data mining techniques are needed to know the users’ preferences and know what would be best for business. According to the decision-making needs of online product sales, combined with the influencing factors of online product sales in various industries and the advantages of deep learning algorithm, this paper constructs a sales prediction model suitable for online products and focuses on evaluating the adaptability of the model in different types of online products. In the research process, the full connection model is compared with the training results of CNN, which proves the accuracy and generalization ability of CNN model. By selecting the non-deep learning model as the comparison baseline, the performance advantages of CNN model under different categories of products are proved. In addition, the experiment concludes that the unsupervised pretrained CNN model is more effective and adaptable in sales forecasting.

Download Full-text

Improving the Performance of Deep Learning Model-Based Classification by the Analysis of Local Probability

Complexity ◽

10.1155/2021/5534385 ◽

2021 ◽

Vol 2021 ◽

pp. 1-11

Author(s):

Guanghao Jin ◽

Yixin Hu ◽

Yuming Jiao ◽

Junfang Wen ◽

Qingzeng Song

Keyword(s):

Deep Learning ◽

Local Environment ◽

Learning Model ◽

Human Beings ◽

Training Samples ◽

Local Probability ◽

Validation Set ◽

Deep Learning Model ◽

New Framework ◽

Better Than

Generally, the performance of deep learning-based classification models is highly related to the captured features of training samples. When a sample is not clear or contains a similar number of features of many objects, we cannot easily classify what it is. Actually, human beings classify objects by not only the features but also some information such as the probability of these objects in an environment. For example, when we know further information such as one object has a higher probability in the environment than the others, we can easily give the answer about what is in the sample. We call this kind of probability as local probability as this is related to the local environment. In this paper, we carried out a new framework that is named L-PDL to improve the performance of deep learning based on the analysis of this kind of local probability. Firstly, our method trains the deep learning model on the training set. Then, we can get the probability of objects on each sample by this trained model. Secondly, we get the posterior local probability of objects on the validation set. Finally, this probability conditionally cooperates with the probability of objects on testing samples. We select three popular deep learning models on three real datasets for the evaluation. The experimental results show that our method can obviously improve the performance on the real datasets, which is better than the state-of-the-art methods.

Download Full-text

Research on Methodology of Correlation Analysis of Sci-Tech Literature Based on Deep Learning Technology in the Big Data

Journal of Database Management ◽

10.4018/jdm.2018070104 ◽

2018 ◽

Vol 29 (3) ◽

pp. 67-88 ◽

Cited By ~ 1

Author(s):

Wen Zeng ◽

Hongjiao Xu ◽

Hui Li ◽

Xiang Li

Keyword(s):

Big Data ◽

Deep Learning ◽

Correlation Analysis ◽

Vector Space ◽

Input Word ◽

Learning Technology ◽

Depth Analysis ◽

Different Types ◽

High Level ◽

Deep Learning Model

In the big data era, it is a great challenge to identify high-level abstract features out of a flood of sci-tech literature to achieve in-depth analysis of data. The deep learning technology has developed rapidly and achieved applications in many fields, but has rarely been utilized in the research of sci-tech literature data. This article introduced the presentation method of vector space of terminologies in sci-tech literature based on the deep learning model. It explored and adopted a deep AE model to reduce the dimensionality of input word vector feature. Also put forward is the methodology of correlation analysis of sci-tech literature based on deep learning technology. The experimental results showed that the processing of sci-tech literature data could be simplified into the computation of vectors in the multi-dimensional vector space, and the similarity in vector space could be used to represent similarity in text semantics. The correlation analysis of subject contents between sci-tech literatures of the same or different types can be made using this method.

Download Full-text

Identifying Genomic Islands with Deep Neural Networks

10.1101/525030 ◽

2019 ◽

Cited By ~ 1

Author(s):

Rida Assaf ◽

Fangfang Xia ◽

Rick Stevens

Keyword(s):

Neural Network ◽

Deep Learning ◽

Deep Neural Network ◽

Intrinsic Value ◽

Genomic Islands ◽

Rapid Spread ◽

Different Types ◽

Single Method ◽

Deep Learning Model ◽

Different Sources

AbstractBackgroundHorizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, often classified by the content of their cargo genes or their means of integration and mobility. Various computational methods have been devised to detect different types of GIs, but no single method currently is capable of detecting all GIs.ResultsWe propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools.ConclusionsWe used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was retrained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Download Full-text

Relation Extraction Between Bacteria and Biotopes from Biomedical Texts with Attention Mechanisms and Domain-Specific Contextual Representations

10.1101/686501 ◽

2019 ◽

Author(s):

Amarin Jettakul ◽

Duangdao Wichadakul ◽

Peerapon Vateekul

Keyword(s):

Deep Learning ◽

Relation Extraction ◽

Semantic Features ◽

Significant Information ◽

Domain Specific ◽

Applied Microbiology ◽

Feature Based ◽

Concise Information ◽

Biomedical Texts ◽

Deep Learning Model

AbstractThe Bacteria Biotope (BB) task is biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations have used feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, sections of significant information necessary to define bacterial location relationships are often neglected. In addition, the traditional word embedding used in previous studies may suffer from word ambiguation across linguistic contexts.Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. In order to assess the model’s robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST’16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models.We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model’s performance. Moreover, we analyzed the trade-off between precision and recall in order to choose the proper cut-off to use in real-world applications.

Download Full-text

Deep Learning Model for Spatial Interpolation of Real-Time Seismic Intensity

Seismological Research Letters ◽

10.1785/0220200006 ◽

2020 ◽

Vol 91 (6) ◽

pp. 3433-3443

Author(s):

Ryota Otake ◽

Jun Kurima ◽

Hiroyuki Goto ◽

Sumio Sawada

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Weighted Average ◽

Learning Model ◽

Seismic Intensity ◽

Validation Data ◽

Seismic Stations ◽

Geological Information ◽

Deep Learning Model ◽

Better Than

Abstract Spatial distribution of seismic intensity plays an important role in emergency response during and immediately after an earthquake. In this study, we propose a deep learning model to predict the seismic intensity based on only the observation records at the seismic stations in a surrounding area. The deep learning model is trained using the observation records at both the input and target stations, and no geological information is used. Once the model is developed, for example, using the data from a temporal seismic array, the model can spatially interpolate the seismic intensity from the sparse layout of the seismic stations. The model consists of long short-term memory cells, which are well-established neural network components for time series analysis. We used observed seismograms in 1996 through 2019 at the Kyoshin Network (K-NET) and Kiban–Kyoshin Network (KiK-net) stations located in the northeastern part of Japan. In our deep learning model, approximately 85% of validation data is successfully classified into seismic intensity scales, which is better than adopting either the maximum or weighted average of the input data. We also apply the deep learning model to earthquake early warning (EEW). The model can predict the seismic intensity accurately and provides a long warning time. We concluded that our approach is a possible future solution for increasing the accuracy of EEW.

Download Full-text

Relation extraction between bacteria and biotopes from biomedical texts with attention mechanisms and domain-specific contextual representations

BMC Bioinformatics ◽

10.1186/s12859-019-3217-3 ◽

2019 ◽

Vol 20 (1) ◽

Cited By ~ 1

Author(s):

Amarin Jettakul ◽

Duangdao Wichadakul ◽

Peerapon Vateekul

Keyword(s):

Deep Learning ◽

Relation Extraction ◽

Semantic Features ◽

Domain Specific ◽

Applied Microbiology ◽

Feature Based ◽

Crucial Information ◽

Concise Information ◽

Biomedical Texts ◽

Deep Learning Model

Abstract Background The Bacteria Biotope (BB) task is a biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations conducted the study by applying feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, some parts of crucial information that is required to define bacterial location relationships are often neglected. Moreover, the traditional word-embedding used in previous studies may suffer from word ambiguation across linguistic contexts. Results Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. To assess the model’s robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST’16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models. Conclusions We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model’s performance. Moreover, we analyzed the trade-off between precision and recall to choose the proper cut-off to use in real-world applications.

Download Full-text

Packet Preprocessing in CNN-Based Network Intrusion Detection System

Electronics ◽

10.3390/electronics9071151 ◽

2020 ◽

Vol 9 (7) ◽

pp. 1151 ◽

Cited By ~ 1

Author(s):

Wooyeon Jo ◽

Sungjin Kim ◽

Changhoon Lee ◽

Taeshik Shon

Keyword(s):

Neural Network ◽

Deep Learning ◽

Intrusion Detection ◽

Intrusion Detection System ◽

Detection System ◽

Learning Model ◽

Learning Models ◽

Packet Data ◽

Deep Learning Model ◽

Better Than

The proliferation of various connected platforms, including Internet of things, industrial control systems (ICSs), connected cars, and in-vehicle networks, has resulted in the simultaneous use of multiple protocols and devices. Chaotic situations caused by the usage of different protocols and various types of devices, such as heterogeneous networks, implemented differently by vendors renders the adoption of a flexible security solution difficult, such as recent deep learning-based intrusion detection system (IDS) studies. These studies optimized the deep learning model for their environment to improve performance, but the basic principle of the deep learning model used was not changed, so this can be called a next-generation IDS with a model that has little or no requirements. Some studies proposed IDS based on unsupervised learning technology that does not require labeled data. However, not using available assets, such as network packet data, is a waste of resources. If the security solution considers the role and importance of the devices constituting the network and the security area of the protocol standard by experts, the assets can be well used, but it will no longer be flexible. Most deep learning model-based IDS studies used recurrent neural network (RNN), which is a supervised learning model, because the characteristics of the RNN model, especially when the long-short term memory (LSTM) is incorporated, are better configured to reflect the flow of the packet data stream over time, and thus perform better than other supervised learning models such as convolutional neural network (CNN). However, if the input data induce the CNN’s kernel to sufficiently reflect the network characteristics through proper preprocessing, it could perform better than other deep learning models in the network IDS. Hence, we propose the first preprocessing method, called “direct”, for network IDS that can use the characteristics of the kernel by using the minimum protocol information, field size, and offset. In addition to direct, we propose two more preprocessing techniques called “weighted” and “compressed”. Each requires additional network information; therefore, direct conversion was compared with related studies. Including direct, the proposed preprocessing methods are based on field-to-pixel philosophy, which can reflect the advantages of CNN by extracting the convolutional features of each pixel. Direct is the most intuitive method of applying field-to-pixel conversion to reflect an image’s convolutional characteristics in the CNN. Weighted and compressed are conversion methods used to evaluate the direct method. Consequently, the IDS constructed using a CNN with the proposed direct preprocessing method demonstrated meaningful performance in the NSL-KDD dataset.

Download Full-text

Predicting the interaction biomolecule types for lncRNA: an ensemble deep learning approach

Briefings in Bioinformatics ◽

10.1093/bib/bbaa228 ◽

2020 ◽

Cited By ~ 1

Author(s):

Yu Zhang ◽

Cangzhi Jia ◽

Chee Keong Kwoh

Keyword(s):

Deep Learning ◽

Molecular Mechanisms ◽

Operating Characteristic ◽

Cross Validation ◽

Characteristic Curve ◽

Published Data ◽

Different Types ◽

Precision Recall Curve ◽

Deep Learning Model ◽

Fold Cross Validation

Abstract Long noncoding RNAs (lncRNAs) play significant roles in various physiological and pathological processes via their interactions with biomolecules like DNA, RNA and protein. The existing in silico methods used for predicting the functions of lncRNA mainly rely on calculating the similarity of lncRNA or investigating whether an lncRNA can interact with a specific biomolecule or disease. In this work, we explored the functions of lncRNA from a different perspective: we presented a tool for predicting the interaction biomolecule type for a given lncRNA. For this purpose, we first investigated the main molecular mechanisms of the interactions of lncRNA–RNA, lncRNA–protein and lncRNA–DNA. Then, we developed an ensemble deep learning model: lncIBTP (lncRNA Interaction Biomolecule Type Prediction). This model predicted the interactions between lncRNA and different types of biomolecules. On the 5-fold cross-validation, the lncIBTP achieves average values of 0.7042 in accuracy, 0.7903 and 0.6421 in macro-average area under receiver operating characteristic curve and precision–recall curve, respectively, which illustrates the model effectiveness. Besides, based on the analysis of the collected published data and prediction results, we hypothesized that the characteristics of lncRNAs that interacted with DNA may be different from those that interacted with only RNA.

Download Full-text