scholarly journals FastSK: fast sequence analysis with gapped string kernels

2020 ◽  
Vol 36 (Supplement_2) ◽  
pp. i857-i865
Author(s):  
Derrick Blakely ◽  
Eamon Collins ◽  
Ritambhara Singh ◽  
Andrew Norton ◽  
Jack Lanchantin ◽  
...  
Keyword(s):  
Sequence Analysis ◽  
Dna Sequences ◽  
English Language ◽  
Computation Time ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Support Vector ◽  
Homology Detection ◽  
Scalable Algorithm ◽  
String Kernels

Abstract Motivation Gapped k-mer kernels with support vector machines (gkm-SVMs) have achieved strong predictive performance on regulatory DNA sequences on modestly sized training sets. However, existing gkm-SVM algorithms suffer from slow kernel computation time, as they depend exponentially on the sub-sequence feature length, number of mismatch positions, and the task’s alphabet size. Results In this work, we introduce a fast and scalable algorithm for calculating gapped k-mer string kernels. Our method, named FastSK, uses a simplified kernel formulation that decomposes the kernel calculation into a set of independent counting operations over the possible mismatch positions. This simplified decomposition allows us to devise a fast Monte Carlo approximation that rapidly converges. FastSK can scale to much greater feature lengths, allows us to consider more mismatches, and is performant on a variety of sequence analysis tasks. On multiple DNA transcription factor binding site prediction datasets, FastSK consistently matches or outperforms the state-of-the-art gkmSVM-2.0 algorithms in area under the ROC curve, while achieving average speedups in kernel computation of ∼100× and speedups of ∼800× for large feature lengths. We further show that FastSK outperforms character-level recurrent and convolutional neural networks while achieving low variance. We then extend FastSK to 7 English-language medical named entity recognition datasets and 10 protein remote homology detection datasets. FastSK consistently matches or outperforms these baselines. Availability and implementation Our algorithm is available as a Python package and as C++ source code at https://github.com/QData/FastSK Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals FastSK: Fast Sequence Analysis with Gapped String Kernels

2020 ◽  
Author(s):  
Derrick Blakely ◽  
Eamon Collins ◽  
Ritambhara Singh ◽  
Andrew Norton ◽  
Jack Lanchantin ◽  
...  
Keyword(s):  
Sequence Analysis ◽  
Dna Sequences ◽  
English Language ◽  
Computation Time ◽  
Predictive Performance ◽  
Entity Recognition ◽  
Support Vector ◽  
Homology Detection ◽  
Scalable Algorithm ◽  
String Kernels

AbstractGapped k-mer kernels with Support Vector Machines (gkm-SVMs) have achieved strong predictive performance on regulatory DNA sequences on modestly-sized training sets. However, existing gkm-SVM algorithms suffer from slow kernel computation time, as they depend exponentially on the sub-sequence feature-length, number of mismatch positions, and the task’s alphabet size. In this work, we introduce a fast and scalable algorithm for calculating gapped k-mer string kernels. Our method, named FastSK, uses a simplified kernel formulation that decomposes the kernel calculation into a set of independent counting operations over the possible mismatch positions. This simplified decomposition allows us to devise a fast Monte Carlo approximation that rapidly converges. FastSK can scale to much greater feature lengths, allows us to consider more mismatches, and is performant on a variety of sequence analysis tasks. On 10 DNA transcription factor binding site (TFBS) prediction datasets, FastSK consistently matches or outperforms the state-of-the-art gkmSVM-2.0 algorithms in AUC, while achieving average speedups in kernel computation of ∼100× and speedups of ∼800× for large feature lengths. We further show that FastSK outperforms character-level recurrent and convolutional neural networks across all 10 TFBS tasks. We then extend FastSK to 7 English-language medical named entity recognition datasets and 10 protein remote homology detection datasets. FastSK consistently matches or outperforms these baselines. Our algorithm is available as a Python package and as C++ source code1.

PROFILE-BASED STRING KERNELS FOR REMOTE HOMOLOGY DETECTION AND MOTIF EXTRACTION

2005 ◽  
Vol 03 (03) ◽  
pp. 527-550 ◽  
Author(s):  
RUI KUANG ◽  
EUGENE IE ◽  
KE WANG ◽  
KAI WANG ◽  
MAHIRA SIDDIQI ◽  
...  
Keyword(s):  
Structural Features ◽  
Support Vector ◽  
Protein Classification ◽  
Svm Classifier ◽  
Sequence Motifs ◽  
Homology Detection ◽  
String Kernels ◽  
Remote Homology ◽  
Structure Information ◽  
Remote Homology Detection

We introduce novel profile-based string kernels for use with support vector machines (SVMs) for the problems of protein classification and remote homology detection. These kernels use probabilistic profiles, such as those produced by the PSI-BLAST algorithm, to define position-dependent mutation neighborhoods along protein sequences for inexact matching of k-length subsequences ("k-mers") in the data. By use of an efficient data structure, the kernels are fast to compute once the profiles have been obtained. For example, the time needed to run PSI-BLAST in order to build the profiles is significantly longer than both the kernel computation time and the SVM training time. We present remote homology detection experiments based on the SCOP database where we show that profile-based string kernels used with SVM classifiers strongly outperform all recently presented supervised SVM methods. We further examine how to incorporate predicted secondary structure information into the profile kernel to obtain a small but significant performance improvement. We also show how we can use the learned SVM classifier to extract "discriminative sequence motifs" — short regions of the original profile that contribute almost all the weight of the SVM classification score — and show that these discriminative motifs correspond to meaningful structural features in the protein data. The use of PSI-BLAST profiles can be seen as a semi-supervised learning technique, since PSI-BLAST leverages unlabeled data from a large sequence database to build more informative profiles. Recently presented "cluster kernels" give general semi-supervised methods for improving SVM protein classification performance. We show that our profile kernel results also outperform cluster kernels while providing much better scalability to large datasets. Supplementary website:.

scholarly journals GkmExplain: fast and accurate interpretation of nonlinear gapped k-mer SVMs

2019 ◽  
Vol 35 (14) ◽  
pp. i173-i182 ◽  
Author(s):  
Avanti Shrikumar ◽  
Eva Prakash ◽  
Anshul Kundaje
Keyword(s):  
Dna Sequences ◽  
Motif Discovery ◽  
Chromatin Accessibility ◽  
Supplementary Information ◽  
Support Vector ◽  
Computationally Efficient ◽  
Sequence Patterns ◽  
Mutation Impact ◽  
Regulatory Dna

Abstract Summary Support Vector Machines with gapped k-mer kernels (gkm-SVMs) have been used to learn predictive models of regulatory DNA sequence. However, interpreting predictive sequence patterns learned by gkm-SVMs can be challenging. Existing interpretation methods such as deltaSVM, in-silico mutagenesis (ISM) or SHAP either do not scale well or make limiting assumptions about the model that can produce misleading results when the gkm kernel is combined with nonlinear kernels. Here, we propose GkmExplain: a computationally efficient feature attribution method for interpreting predictive sequence patterns from gkm-SVM models that has theoretical connections to the method of Integrated Gradients. Using simulated regulatory DNA sequences, we show that GkmExplain identifies predictive patterns with high accuracy while avoiding pitfalls of deltaSVM and ISM and being orders of magnitude more computationally efficient than SHAP. By applying GkmExplain and a recently developed motif discovery method called TF-MoDISco to gkm-SVM models trained on in vivo transcription factor (TF) binding data, we recover consolidated, non-redundant TF motifs. Mutation impact scores derived using GkmExplain consistently outperform deltaSVM and ISM at identifying regulatory genetic variants from gkm-SVM models of chromatin accessibility in lymphoblastoid cell-lines. Availability and implementation Code and example notebooks to reproduce results are at https://github.com/kundajelab/gkmexplain. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

2021 ◽  
Author(s):  
Yanrong Ji ◽  
Zhihan Zhou ◽  
Han Liu ◽  
Ramana V Davuluri
Keyword(s):  
Dna Sequences ◽  
Regulatory Elements ◽  
Ease Of Use ◽  
Fine Tuning ◽  
Supplementary Information ◽  
Sequence Motifs ◽  
Semantic Relationship ◽  
Accurate Identification ◽  
Conserved Sequence ◽  
Genome Wide

Abstract Motivation Deciphering the language of non-coding DNA is one of the fundamental problems in genome research. Gene regulatory code is highly complex due to the existence of polysemy and distant semantic relationship, which previous informatics methods often fail to capture especially in data-scarce scenarios. Results To address this challenge, we developed a novel pre-trained bidirectional encoder representation, named DNABERT, to capture global and transferrable understanding of genomic DNA sequences based on up and downstream nucleotide contexts. We compared DNABERT to the most widely used programs for genome-wide regulatory elements prediction and demonstrate its ease of use, accuracy and efficiency. We show that the single pre-trained transformers model can simultaneously achieve state-of-the-art performance on prediction of promoters, splice sites and transcription factor binding sites, after easy fine-tuning using small task-specific labeled data. Further, DNABERT enables direct visualization of nucleotide-level importance and semantic relationship within input sequences for better interpretability and accurate identification of conserved sequence motifs and functional genetic variant candidates. Finally, we demonstrate that pre-trained DNABERT with human genome can even be readily applied to other organisms with exceptional performance. We anticipate that the pre-trained DNABERT model can be fined tuned to many other sequence analyses tasks. Availability and implementation The source code, pretrained and finetuned model for DNABERT are available at GitHub (https://github.com/jerryji1993/DNABERT). Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals De-identifying Spanish medical texts - named entity recognition applied to radiology reports

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Irene Pérez-Díez ◽  
Raúl Pérez-Moraga ◽  
Adolfo López-Cerdán ◽  
Jose-Maria Salinas-Serrano ◽  
María de la Iglesia-Vayá
Keyword(s):  
Electronic Health Records ◽  
English Language ◽  
Personal Information ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Medical Texts ◽  
Health Records ◽  
Named Entity ◽  
Radiology Reports ◽  
Electronic Health

Abstract Background Medical texts such as radiology reports or electronic health records are a powerful source of data for researchers. Anonymization methods must be developed to de-identify documents containing personal information from both patients and medical staff. Although currently there are several anonymization strategies for the English language, they are also language-dependent. Here, we introduce a named entity recognition strategy for Spanish medical texts, translatable to other languages. Results We tested 4 neural networks on our radiology reports dataset, achieving a recall of 97.18% of the identifying entities. Alongside, we developed a randomization algorithm to substitute the detected entities with new ones from the same category, making it virtually impossible to differentiate real data from synthetic data. The three best architectures were tested with the MEDDOCAN challenge dataset of electronic health records as an external test, achieving a recall of 69.18%. Conclusions The strategy proposed, combining named entity recognition tasks with randomization of entities, is suitable for Spanish radiology reports. It does not require a big training corpus, thus it could be easily extended to other languages and medical texts, such as electronic health records.

Data-Driven Analysis of Engine Mission Severity Using Non-Dimensional Groups

2021 ◽  
Author(s):  
Tim Brandes ◽  
Stefano Scarso ◽  
Christian Koch ◽  
Stephan Staudacher
Keyword(s):  
Support Vector Machine ◽  
Computation Time ◽  
Principal Component ◽  
General Term ◽  
Support Vector ◽  
Physical Parameters ◽  
Machine Model ◽  
Precision Error ◽  
Operational Conditions ◽  
Wide Range

Abstract A numerical experiment of intentionally reduced complexity is used to demonstrate a method to classify flight missions in terms of the operational severity experienced by the engines. In this proof of concept, the general term of severity is limited to the erosion of the core flow compressor blade and vane leading edges. A Monte Carlo simulation of varying operational conditions generates a required database of 10000 flight missions. Each flight is sampled at a rate of 1 Hz. Eleven measurable or synthesizable physical parameters are deemed to be relevant for the problem. They are reduced to seven universal non-dimensional groups which are averaged for each flight. The application of principal component analysis allows a further reduction to three principal components. They are used to run a support-vector machine model in order to classify the flights. A linear kernel function is chosen for the support-vector machine due to its low computation time compared to other functions. The robustness of the classification approach against measurement precision error is evaluated. In addition, a minimum number of flights required for training and a sensible number of severity classes are documented. Furthermore, the importance to train the algorithms on a sufficiently wide range of operations is presented.

mAHTPred: a sequence-based meta-predictor for improving the prediction of anti-hypertensive peptides using effective feature representation

2018 ◽  
Vol 35 (16) ◽  
pp. 2757-2765 ◽  
Author(s):  
Balachandran Manavalan ◽  
Shaherin Basith ◽  
Tae Hwan Shin ◽  
Leyi Wei ◽  
Gwang Lee
Keyword(s):  
Nearest Neighbor ◽  
Feature Representation ◽  
Superior Performance ◽  
Supplementary Information ◽  
Gradient Boosting ◽  
Support Vector ◽  
Pharmaceutical Drugs ◽  
K Nearest Neighbor ◽  
Feature Descriptors ◽  
Predicted Probability

AbstractMotivationCardiovascular disease is the primary cause of death globally accounting for approximately 17.7 million deaths per year. One of the stakes linked with cardiovascular diseases and other complications is hypertension. Naturally derived bioactive peptides with antihypertensive activities serve as promising alternatives to pharmaceutical drugs. So far, there is no comprehensive analysis, assessment of diverse features and implementation of various machine-learning (ML) algorithms applied for antihypertensive peptide (AHTP) model construction.ResultsIn this study, we utilized six different ML algorithms, namely, Adaboost, extremely randomized tree (ERT), gradient boosting (GB), k-nearest neighbor, random forest (RF) and support vector machine (SVM) using 51 feature descriptors derived from eight different feature encodings for the prediction of AHTPs. While ERT-based trained models performed consistently better than other algorithms regardless of various feature descriptors, we treated them as baseline predictors, whose predicted probability of AHTPs was further used as input features separately for four different ML-algorithms (ERT, GB, RF and SVM) and developed their corresponding meta-predictors using a two-step feature selection protocol. Subsequently, the integration of four meta-predictors through an ensemble learning approach improved the balanced prediction performance and model robustness on the independent dataset. Upon comparison with existing methods, mAHTPred showed superior performance with an overall improvement of approximately 6–7% in both benchmarking and independent datasets.Availability and implementationThe user-friendly online prediction tool, mAHTPred is freely accessible at http://thegleelab.org/mAHTPred.Supplementary informationSupplementary data are available at Bioinformatics online.

scholarly journals Novel Endogenous Retrovirus in Rabbits Previously Reported as Human Retrovirus 5

2002 ◽  
Vol 76 (14) ◽  
pp. 7094-7102 ◽  
Author(s):  
David J. Griffiths ◽  
Cécile Voisset ◽  
Patrick J. W. Venables ◽  
Robin A. Weiss
Keyword(s):  
Sequence Analysis ◽  
Dna Sequences ◽  
Inflammatory Diseases ◽  
Endogenous Retrovirus ◽  
European Rabbit ◽  
Pcr Analysis ◽  
Northern Hybridization ◽  
Human Dna ◽  
Integration Sites ◽  
Human Retrovirus

ABSTRACT Human retrovirus 5 (HRV-5) represented a fragment of a novel retrovirus sequence identified in human RNA and DNA preparations. In this study, the genome of HRV-5 was cloned and sequenced and integration sites were analyzed. Using PCR and Southern hybridization, we showed that HRV-5 is not integrated into human DNA. A survey of other species revealed that HRV-5 is present in the genomic DNA of the European rabbit (Oryctolagus cuniculus) and belongs to an endogenous retrovirus family found in rabbits. The presence of rabbit sequences flanking HRV-5 proviruses in human DNA extracts suggested that rabbit DNA was present in our human extracts, and this was confirmed by PCR analysis that revealed the presence of rabbit mitochondrial DNA sequences in four of five human DNA preparations tested. The origin of the rabbit DNA and HRV-5 in human DNA preparations remains unclear, but laboratory contamination cannot explain the preferential detection of HRV-5 in inflammatory diseases and lymphomas reported previously. This is the first description of a retrovirus genome in rabbits, and sequence analysis shows that it is related to but distinct from A-type retroelements of mice and other rodents. The species distribution of HRV-5 is restricted to rabbits; other species, including other members of the order Lagomorpha, do not contain this sequence. Analysis of HRV-5 expression by Northern hybridization and reverse transcriptase PCR indicates that the virus is transcribed at a low level in many rabbit tissues. In light of these findings we propose that the sequence previously designated HRV-5 should now be denoted RERV-H (for rabbit endogenous retrovirus H).

scholarly journals A Kernel-Based Approach for Biomedical Named Entity Recognition

2013 ◽  
Vol 2013 ◽  
pp. 1-7 ◽  
Author(s):  
Rakesh Patra ◽  
Sujan Kumar Saha
Keyword(s):  
Kernel Function ◽  
Text Processing ◽  
Named Entity Recognition ◽  
Kernel Functions ◽  
Entity Recognition ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Svm Classifier ◽  
Named Entity ◽  
Tree Kernel

Support vector machine (SVM) is one of the popular machine learning techniques used in various text processing tasks including named entity recognition (NER). The performance of the SVM classifier largely depends on the appropriateness of the kernel function. In the last few years a number of task-specific kernel functions have been proposed and used in various text processing tasks, for example, string kernel, graph kernel, tree kernel and so on. So far very few efforts have been devoted to the development of NER task specific kernel. In the literature we found that the tree kernel has been used in NER task only for entity boundary detection or reannotation. The conventional tree kernel is unable to execute the complete NER task on its own. In this paper we have proposed a kernel function, motivated by the tree kernel, which is able to perform the complete NER task. To examine the effectiveness of the proposed kernel, we have applied the kernel function on the openly available JNLPBA 2004 data. Our kernel executes the complete NER task and achieves reasonable accuracy.

scholarly journals EventEpi–A Natural Language Processing Framework for Event-Based Surveillance

2019 ◽  
Author(s):  
Auss Abbood ◽  
Alexander Ullrich ◽  
Rüdiger Busche ◽  
Stéphane Ghozzi
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Web Application ◽  
Fine Tuning ◽  
Entity Recognition ◽  
World Health ◽  
Support Vector ◽  
Event Based ◽  
Processing Framework

AbstractAccording to the World Health Organization (WHO), around 60% of all outbreaks are detected using informal sources. In many public health institutes, including the WHO and the Robert Koch Institute (RKI), dedicated groups of epidemiologists sift through numerous articles and newsletters to detect relevant events. This media screening is one important part of event-based surveillance (EBS). Reading the articles, discussing their relevance, and putting key information into a database is a time-consuming process. To support EBS, but also to gain insights into what makes an article and the event it describes relevant, we developed a natural-language-processing framework for automated information extraction and relevance scoring. First, we scraped relevant sources for EBS as done at RKI (WHO Disease Outbreak News and ProMED) and automatically extracted the articles’ key data: disease, country, date, and confirmed-case count. For this, we performed named entity recognition in two steps: EpiTator, an open-source epidemiological annotation tool, suggested many different possibilities for each. We trained a naive Bayes classifier to find the single most likely one using RKI’s EBS database as labels. Then, for relevance scoring, we defined two classes to which any article might belong: The article is relevant if it is in the EBS database and irrelevant otherwise. We compared the performance of different classifiers, using document and word embeddings. Two of the tested algorithms stood out: The multilayer perceptron performed best overall, with a precision of 0.19, recall of 0.50, specificity of 0.89, F1 of 0.28, and the highest tested index balanced accuracy of 0.46. The support-vector machine, on the other hand, had the highest recall (0.88) which can be of higher interest for epidemiologists. Finally, we integrated these functionalities into a web application called EventEpi where relevant sources are automatically analyzed and put into a database. The user can also provide any URL or text, that will be analyzed in the same way and added to the database. Each of these steps could be improved, in particular with larger labeled datasets and fine-tuning of the learning algorithms. The overall framework, however, works already well and can be used in production, promising improvements in EBS. The source code is publicly available at https://github.com/aauss/EventEpi.

Sign in / Sign up

Export Citation Format

Share Document