STonKGs: A Sophisticated Transformer Trained on Biomedical Text and Knowledge Graphs

The majority of biomedical knowledge is stored in structured databases or as unstructured text in scientific publications. This vast amount of information has led to numerous machine learning-based biological applications using either text through natural language processing (NLP) or structured data through knowledge graph embedding models (KGEMs). However, representations based on a single modality are inherently limited. To generate better representations of biological knowledge, we propose STonKGs, a Sophisticated Transformer trained on biomedical text and Knowledge Graphs. This multimodal Transformer uses combined input sequences of structured information from KGs and unstructured text data from biomedical literature to learn joint representations. First, we pre-trained STonKGs on a knowledge base assembled by the Integrated Network and Dynamical Reasoning Assembler (INDRA) consisting of millions of text-triple pairs extracted from biomedical literature by multiple NLP systems. Then, we benchmarked STonKGs against two baseline models trained on either one of the modalities (i.e., text or KG) across eight different classification tasks, each corresponding to a different biological application. Our results demonstrate that STonKGs outperforms both baselines, especially on the more challenging tasks with respect to the number of classes, improving upon the F1-score of the best baseline by up to 0.083. Additionally, our pre-trained model as well as the model architecture can be adapted to various other transfer learning applications. Finally, the source code and pre-trained STonKGs models are available at https://github.com/stonkgs/stonkgs and https://huggingface.co/stonkgs/stonkgs-150k.

Download Full-text

Recognition of Disease Genetic Information from Unstructured Text Data Based on BiLSTM-CRF for Molecular Mechanisms

Security and Communication Networks ◽

10.1155/2021/6635027 ◽

2021 ◽

Vol 2021 ◽

pp. 1-8

Author(s):

Lejun Gong ◽

Xingxing Zhang ◽

Tianyin Chen ◽

Li Zhang

Keyword(s):

Genetic Information ◽

Molecular Mechanisms ◽

Short Term Memory ◽

Conditional Random Field ◽

Autism Spectrum ◽

Biomedical Literature ◽

Repetitive Behaviour ◽

Biomedical Knowledge ◽

Text Data ◽

Unstructured Text

Disease relevant entities are an important task in mining unstructured text data from the biomedical literature for achieving biomedical knowledge. Autism spectrum disorder (ASD) is a disease related to a neurological and developmental disorder characterized by deficits in communication and social interaction and by repetitive behaviour. However, this kind of disease remains unclear to date. In this study, it identifies entities associated with disease using the machine learning of a computational way from text data collection for molecular mechanisms related to ASD. Entities related to disease are extracted from the biomedical literature related to autism by using deep learning with bidirectional long short-term memory (BiLSTM) and conditional random field (CRF) model. Compared other previous works, the approach is promising for identifying entities related to disease. The proposed approach including five types of molecular entities is evaluated by GENIA corpus to obtain an F-score of 76.81%. The work has extracted 9146 proteins, 145 RNAs, 7680 DNAs, 1058 cell-types, and 981 cell-lines from the autism biomedical literature after removing repeated molecular entities. Finally, we perform GO and KEGG analyses of the test dataset. This study could serve as a reference for further studies on the etiology of disease on the basis of molecular mechanisms and provide a way to explore disease genetic information.

Download Full-text

Citizen Science for Mining the Biomedical Literature

10.1101/038083 ◽

2016 ◽

Cited By ~ 1

Author(s):

Ginger Tsueng ◽

Steven M. Nanis ◽

Jennifer Fouquier ◽

Benjamin M Good ◽

Andrew I Su

Keyword(s):

Information Extraction ◽

Citizen Science ◽

Language Processing ◽

Named Entity Recognition ◽

Biomedical Literature ◽

Entity Recognition ◽

Amazon Mechanical Turk ◽

Biomedical Knowledge ◽

Named Entity ◽

Relationship Extraction

I.AbstractBiomedical literature represents one of the largest and fastest growing collections of unstructured biomedical knowledge. Finding critical information buried in the literature can be challenging. In order to extract information from freeflowing text, researchers need to: 1. identify the entities in the text (named entity recognition), 2. apply a standardized vocabulary to these entities (normalization), and 3. identify how entities in the text are related to one another (relationship extraction). Researchers have primarily approached these information extraction tasks through manual expert curation, and computational methods. We have previously demonstrated that named entity recognition (NER) tasks can be crowdsourced to a group of nonexperts via the paid microtask platform, Amazon Mechanical Turk (AMT); and can dramatically reduce the cost and increase the throughput of biocuration efforts. However, given the size of the biomedical literature even information extraction via paid microtask platforms is not scalable. With our web-based application Mark2Cure (http://mark2cure.org), we demonstrate that NER tasks can also be performed by volunteer citizen scientists with high accuracy. We apply metrics from the Zooniverse Matrices of Citizen Science Success and provide the results here to serve as a basis of comparison for other citizen science projects. Further, we discuss design considerations, issues, and the application of analytics for successfully moving a crowdsourcing workflow from a paid microtask platform to a citizen science platform. To our knowledge, this study is the first application of citizen science to a natural language processing task.

Download Full-text

Identifying and Classifying Goals For Scientific Knowledge

10.1101/2021.05.27.445866 ◽

2021 ◽

Author(s):

Mayla R Boguslav ◽

Nourah M Salem ◽

Elizabeth K White ◽

Sonia M Leach ◽

Lawrence E Hunter

Keyword(s):

Natural Language Processing ◽

Language Processing ◽

Scientific Knowledge ◽

Gold Standard ◽

Source Code ◽

Biomedical Literature ◽

Biomedical Text ◽

Biomedical Text Mining ◽

Text Documents ◽

Prenatal Nutrition

Motivation: Science progresses by posing good questions, yet work in biomedical text mining has not focused on them much. We propose a novel idea for biomedical natural language processing: identifying and characterizing the questions stated in the biomedical literature. Formally, the task is to identify and characterize ignorance statements, statements where scientific knowledge is missing or incomplete. The creation of such technology could have many significant impacts, from the training of PhD students to ranking publications and prioritizing funding based on particular questions of interest. The work presented here is intended as the first step towards these goals. Results: We present a novel ignorance taxonomy driven by the role ignorance statements play in the research, identifying specific goals for future scientific knowledge. Using this taxonomy and reliable annotation guidelines (inter-annotator agreement above 80%), we created a gold standard ignorance corpus of 60 full-text documents from the prenatal nutrition literature with over 10,000 annotations and used it to train classifiers that achieved over 0.80 F1 scores. Availability: Corpus and source code freely available for download at https://github.com/UCDenver-ccp/Ignorance-Question-Work. The source code is implemented in Python.

Download Full-text

Knowledge synthesis of 100 million biomedical documents augments the deep expression profiling of coronavirus receptors

eLife ◽

10.7554/elife.58040 ◽

2020 ◽

Vol 9 ◽

Cited By ~ 21

Author(s):

AJ Venkatakrishnan ◽

Arjun Puranik ◽

Akash Anand ◽

David Zemmour ◽

Xiang Yao ◽

...

Keyword(s):

Data Science ◽

Knowledge Synthesis ◽

Biomedical Literature ◽

Unstructured Data ◽

Rna Seq ◽

Ciliated Cells ◽

Biomedical Knowledge ◽

Transcriptional Signature ◽

Unstructured Text ◽

Club Cells

The COVID-19 pandemic demands assimilation of all biomedical knowledge to decode mechanisms of pathogenesis. Despite the recent renaissance in neural networks, a platform for the real-time synthesis of the exponentially growing biomedical literature and deep omics insights is unavailable. Here, we present the nferX platform for dynamic inference from over 45 quadrillion possible conceptual associations from unstructured text, and triangulation with insights from single-cell RNA-sequencing, bulk RNA-seq and proteomics from diverse tissue types. A hypothesis-free profiling of ACE2 suggests tongue keratinocytes, olfactory epithelial cells, airway club cells and respiratory ciliated cells as potential reservoirs of the SARS-CoV-2 receptor. We find the gut as the putative hotspot of COVID-19, where a maturation correlated transcriptional signature is shared in small intestine enterocytes among coronavirus receptors (ACE2, DPP4, ANPEP). A holistic data science platform triangulating insights from structured and unstructured data holds potential for accelerating the generation of impactful biological insights and hypotheses.

Download Full-text

FamPlex: a resource for entity recognition and relationship resolution of human protein families and complexes in biomedical text mining

10.1101/225698 ◽

2017 ◽

Author(s):

John A Bachman ◽

Benjamin M Gyori ◽

Peter K Sorger

Keyword(s):

Molecular Mechanisms ◽

Named Entity Recognition ◽

Biomedical Literature ◽

Hierarchical Organization ◽

Event Extraction ◽

Entity Recognition ◽

Biomedical Text ◽

Protein Families ◽

Scientific Publications ◽

Named Entity

AbstractBackgroundFor automated reading of scientific publications to extract useful information about molecular mechanisms it is critical that genes, proteins and other entities be correctly associated with uniform identifiers, a process known as named entity linking or “grounding.” Correct grounding is essential for resolving relationships among mined information, curated interaction databases, and biological datasets. The accuracy of this process is largely dependent on the availability of machine-readable resources associating synonyms and abbreviations commonly found in biomedical literature with uniform identifiers.ResultsIn a task involving automated reading of ∼215,000 articles using the REACH event extraction software we found that grounding was disproportionately inaccurate for multi-protein families (e.g., “AKT”) and complexes with multiple subunits (e.g.”NF-κB”). To address this problem we constructed FamPlex, a manually curated resource defining protein families and complexes as they are commonly encountered in biomedical text. In FamPlex the gene-level constituents of families and complexes are defined in a flexible format allowing for multi-level, hierarchical membership. To create FamPlex, text strings corresponding to entities were identified empirically from literature and linked manually to uniform identifiers; these identifiers were also mapped to equivalent entries in multiple related databases. FamPlex also includes curated prefix and suffix patterns that improve named entity recognition and event extraction. Evaluation of REACH extractions on a test corpus of ∼54,000 articles showed that FamPlex significantly increased grounding accuracy for families and complexes (from 15% to 71%). The hierarchical organization of entities in FamPlex also made it possible to integrate otherwise unconnected mechanistic information across families, subfamilies, and individual proteins. Applications of FamPlex to the TRIPS/DRUM reading system and the Biocreative VI Bioentity Normalization Task dataset demonstrated the utility of FamPlex in other settings.ConclusionFamPlex is an effective resource for improving named entity recognition, grounding, and relationship resolution in automated reading of biomedical text. The content in FamPlex is available in both tabular and Open Biomedical Ontology formats at https://github.com/sorgerlab/famplex under the Creative Commons CC0 license and has been integrated into the TRIPS/DRUM and REACH reading systems.

Download Full-text

TO THE QUESTION OF THE SYSTEMATIC NATURE OF ELECTRONIC EDUCATIONAL MATERIALS FOR HIGHER EDUCATION

Informatics and Education ◽

10.32517/0234-0453-2019-34-7-54-62 ◽

2019 ◽

pp. 54-62

Author(s):

K. R. Ovchinnikova

Keyword(s):

Learning Process ◽

Information Technologies ◽

Electronic Materials ◽

Subject Area ◽

Design Activity ◽

Learning Technology ◽

Educational Materials ◽

Scientific Publications ◽

Structured Information ◽

Single Complex

The relevance of the issue under consideration in the article is connected with the confusion in scientific publications of the concepts of “electronic educational materials” and “electronic educational resources”. The article discusses the concept of “electronic educational materials” from the perspective of general systems theory. And their system character is proved. This allows them to be represented as a single complex of structured information of a specific subject area and didactic materials. These didactic materials support the learning process at all stages of its didactic cycle in accordance with the chosen learning technology based on the didactic capabilities of information technologies. It is concluded that the system of high school electronic materials allows to expand the boundaries of the design activity of the teacher, provide management of the student’s thinking activity, to implement a competence approach to the learning process at university

Download Full-text

Voice of the Customer Oriented New Product Synthesis Over Knowledge Graphs

Volume 1A: 38th Computers and Information in Engineering Conference ◽

10.1115/detc2018-85909 ◽

2018 ◽

Author(s):

Feiwei Qin ◽

Hairui Xu ◽

Weicheng Zhang ◽

Lin Yuan ◽

Ming Li ◽

...

Keyword(s):

Language Processing ◽

Online Shopping ◽

Specific Aspect ◽

Product Reviews ◽

Product Knowledge ◽

Online Data ◽

Challenges And Opportunities ◽

Knowledge Graphs ◽

The Voice ◽

Voice Of The Customer

The online shopping has been much easier and popular, and meanwhile brings new challenges and opportunities to the field of product design and marketing sale. On one hand, product manufacturers find it challenging to produce new popularly accepted products to meet the customers’ needs; on the other hand, end customers usually feel it difficult to buy ideal goods that they really want, even if navigating a huge amount of commodities. There are indeed a ‘communication gap’ between the customers and manufacturers. As an effort to partially resolve the issue, this paper proposes a novel product synthesis approach from ‘voice of the customer’ over product knowledge graphs. Here the voice of customers mainly refer to the buyers’ product reviews from online shopping platforms or blogs, while the product knowledge graph is constructed containing professional hierarchical product knowledge on its properties based on ontological models. Using the technologies of natural language processing, we first extract the customs’ polarities on each specific aspect of a product, which are then transited to design requirements on the product’s design components. Based on the requirement extractions, and the pre-built product knowledge, semantic web and reasoning techniques are utilized to synthesize a novel product that meets more customer needs. Typical case studies on mobile phones from raw online data demonstrate the proposed approach’s performance.

Download Full-text

GRAPHICAL INDUCTION OF QUALIFIED MEDICAL KNOWLEDGE

International Journal of Semantic Computing ◽

10.1142/s1793351x13400126 ◽

2013 ◽

Vol 07 (04) ◽

pp. 377-405 ◽

Cited By ~ 1

Author(s):

TRAVIS GOODWIN ◽

SANDA M. HARABAGIU

Keyword(s):

Language Processing ◽

Clinical Data ◽

Medical Knowledge ◽

Knowledge Bases ◽

Clinical Knowledge ◽

Biomedical Knowledge ◽

Clinical Text ◽

Relational Information ◽

Processing Techniques ◽

Medical Concepts

The introduction of electronic medical records (EMRs) enabled the access of unprecedented volumes of clinical data, both in structured and unstructured formats. A significant amount of this clinical data is expressed within the narrative portion of the EMRs, requiring natural language processing techniques to unlock the medical knowledge referred to by physicians. This knowledge, derived from the practice of medical care, complements medical knowledge already encoded in various structured biomedical ontologies. Moreover, the clinical knowledge derived from EMRs also exhibits relational information between medical concepts, derived from the cohesion property of clinical text, which is an attractive attribute that is currently missing from the vast biomedical knowledge bases. In this paper, we describe an automatic method of generating a graph of clinically related medical concepts by considering the belief values associated with those concepts. The belief value is an expression of the clinician's assertion that the concept is qualified as present, absent, suggested, hypothetical, ongoing, etc. Because the method detailed in this paper takes into account the hedging used by physicians when authoring EMRs, the resulting graph encodes qualified medical knowledge wherein each medical concept has an associated assertion (or belief value) and such qualified medical concepts are spanned by relations of different strengths, derived from the clinical contexts in which concepts are used. In this paper, we discuss the construction of a qualified medical knowledge graph (QMKG) and treat it as a BigData problem addressed by using MapReduce for deriving the weighted edges of the graph. To be able to assess the value of the QMKG, we demonstrate its usage for retrieving patient cohorts by enabling query expansion that produces greatly enhanced results against state-of-the-art methods.

Download Full-text

Ekstraksi Informasi Halaman Web Menggunakan Pendekatan Bootstrapping pada Ontology-Based Information Extraction

IJCCS (Indonesian Journal of Computing and Cybernetics Systems) ◽

10.22146/ijccs.7540 ◽

2015 ◽

Vol 9 (2) ◽

pp. 111 ◽

Cited By ~ 1

Author(s):

Erma Susanti ◽

Khabib Mustofa

Keyword(s):

Information Extraction ◽

Language Processing ◽

Semantic Content ◽

Extraction Process ◽

Web Pages ◽

Structured Information ◽

Improved Performance ◽

Types Of Information ◽

Unstructured Information

AbstrakEkstraksi informasi merupakan suatu bidang ilmu untuk pengolahan bahasa alami, dengan cara mengubah teks tidak terstruktur menjadi informasi dalam bentuk terstruktur. Berbagai jenis informasi di Internet ditransmisikan secara tidak terstruktur melalui website, menyebabkan munculnya kebutuhan akan suatu teknologi untuk menganalisa teks dan menemukan pengetahuan yang relevan dalam bentuk informasi terstruktur. Contoh informasi tidak terstruktur adalah informasi utama yang ada pada konten halaman web. Bermacam pendekatan untuk ekstraksi informasi telah dikembangkan oleh berbagai peneliti, baik menggunakan metode manual atau otomatis, namun masih perlu ditingkatkan kinerjanya terkait akurasi dan kecepatan ekstraksi. Pada penelitian ini diusulkan suatu penerapan pendekatan ekstraksi informasi dengan mengkombinasikan pendekatan bootstrapping dengan Ontology-based Information Extraction (OBIE). Pendekatan bootstrapping dengan menggunakan sedikit contoh data berlabel, digunakan untuk memimalkan keterlibatan manusia dalam proses ekstraksi informasi, sedangkan penggunakan panduan ontologi untuk mengekstraksi classes (kelas), properties dan instance digunakan untuk menyediakan konten semantik untuk web semantik. Pengkombinasian kedua pendekatan tersebut diharapkan dapat meningkatan kecepatan proses ekstraksi dan akurasi hasil ekstraksi. Studi kasus untuk penerapan sistem ekstraksi informasi menggunakan dataset “LonelyPlanet”. Kata kunci—Ekstraksi informasi, ontologi, bootstrapping, Ontology-Based Information Extraction, OBIE, kinerja Abstract Information extraction is a field study of natural language processing by converting unstructured text into structured information. Several types of information on the Internet is transmitted through unstructured information via websites, led to emergence of the need a technology to analyze text and found relevant knowledge into structured information. For example of unstructured information is existing main information on the content of web pages. Various approaches for information extraction have been developed by many researchers, either using manual or automatic method, but still need to be improved performance related accuracy and speed of extraction. This research proposed an approach of information extraction that combines bootstrapping approach with Ontology-Based Information Extraction (OBIE). Bootstrapping approach using small seed of labelled data, is used to minimize human intervention on information extraction process, while the use of guide ontology for extracting classes, properties and instances, using for provide semantic content for semantic web. Combining both approaches expected to increase speed of extraction process and accuracy of extraction results. Case study to apply information extraction system using “LonelyPlanet” datasets. Keywords— Information extraction, ontology, bootstrapping, Ontology-Based Information Extraction, OBIE, performance

Download Full-text

Exploring the Adaptation of Recurrent Neural Network Approaches for Extracting Drug–Drug Interactions from Biomedical Text

International Journal of Machine Learning and Computing ◽

10.18178/ijmlc.2021.11.4.1046 ◽

2021 ◽

Vol 11 (4) ◽

pp. 267-273

Author(s):

Wen-Juan Hou ◽

◽

Bamfa Ceesay

Keyword(s):

Text Processing ◽

Named Entity Recognition ◽

Event Extraction ◽

Entity Recognition ◽

Biomedical Text ◽

Automatic Extraction ◽

Named Entity ◽

Structured Information ◽

Network Approaches ◽

Form Information

Information extraction (IE) is the process of automatically identifying structured information from unstructured or partially structured text. IE processes can involve several activities, such as named entity recognition, event extraction, relationship discovery, and document classification, with the overall goal of translating text into a more structured form. Information on the changes in the effect of a drug, when taken in combination with a second drug, is known as drug–drug interaction (DDI). DDIs can delay, decrease, or enhance absorption of drugs and thus decrease or increase their efficacy or cause adverse effects. Recent research trends have shown several adaptation of recurrent neural networks (RNNs) from text. In this study, we highlight significant challenges of using RNNs in biomedical text processing and propose automatic extraction of DDIs aiming at overcoming some challenges. Our results show that the system is competitive against other systems for the task of extracting DDIs.

Download Full-text