scholarly journals Applying Citizen Science to Gene, Drug, and Disease Relationship Extraction from Biomedical Abstracts

2019 ◽  
Author(s):  
Ginger Tsueng ◽  
Max Nanis ◽  
Jennifer T Fouquier ◽  
Michael Mayers ◽  
Benjamin M Good ◽  
...  
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Computational Methods ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Named Entity ◽  
Relationship Extraction

Abstract Motivation Biomedical literature is growing at a rate that outpaces our ability to harness the knowledge contained therein. To mine valuable inferences from the large volume of literature, many researchers use information extraction algorithms to harvest information in biomedical texts. Information extraction is usually accomplished via a combination of manual expert curation and computational methods. Advances in computational methods usually depends on the time-consuming generation of gold standards by a limited number of expert curators. Citizen science is public participation in scientific research. We previously found that citizen scientists are willing and capable of performing named entity recognition of disease mentions in biomedical abstracts, but did not know if this was true with relationship extraction. Results In this paper, we introduce the Relationship Extraction Module of the web-based application Mark2Cure and demonstrate that citizen scientists can perform relationship extraction. We confirm the importance of accurate named entity recognition on user performance of relationship extraction and identify design issues that impacted data quality. We find that the data generated by citizen scientists can be used to identify relationship types not currently available in the Mark2Cure Relationship Extraction Module. We compare the citizen science-generated data with algorithm-mined data and identify ways in which the two approaches may complement one another. We also discuss opportunities for future improvement of this system, as well as the potential synergies between citizen science, manual biocuration, and natural language processing. Availability Mark2Cure platform: https://mark2cure.org. Mark2Cure source code: https://github.com/sulab/mark2cure Data and analysis code for this paper: https://github.com/gtsueng/M2C_rel_nb Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Applying Citizen Science to Gene, Drug, Disease Relationship Extraction from Biomedical Abstracts

2019 ◽  
Author(s):  
Ginger Tsueng ◽  
Max Nanis ◽  
Jennifer T. Fouquier ◽  
Michael Mayers ◽  
Benjamin M. Good ◽  
...  
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Computational Methods ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Named Entity ◽  
Relationship Extraction ◽  
Biomedical Texts

AbstractBiomedical literature is growing at a rate that outpaces our ability to harness the knowledge contained therein. In order to mine valuable inferences from the large volume of literature, many researchers have turned to information extraction algorithms to harvest information in biomedical texts. Information extraction is usually accomplished via a combination of manual expert curation and computational methods. Advances in computational methods usually depends on the generation of gold standards by a limited number of expert curators. This process can be time consuming and represents an area of biomedical research that is ripe for exploration with citizen science. Citizen scientists have been previously found to be willing and capable of performing named entity recognition of disease mentions in biomedical abstracts, but it was uncertain whether or not the same could be said of relationship extraction. Relationship extraction requires training on identifying named entities as well as a deeper understanding of how different entity types can relate to one another. Here, we used the web-based application Mark2Cure (https://mark2cure.org) to demonstrate that citizen scientists can perform relationship extraction and confirm the importance of accurate named entity recognition on this task. We also discuss opportunities for future improvement of this system, as well as the potential synergies between citizen science, manual biocuration, and natural language processing.

scholarly journals Citizen Science for Mining the Biomedical Literature

2016 ◽  
Author(s):  
Ginger Tsueng ◽  
Steven M. Nanis ◽  
Jennifer Fouquier ◽  
Benjamin M Good ◽  
Andrew I Su
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Amazon Mechanical Turk ◽  
Biomedical Knowledge ◽  
Named Entity ◽  
Relationship Extraction

I.AbstractBiomedical literature represents one of the largest and fastest growing collections of unstructured biomedical knowledge. Finding critical information buried in the literature can be challenging. In order to extract information from freeflowing text, researchers need to: 1. identify the entities in the text (named entity recognition), 2. apply a standardized vocabulary to these entities (normalization), and 3. identify how entities in the text are related to one another (relationship extraction). Researchers have primarily approached these information extraction tasks through manual expert curation, and computational methods. We have previously demonstrated that named entity recognition (NER) tasks can be crowdsourced to a group of nonexperts via the paid microtask platform, Amazon Mechanical Turk (AMT); and can dramatically reduce the cost and increase the throughput of biocuration efforts. However, given the size of the biomedical literature even information extraction via paid microtask platforms is not scalable. With our web-based application Mark2Cure (http://mark2cure.org), we demonstrate that NER tasks can also be performed by volunteer citizen scientists with high accuracy. We apply metrics from the Zooniverse Matrices of Citizen Science Success and provide the results here to serve as a basis of comparison for other citizen science projects. Further, we discuss design considerations, issues, and the application of analytics for successfully moving a crowdsourcing workflow from a paid microtask platform to a citizen science platform. To our knowledge, this study is the first application of citizen science to a natural language processing task.

scholarly journals Biomedical named entity recognition and linking datasets: survey and our recent development

2020 ◽  
Vol 21 (6) ◽  
pp. 2219-2238 ◽  
Author(s):  
Ming-Siang Huang ◽  
Po-Ting Lai ◽  
Pei-Yen Lin ◽  
Yu-Ting You ◽  
Richard Tzong-Han Tsai ◽  
...  
Keyword(s):  
Language Processing ◽  
Protein Interaction ◽  
Named Entity Recognition ◽  
Paper Analysis ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Protein Protein Interaction ◽  
Named Entity ◽  
Biomedical Named Entity Recognition

Abstract Natural language processing (NLP) is widely applied in biological domains to retrieve information from publications. Systems to address numerous applications exist, such as biomedical named entity recognition (BNER), named entity normalization (NEN) and protein–protein interaction extraction (PPIE). High-quality datasets can assist the development of robust and reliable systems; however, due to the endless applications and evolving techniques, the annotations of benchmark datasets may become outdated and inappropriate. In this study, we first review commonlyused BNER datasets and their potential annotation problems such as inconsistency and low portability. Then, we introduce a revised version of the JNLPBA dataset that solves potential problems in the original and use state-of-the-art named entity recognition systems to evaluate its portability to different kinds of biomedical literature, including protein–protein interaction and biology events. Lastly, we introduce an ensembled biomedical entity dataset (EBED) by extending the revised JNLPBA dataset with PubMed Central full-text paragraphs, figure captions and patent abstracts. This EBED is a multi-task dataset that covers annotations including gene, disease and chemical entities. In total, it contains 85000 entity mentions, 25000 entity mentions with database identifiers and 5000 attribute tags. To demonstrate the usage of the EBED, we review the BNER track from the AI CUP Biomedical Paper Analysis challenge. Availability: The revised JNLPBA dataset is available at https://iasl-btm.iis.sinica.edu.tw/BNER/Content/Re vised_JNLPBA.zip. The EBED dataset is available at https://iasl-btm.iis.sinica.edu.tw/BNER/Content/AICUP _EBED_dataset.rar. Contact: Email: [email protected], Tel. 886-3-4227151 ext. 35203, Fax: 886-3-422-2681 Email: [email protected], Tel. 886-2-2788-3799 ext. 2211, Fax: 886-2-2782-4814 Supplementary information: Supplementary data are available at Briefings in Bioinformatics online.

scholarly journals A Survey on Recent Named Entity Recognition and Relationship Extraction Techniques on Clinical Texts

2021 ◽  
Vol 11 (18) ◽  
pp. 8319
Author(s):  
Priyankar Bose ◽  
Sriram Srinivasan ◽  
William C. Sleeman ◽  
Jatinder Palta ◽  
Rishabh Kapoor ◽  
...  
Keyword(s):  
Information Extraction ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Future Research ◽  
Clinical Text ◽  
Named Entity ◽  
Relationship Extraction ◽  
New Information ◽  
Comprehensive Survey ◽  
Future Research Directions

Significant growth in Electronic Health Records (EHR) over the last decade has provided an abundance of clinical text that is mostly unstructured and untapped. This huge amount of clinical text data has motivated the development of new information extraction and text mining techniques. Named Entity Recognition (NER) and Relationship Extraction (RE) are key components of information extraction tasks in the clinical domain. In this paper, we highlight the present status of clinical NER and RE techniques in detail by discussing the existing proposed NLP models for the two tasks and their performances and discuss the current challenges. Our comprehensive survey on clinical NER and RE encompass current challenges, state-of-the-art practices, and future directions in information extraction from clinical text. This is the first attempt to discuss both of these interrelated topics together in the clinical context. We identified many research articles published based on different approaches and looked at applications of these tasks. We also discuss the evaluation metrics that are used in the literature to measure the effectiveness of the two these NLP methods and future research directions.

scholarly journals Combining word embeddings to extract chemical and drug entities in biomedical literature

2021 ◽  
Vol 22 (S1) ◽  
Author(s):  
Pilar López-Úbeda ◽  
Manuel Carlos Díaz-Galiano ◽  
L. Alfonso Ureña-López ◽  
M. Teresa Martín-Valdivia
Keyword(s):  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Levenshtein Distance ◽  
Word Embeddings ◽  
Snomed Ct ◽  
Named Entity ◽  
Indexing System ◽  
The One

Abstract Background Natural language processing (NLP) and text mining technologies for the extraction and indexing of chemical and drug entities are key to improving the access and integration of information from unstructured data such as biomedical literature. Methods In this paper we evaluate two important tasks in NLP: the named entity recognition (NER) and Entity indexing using the SNOMED-CT terminology. For this purpose, we propose a combination of word embeddings in order to improve the results obtained in the PharmaCoNER challenge. Results For the NER task we present a neural network composed of BiLSTM with a CRF sequential layer where different word embeddings are combined as an input to the architecture. A hybrid method combining supervised and unsupervised models is used for the concept indexing task. In the supervised model, we use the training set to find previously trained concepts, and the unsupervised model is based on a 6-step architecture. This architecture uses a dictionary of synonyms and the Levenshtein distance to assign the correct SNOMED-CT code. Conclusion On the one hand, the combination of word embeddings helps to improve the recognition of chemicals and drugs in the biomedical literature. We achieved results of 91.41% for precision, 90.14% for recall, and 90.77% for F1-score using micro-averaging. On the other hand, our indexing system achieves a 92.67% F1-score, 92.44% for recall, and 92.91% for precision. With these results in a final ranking, we would be in the first position.

scholarly journals ChEMU 2020: Natural Language Processing Methods Are Effective for Information Extraction From Chemical Patents

2021 ◽  
Vol 6 ◽  
Author(s):  
Jiayuan He ◽  
Dat Quoc Nguyen ◽  
Saber A. Akhondi ◽  
Christian Druckenbrodt ◽  
Camilo Thorne ◽  
...  
Keyword(s):  
Information Extraction ◽  
Chemical Reactions ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Event Extraction ◽  
Entity Recognition ◽  
Named Entity ◽  
Fundamental Information ◽  
Essential Chemical ◽  
Reaction Processes

Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1) chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2) event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.

scholarly journals An Annotated Corpus of Crime-Related Portuguese Documents for NLP and Machine Learning Processing

Data ◽  
2021 ◽  
Vol 6 (7) ◽  
pp. 71
Author(s):  
Gonçalo Carnaz ◽  
Mário Antunes ◽  
Vitor Beires Nogueira
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Automatic Identification ◽  
Named Entities ◽  
Related Data ◽  
Named Entity ◽  
Chain Of Custody ◽  
Evidence Collection

Criminal investigations collect and analyze the facts related to a crime, from which the investigators can deduce evidence to be used in court. It is a multidisciplinary and applied science, which includes interviews, interrogations, evidence collection, preservation of the chain of custody, and other methods and techniques of investigation. These techniques produce both digital and paper documents that have to be carefully analyzed to identify correlations and interactions among suspects, places, license plates, and other entities that are mentioned in the investigation. The computerized processing of these documents is a helping hand to the criminal investigation, as it allows the automatic identification of entities and their relations, being some of which difficult to identify manually. There exists a wide set of dedicated tools, but they have a major limitation: they are unable to process criminal reports in the Portuguese language, as an annotated corpus for that purpose does not exist. This paper presents an annotated corpus, composed of a collection of anonymized crime-related documents, which were extracted from official and open sources. The dataset was produced as the result of an exploratory initiative to collect crime-related data from websites and conditioned-access police reports. The dataset was evaluated and a mean precision of 0.808, recall of 0.722, and F1-score of 0.733 were obtained with the classification of the annotated named-entities present in the crime-related documents. This corpus can be employed to benchmark Machine Learning (ML) and Natural Language Processing (NLP) methods and tools to detect and correlate entities in the documents. Some examples are sentence detection, named-entity recognition, and identification of terms related to the criminal domain.

Towards corpus and model: Hierarchical structured-attention-based features for Indonesian named entity recognition

2021 ◽  
pp. 1-12
Author(s):  
Yingwen Fu ◽  
Nankai Lin ◽  
Xiaotian Lin ◽  
Shengyi Jiang
Keyword(s):  
Language Processing ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Language Models ◽  
Neural Models ◽  
Performance Models ◽  
Named Entity ◽  
High Resource ◽  
Benchmark Datasets

Named entity recognition (NER) is fundamental to natural language processing (NLP). Most state-of-the-art researches on NER are based on pre-trained language models (PLMs) or classic neural models. However, these researches are mainly oriented to high-resource languages such as English. While for Indonesian, related resources (both in dataset and technology) are not yet well-developed. Besides, affix is an important word composition for Indonesian language, indicating the essentiality of character and token features for token-wise Indonesian NLP tasks. However, features extracted by currently top-performance models are insufficient. Aiming at Indonesian NER task, in this paper, we build an Indonesian NER dataset (IDNER) comprising over 50 thousand sentences (over 670 thousand tokens) to alleviate the shortage of labeled resources in Indonesian. Furthermore, we construct a hierarchical structured-attention-based model (HSA) for Indonesian NER to extract sequence features from different perspectives. Specifically, we use an enhanced convolutional structure as well as an enhanced attention structure to extract deeper features from characters and tokens. Experimental results show that HSA establishes competitive performance on IDNER and three benchmark datasets.

scholarly journals Obtaining Knowledge in Pathology Reports Through a Natural Language Processing Approach With Classification, Named-Entity Recognition, and Relation-Extraction Heuristics

2019 ◽  
pp. 1-8 ◽  
Author(s):  
Tomasz Oliwa ◽  
Steven B. Maron ◽  
Leah M. Chase ◽  
Samantha Lomnicki ◽  
Daniel V.T. Catenacci ◽  
...  
Keyword(s):  
Machine Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Classification Model ◽  
Supervised Machine Learning ◽  
Named Entity ◽  
Pathology Reports

PURPOSE Robust institutional tumor banks depend on continuous sample curation or else subsequent biopsy or resection specimens are overlooked after initial enrollment. Curation automation is hindered by semistructured free-text clinical pathology notes, which complicate data abstraction. Our motivation is to develop a natural language processing method that dynamically identifies existing pathology specimen elements necessary for locating specimens for future use in a manner that can be re-implemented by other institutions. PATIENTS AND METHODS Pathology reports from patients with gastroesophageal cancer enrolled in The University of Chicago GI oncology tumor bank were used to train and validate a novel composite natural language processing-based pipeline with a supervised machine learning classification step to separate notes into internal (primary review) and external (consultation) reports; a named-entity recognition step to obtain label (accession number), location, date, and sublabels (block identifiers); and a results proofreading step. RESULTS We analyzed 188 pathology reports, including 82 internal reports and 106 external consult reports, and successfully extracted named entities grouped as sample information (label, date, location). Our approach identified up to 24 additional unique samples in external consult notes that could have been overlooked. Our classification model obtained 100% accuracy on the basis of 10-fold cross-validation. Precision, recall, and F1 for class-specific named-entity recognition models show strong performance. CONCLUSION Through a combination of natural language processing and machine learning, we devised a re-implementable and automated approach that can accurately extract specimen attributes from semistructured pathology notes to dynamically populate a tumor registry.

Sign in / Sign up

Export Citation Format

Share Document