Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models

The paper describes two hybrid neural network models for named entity recognition (NER) in texts, as well as results of experiments with them. The first model, namely Bi-LSTM-CRF, is known and used for NER, while the other model named Gated-CNN- CRF is proposed in this work. It combines convolutional neural network (CNN), gated linear units, and conditional random fields (CRF). Both models were tested for NER on three different language datasets, for English, Russian, and Chinese. All resulted scores of precision, recall and F1-measure for both models are close to the state-of-the-art for NER, and for the English dataset CoNLL-2003, Gated-CNN-CRF model achieves 92.66 of F1-measure, outperforming the known result.

Download Full-text

Improving Named Entity Recognition for Biomedical and Patent Data Using Bi-LSTM Deep Neural Network Models

Natural Language Processing and Information Systems - Lecture Notes in Computer Science ◽

10.1007/978-3-030-51310-8_3 ◽

2020 ◽

pp. 25-36

Author(s):

Farag Saad ◽

Hidir Aras ◽

René Hackl-Sommer

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Named Entity Recognition ◽

Network Models ◽

Entity Recognition ◽

Patent Data ◽

Neural Network Models ◽

Named Entity

Download Full-text

Coreference Aware Representation Learning for Neural Named Entity Recognition

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/687 ◽

2019 ◽

Cited By ~ 2

Author(s):

Zeyu Dai ◽

Hongliang Fei ◽

Ping Li

Keyword(s):

Neural Network ◽

State Of The Art ◽

Structural Information ◽

Named Entity Recognition ◽

Network Models ◽

Third Party ◽

Entity Recognition ◽

Neural Network Models ◽

Named Entity ◽

Art Performance

Recent neural network models have achieved state-of-the-art performance on the task of named entity recognition (NER). However, previous neural network models typically treat the input sentences as a linear sequence of words but ignore rich structural information, such as the coreference relations among non-adjacent words, phrases or entities. In this paper, we propose a novel approach to learn coreference-aware word representations for the NER task at the document level. In particular, we enrich the well-known neural architecture ``CNN-BiLSTM-CRF'' with a coreference layer on top of the BiLSTM layer to incorporate coreferential relations. Furthermore, we introduce the coreference regularization to ensure the coreferential entities to share similar representations and consistent predictions within the same coreference cluster. Our proposed model achieves new state-of-the-art performance on two NER benchmarks: CoNLL-2003 and OntoNotes v5.0. More importantly, we demonstrate that our framework does not rely on gold coreference knowledge, and can still work well even when the coreferential relations are generated by a third-party toolkit.

Download Full-text

Named Entity Correction in Neural Machine Translation Using the Attention Alignment Map

Applied Sciences ◽

10.3390/app11157026 ◽

2021 ◽

Vol 11 (15) ◽

pp. 7026

Author(s):

Jangwon Lee ◽

Jungi Lee ◽

Minho Lee ◽

Gil-Jin Jang

Keyword(s):

Machine Translation ◽

Named Entity Recognition ◽

Network Models ◽

Entity Recognition ◽

Neural Network Models ◽

Named Entities ◽

Neural Machine Translation ◽

Named Entity ◽

Input And Output ◽

Artificial Neural Network Models

Neural machine translation (NMT) methods based on various artificial neural network models have shown remarkable performance in diverse tasks and have become mainstream for machine translation currently. Despite the recent successes of NMT applications, a predefined vocabulary is still required, meaning that it cannot cope with out-of-vocabulary (OOV) or rarely occurring words. In this paper, we propose a postprocessing method for correcting machine translation outputs using a named entity recognition (NER) model to overcome the problem of OOV words in NMT tasks. We use attention alignment mapping (AAM) between the named entities of input and output sentences, and mistranslated named entities are corrected using word look-up tables. The proposed method corrects named entities only, so it does not require retraining of existing NMT models. We carried out translation experiments on a Chinese-to-Korean translation task for Korean historical documents, and the evaluation results demonstrated that the proposed method improved the bilingual evaluation understudy (BLEU) score by 3.70 from the baseline.

Download Full-text

Quantifying Uncertainties in Natural Language Processing Tasks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33017322 ◽

2019 ◽

Vol 33 ◽

pp. 7322-7329 ◽

Cited By ~ 1

Author(s):

Yijun Xiao ◽

William Yang Wang

Keyword(s):

Natural Language Processing ◽

Natural Language ◽

Language Processing ◽

Intelligent Systems ◽

Named Entity Recognition ◽

Network Models ◽

Entity Recognition ◽

Neural Network Models ◽

Named Entity ◽

Modeling Uncertainties

Reliable uncertainty quantification is a first step towards building explainable, transparent, and accountable artificial intelligent systems. Recent progress in Bayesian deep learning has made such quantification realizable. In this paper, we propose novel methods to study the benefits of characterizing model and data uncertainties for natural language processing (NLP) tasks. With empirical experiments on sentiment analysis, named entity recognition, and language modeling using convolutional and recurrent neural network models, we show that explicitly modeling uncertainties is not only necessary to measure output confidence levels, but also useful at enhancing model performances in various NLP tasks.

Download Full-text

Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER

Applied Sciences ◽

10.3390/app10217557 ◽

2020 ◽

Vol 10 (21) ◽

pp. 7557

Author(s):

Chirawan Ronran ◽

Seungwoo Lee ◽

Hong Jun Jang

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Conditional Random Field ◽

Named Entity Recognition ◽

Network Models ◽

Word Embedding ◽

Entity Recognition ◽

Neural Network Models ◽

Bidirectional Lstm

Named Entity Recognition (NER) plays a vital role in natural language processing (NLP). Currently, deep neural network models have achieved significant success in NER. Recent advances in NER systems have introduced various feature selections to identify appropriate representations and handle Out-Of-the-Vocabulary (OOV) words. After selecting the features, they are all concatenated at the embedding layer before being fed into a model to label the input sequences. However, when concatenating the features, information collisions may occur and this would cause the limitation or degradation of the performance. To overcome the information collisions, some works tried to directly connect some features to latter layers, which we call the delayed combination and show its effectiveness by comparing it to the early combination. As feature encodings for input, we selected the character-level Convolutional Neural Network (CNN) or Long Short-Term Memory (LSTM) word encoding, the pre-trained word embedding, and the contextual word embedding and additionally designed CNN-based sentence encoding using a dictionary. These feature encodings are combined at early or delayed position of the bidirectional LSTM Conditional Random Field (CRF) model according to each feature’s characteristics. We evaluated the performance of this model on the CoNLL 2003 and OntoNotes 5.0 datasets using the F1 score and compared the delayed combination model with our own implementation of the early combination as well as the previous works. This comparison convinces us that our delayed combination is more effective than the early one and also highly competitive.

Download Full-text

Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

10.1101/256453 ◽

2018 ◽

Cited By ~ 7

Author(s):

Xuan Wang ◽

Yu Zhang ◽

Xiang Ren ◽

Yuhao Zhang ◽

Marinka Zitnik ◽

...

Keyword(s):

State Of The Art ◽

Named Entity Recognition ◽

Network Models ◽

Training Data ◽

Entity Recognition ◽

Neural Network Models ◽

Named Entity ◽

Task Learning ◽

Level Information ◽

Biomedical Named Entity Recognition

AbstractMotivationState-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases. Although recent studies explored using neural network models for BioNER to free experts from manual feature engineering, the performance remains limited by the available training data for each entity type.ResultsWe propose a multi-task learning framework for BioNER to collectively use the training data of different types of entities and improve the performance on each of them. In experiments on 15 benchmark BioNER datasets, our multi-task model achieves substantially better performance compared with state-of-the-art BioNER systems and baseline neural sequence labeling models. Further analysis shows that the large performance gains come from sharing character- and word-level information among relevant biomedical entities across differently labeled corpora.AvailabilityOur source code is available at https://github.com/yuzhimanhua/[email protected], [email protected].

Download Full-text