Incorporating multi-level CNN and attention mechanism for Chinese clinical named entity recognition

2021 ◽  
Vol 116 ◽  
pp. 103737
Author(s):  
Jun Kong ◽  
Leixin Zhang ◽  
Min Jiang ◽  
Tianshan Liu
Keyword(s):  
Named Entity Recognition ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Named Entity ◽  
Multi Level

scholarly journals Combined Self-Attention Mechanism for Chinese Named Entity Recognition in Military

2019 ◽  
Vol 11 (8) ◽  
pp. 180
Author(s):  
Fei Liao ◽  
Liangli Ma ◽  
Jingjing Pei ◽  
Linshan Tan
Keyword(s):  
Domain Knowledge ◽  
Short Term Memory ◽  
Contextual Information ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
The Self ◽  
Entity Recognition ◽  
Named Entity ◽  
Vector Representations ◽  
The Military

Military named entity recognition (MNER) is one of the key technologies in military information extraction. Traditional methods for the MNER task rely on cumbersome feature engineering and specialized domain knowledge. In order to solve this problem, we propose a method employing a bidirectional long short-term memory (BiLSTM) neural network with a self-attention mechanism to identify the military entities automatically. We obtain distributed vector representations of the military corpus by unsupervised learning and the BiLSTM model combined with the self-attention mechanism is adopted to capture contextual information fully carried by the character vector sequence. The experimental results show that the self-attention mechanism can improve effectively the performance of MNER task. The F-score of the military documents and network military texts identification was 90.15% and 89.34%, respectively, which was better than other models.

scholarly journals CWPC_BiAtt: Character–Word–Position Combined BiLSTM-Attention for Chinese Named Entity Recognition

Information ◽  
2020 ◽  
Vol 11 (1) ◽  
pp. 45 ◽  
Author(s):  
Shardrom Johnson ◽  
Sherlock Shen ◽  
Yuanchen Liu
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Position Information ◽  
Named Entity ◽  
Pos Tagging ◽  
Word Position

Usually taken as linguistic features by Part-Of-Speech (POS) tagging, Named Entity Recognition (NER) is a major task in Natural Language Processing (NLP). In this paper, we put forward a new comprehensive-embedding, considering three aspects, namely character-embedding, word-embedding, and pos-embedding stitched in the order we give, and thus get their dependencies, based on which we propose a new Character–Word–Position Combined BiLSTM-Attention (CWPC_BiAtt) for the Chinese NER task. Comprehensive-embedding via the Bidirectional Llong Short-Term Memory (BiLSTM) layer can get the connection between the historical and future information, and then employ the attention mechanism to capture the connection between the content of the sentence at the current position and that at any location. Finally, we utilize Conditional Random Field (CRF) to decode the entire tagging sequence. Experiments show that CWPC_BiAtt model we proposed is well qualified for the NER task on Microsoft Research Asia (MSRA) dataset and Weibo NER corpus. A high precision and recall were obtained, which verified the stability of the model. Position-embedding in comprehensive-embedding can compensate for attention-mechanism to provide position information for the disordered sequence, which shows that comprehensive-embedding has completeness. Looking at the entire model, our proposed CWPC_BiAtt has three distinct characteristics: completeness, simplicity, and stability. Our proposed CWPC_BiAtt model achieved the highest F-score, achieving the state-of-the-art performance in the MSRA dataset and Weibo NER corpus.

Research on named entity recognition of chinese electronic medical records based on multi-head attention mechanism and character-word information fusion

2021 ◽  
pp. 1-12
Author(s):  
Qinghui Zhang ◽  
Meng Wu ◽  
Pengtao Lv ◽  
Mengya Zhang ◽  
Hongwei Yang
Keyword(s):  
Medical Record ◽  
Electronic Medical Record ◽  
Electronic Medical Records ◽  
Medical Records ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Long Distance ◽  
Named Entity ◽  
The Impact

In the medical field, Named Entity Recognition (NER) plays a crucial role in the process of information extraction through electronic medical records and medical texts. To address the problems of long distance entity, entity confusion, and difficulty in boundary division in the Chinese electronic medical record NER task, we propose a Chinese electronic medical record NER method based on the multi-head attention mechanism and character-word fusion. This method uses a new character-word joint feature representation based on the pre-training model BERT and self-constructed domain dictionary, which can accurately divide the entity boundary and solve the impact of unregistered words. Subsequently, on the basis of the BiLSTM-CRF model, a multi-head attention mechanism is introduced to learn the dependency relationship between remote entities and entity information in different semantic spaces, which effectively improves the performance of the model. Experiments show that our models have better performance and achieves significant improvement compared to baselines. The specific performance is that the F1 value on the Chinese electronic medical record data set reaches 95.22%, which is 2.67%higher than the F1 value of the baseline model.

Aspect-Based and Multi-Level Sentiment Information Applying Contrast Dictionary

2022 ◽  
Vol 14 (1) ◽  
pp. 0-0
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Word Count ◽  
Feedback Information ◽  
Domain Experts ◽  
Named Entity ◽  
Multiple Dimensions ◽  
Multi Level ◽  
Multiple Levels ◽  
Recognition Technique

The customer feedbacks provide alternative and important sources to discover knowledge supporting the marketers and customers to make better decisions. However, the manual process to extract useful information depends on domain experts. This paper focuses on improving the performance of the automatic sentiment information extraction from customer feedbacks. The article proposes a new extraction method that consider multiple dimensions of feedback information, aspect, word, contrast, sentence or phrase, and document levels. The aspect-based sentiment extraction uses a named entity recognition technique to extract the desired aspects of a target product. The aspect-based sentiment combines with sentiment information from multiple levels of feedback contexts resulting in the fused sentiment information improves the extraction performance. We validate the effectiveness by measuring the accuracy of the sentiment and aspect recognition methods comparing with SentiStrength and Word-Count. This information gives some insights on customer satisfaction and can be applied in an alarming tool.

scholarly journals An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records

2019 ◽  
Vol 19 (S5) ◽  
Author(s):  
Luqi Li ◽  
Jie Zhao ◽  
Li Hou ◽  
Yunkai Zhai ◽  
Jinming Shi ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Electronic Medical Records ◽  
Medical Records ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Long Distance ◽  
Named Entity ◽  
Text Features

Abstract Background Clinical named entity recognition (CNER) is important for medical information mining and establishment of high-quality knowledge map. Due to the different text features from natural language and a large number of professional and uncommon clinical terms in Chinese electronic medical records (EMRs), there are still many difficulties in clinical named entity recognition of Chinese EMRs. It is of great importance to eliminate semantic interference and improve the ability of autonomous learning of internal features of the model under the small training corpus. Methods From the perspective of deep learning, we integrated the attention mechanism into neural network, and proposed an improved clinical named entity recognition method for Chinese electronic medical records called BiLSTM-Att-CRF, which could capture more useful information of the context and avoid the problem of missing information caused by long-distance factors. In addition, medical dictionaries and part-of-speech (POS) features were also introduced to improve the performance of the model. Results Based on China Conference on Knowledge Graph and Semantic Computing (CCKS) 2017 and 2018 Chinese EMRs corpus, our BiLSTM-Att-CRF model finally achieved better performance than other widely-used models without additional features(F1-measure of 85.4% in CCKS 2018, F1-measure of 90.29% in CCKS 2017), and achieved the best performance with POS and dictionary features (F1-measure of 86.11% in CCKS 2018, F1-measure of 90.48% in CCKS 2017). In particular, the BiLSTM-Att-CRF model had significant effect on the improvement of Recall. Conclusions Our work preliminarily confirmed the validity of attention mechanism in discovering key information and mining text features, which might provide useful ideas for future research in clinical named entity recognition of Chinese electronic medical records. In the future, we will explore the deeper application of attention mechanism in neural network.

Sign in / Sign up

Export Citation Format

Share Document