scholarly journals Dynamic Transfer Learning for Named Entity Recognition

2019 ◽  
pp. 69-81 ◽  
Author(s):  
Parminder Bhatia ◽  
Kristjan Arumae ◽  
E. Busra Celikkaya
Keyword(s):  
Transfer Learning ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity

scholarly journals MasakhaNER: Named Entity Recognition for African Languages

2021 ◽  
Vol 9 ◽  
pp. 1116-1131
Author(s):  
David Ifeoluwa Adelani ◽  
Jade Abbott ◽  
Graham Neubig ◽  
Daniel D’souza ◽  
Julia Kreutzer ◽  
...  
Keyword(s):  
Transfer Learning ◽  
State Of The Art ◽  
Empirical Evaluation ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Future Research ◽  
African Continent ◽  
High Quality ◽  
African Languages ◽  
Named Entity

Abstract We take a step towards addressing the under- representation of the African continent in NLP research by bringing together different stakeholders to create the first large, publicly available, high-quality dataset for named entity recognition (NER) in ten African languages. We detail the characteristics of these languages to help researchers and practitioners better understand the challenges they pose for NER tasks. We analyze our datasets and conduct an extensive empirical evaluation of state- of-the-art methods across both supervised and transfer learning settings. Finally, we release the data, code, and models to inspire future research on African NLP.1

TL-NER: A Transfer Learning Model for Chinese Named Entity Recognition

2019 ◽  
Vol 22 (6) ◽  
pp. 1291-1304 ◽  
Author(s):  
DunLu Peng ◽  
YinRui Wang ◽  
Cong Liu ◽  
Zhang Chen
Keyword(s):  
Transfer Learning ◽  
Named Entity Recognition ◽  
Learning Model ◽  
Entity Recognition ◽  
Named Entity

Disease Named Entity Recognition in Medical Records: Analysis of Transfer Learning between Different Hospital Departments (Preprint)

2021 ◽  
Author(s):  
Jong-Kang Lee ◽  
Jue-Ni Huang ◽  
Kun-Ju Lin ◽  
Richard Tzong-Han Tsai
Keyword(s):  
Transfer Learning ◽  
Medical Records ◽  
Named Entity Recognition ◽  
Model Performance ◽  
Entity Recognition ◽  
Similarity Metrics ◽  
Medical Department ◽  
Named Entity ◽  
Data Source ◽  
Mimic Iii

BACKGROUND Electronic records provide rich clinical information for biomedical text mining. However, a system developed on one hospital department may not generalize to other departments. Here, we use hospital medical records as a research data source and explore the heterogeneous problem posed by different hospital departments. OBJECTIVE We use MIMIC-III hospital medical records as the research data source. We collaborate with medical experts to annotate the data, with 328 records being included in analyses. Disease named entity recognition (NER), which helps medical experts in consolidating diagnoses, is undertaken as a case study. METHODS To compare heterogeneity of medical records across departments, we access text from multiple departments and employ the similarity metrics. We apply transfer learning to NER in different departments’ records and test the correlation between performance and similarity metrics. We use TF-IDF cosine similarity of the named entities as our similarity metric. We use three pretrained model on the disease NER task to valid the consistency of the result. RESULTS The disease NER dataset we release consists of 328 medical records from MIMIC-III, with 95629 sentences and 8884 disease mentions in total. The inter annotator agreement Cohen’s kappa coefficient is 0.86. Similarity metrics support that medical records from different departments are heterogeneous, ranges from 0.1004 to 0.3541 compare to Medical department. In the transfer learning task using the Medical department as the training set, F1 score performs in three pretrained models average from 0.847 to 0.863. F1 scores correlate with similarity metrics with Spearman’s coefficient of 0.4285. CONCLUSIONS We propose a disease NER dataset based on medical records from MIMIC-III and demonstrate the effectiveness of transfer learning using BERT. Similarity metrics reveal noticeable heterogeneity between department records. The deep learning-based transfer learning method demonstrates good ability to generalize across departments and achieve decent NER performance thus eliminates the concern that training material from one hospital might compromise model performance when applied to another. However, the model performance does not show high correlation to the departments’ similarity.

scholarly journals Combining Multi-task Learning with Transfer Learning for Biomedical Named Entity Recognition

2020 ◽  
Vol 176 ◽  
pp. 848-857 ◽  
Author(s):  
Tahir Mehmood ◽  
Alfonso E. Gerevini ◽  
Alberto Lavelli ◽  
Ivan Serina
Keyword(s):  
Transfer Learning ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Task Learning ◽  
Biomedical Named Entity Recognition

scholarly journals Transfer Learning for Arabic Named Entity Recognition With Deep Neural Networks

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 37736-37745 ◽  
Author(s):  
Mohammad Al-Smadi ◽  
Saad Al-Zboon ◽  
Yaser Jararweh ◽  
Patrick Juola
Keyword(s):  
Neural Networks ◽  
Transfer Learning ◽  
Deep Neural Networks ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity

scholarly journals A pre-training and self-training approach for biomedical named entity recognition

PLoS ONE ◽  
2021 ◽  
Vol 16 (2) ◽  
pp. e0246310
Author(s):  
Shang Gao ◽  
Olivera Kotevska ◽  
Alexandre Sorokine ◽  
J. Blair Christian
Keyword(s):  
Transfer Learning ◽  
Question Answering ◽  
Named Entity Recognition ◽  
Model Performance ◽  
Training Data ◽  
Entity Recognition ◽  
Literature Mining ◽  
Unified Medical Language System ◽  
Named Entity ◽  
Training Approach

Named entity recognition (NER) is a key component of many scientific literature mining tasks, such as information retrieval, information extraction, and question answering; however, many modern approaches require large amounts of labeled training data in order to be effective. This severely limits the effectiveness of NER models in applications where expert annotations are difficult and expensive to obtain. In this work, we explore the effectiveness of transfer learning and semi-supervised self-training to improve the performance of NER models in biomedical settings with very limited labeled data (250-2000 labeled samples). We first pre-train a BiLSTM-CRF and a BERT model on a very large general biomedical NER corpus such as MedMentions or Semantic Medline, and then we fine-tune the model on a more specific target NER task that has very limited training data; finally, we apply semi-supervised self-training using unlabeled data to further boost model performance. We show that in NER tasks that focus on common biomedical entity types such as those in the Unified Medical Language System (UMLS), combining transfer learning with self-training enables a NER model such as a BiLSTM-CRF or BERT to obtain similar performance with the same model trained on 3x-8x the amount of labeled data. We further show that our approach can also boost performance in a low-resource application where entities types are more rare and not specifically covered in UMLS.

Sign in / Sign up

Export Citation Format

Share Document