Multi-level transfer learning for improving the performance of deep neural networks: Theory and practice from the tasks of facial emotion recognition and named entity recognition

Abstract Motivation Named entity recognition is a critical and fundamental task for biomedical text mining. Recently, researchers have focused on exploiting deep neural networks for biomedical named entity recognition (Bio-NER). The performance of deep neural networks on a single dataset mostly depends on data quality and quantity while high-quality data tends to be limited in size. To alleviate task-specific data limitation, some studies explored the multi-task learning (MTL) for Bio-NER and achieved state-of-the-art performance. However, these MTL methods did not make full use of information from various datasets of Bio-NER. The performance of state-of-the-art MTL method was significantly limited by the number of training datasets. Results We propose two dataset-aware MTL approaches for Bio-NER which jointly train all models for numerous Bio-NER datasets, thus each of these models could discriminatively exploit information from all of related training datasets. Both of our two approaches achieve substantially better performance compared with the state-of-the-art MTL method on 14 out of 15 Bio-NER datasets. Furthermore, we implemented our approaches by incorporating Bio-NER and biomedical part-of-speech (POS) tagging datasets. The results verify Bio-NER and POS can significantly enhance one another. Availability and implementation Our source code is available at https://github.com/zmmzGitHub/MTL-BC-LBC-BioNER and all datasets are publicly available at https://github.com/cambridgeltl/MTL-Bioinformatics-2016. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text

Deep Neural Networks for Named Entity Recognition in Italian

Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015 ◽

10.4000/books.aaccademia.1317 ◽

2015 ◽

pp. 51-55 ◽

Cited By ~ 8

Author(s):

Daniele Bonadiman ◽

Aliaksei Severyn ◽

Alessandro Moschitti

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Named Entity Recognition ◽

Entity Recognition ◽

Named Entity

Download Full-text

Applying Deep Neural Networks to Named Entity Recognition in Portuguese Texts

2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS) ◽

10.1109/snams.2018.8554782 ◽

2018 ◽

Author(s):

Ivo Fernandes ◽

Henrique Lopes Cardoso ◽

Eugenio Oliveira

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Named Entity Recognition ◽

Entity Recognition ◽

Named Entity

Download Full-text

Domain Specific Named Entity Recognition Referring to the Real World by Deep Neural Networks

10.18653/v1/p16-2039 ◽

2016 ◽

Cited By ~ 7

Author(s):

Suzushi Tomori ◽

Takashi Ninomiya ◽

Shinsuke Mori

Keyword(s):

Neural Networks ◽

Real World ◽

Deep Neural Networks ◽

Named Entity Recognition ◽

Entity Recognition ◽

The Real ◽

Named Entity ◽

Domain Specific

Download Full-text

NAMED ENTITY RECOGNITION IN TEXTS OF ADMINISTRATIVE DOCUMENTS WITH DEEP NEURAL NETWORKS

System Informatics ◽

10.31144/si.2307-6410.2020.n16.p137-148 ◽

2020 ◽

Author(s):

Sergey Berezin ◽

◽

Ivan Bondarenko ◽

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Language Model ◽

Named Entity Recognition ◽

Semantic Relatedness ◽

Entity Recognition ◽

General Nature ◽

Preliminary Training ◽

Entity Extraction ◽

Named Entity

Named Entity Extraction (NER) is the task of extracting information from text data that belongs to predefined categories, such as organizations names, place names, people's names, etc. Within the framework of the presented work, was developed an approach for the additional training of deep neural networks with the attention mechanism (BERT architecture). It is shown that the preliminary training of the language model in the tasks of recovering the masked word and determining the semantic relatedness of two sentences can significantly improve the quality of solving the problem of NER. One of the best results has been achieved in the task of extracting named entities on the RuREBus dataset. One of the key features of the described solution is the closeness of the formulation to real business problems and the selection of entities not of a general nature, but specific to the economic industry.

Download Full-text