scholarly journals Measurement of Text Similarity: A Survey

Information ◽  
2020 ◽  
Vol 11 (9) ◽  
pp. 421
Author(s):  
Jiapeng Wang ◽  
Yihong Dong
Keyword(s):  
Language Processing ◽  
Question Answering ◽  
Semantic Distance ◽  
Similarity Measurement ◽  
Text Representation ◽  
Text Similarity ◽  
Discussion Section ◽  
Advantages And Disadvantages ◽  
Comprehensive Classification ◽  
Distribution Distance

Text similarity measurement is the basis of natural language processing tasks, which play an important role in information retrieval, automatic question answering, machine translation, dialogue systems, and document matching. This paper systematically combs the research status of similarity measurement, analyzes the advantages and disadvantages of current methods, develops a more comprehensive classification description system of text similarity measurement algorithms, and summarizes the future development direction. With the aim of providing reference for related research and application, the text similarity measurement method is described by two aspects: text distance and text representation. The text distance can be divided into length distance, distribution distance, and semantic distance; text representation is divided into string-based, corpus-based, single-semantic text, multi-semantic text, and graph-structure-based representation. Finally, the development of text similarity is also summarized in the discussion section.

ALBERT-QM: An ALBERT Based Method for Chinese Health Related Question Matching (Preprint)

2020 ◽  
Author(s):  
Feihong Yang ◽  
Jiao Li
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Medical Knowledge ◽  
Related Question ◽  
Text Similarity ◽  
Web Based ◽  
Health Related ◽  
Health Question

BACKGROUND Question answering (QA) system is widely used in web-based health-care applications. Health consumers likely asked similar questions in various natural language expression due to the lack of medical knowledge. It’s challenging to match a new question to previous similar questions for answering. In health QA system development, question matching (QM) is a task to judge whether a pair of questions express the same meaning and is used to map the answer of matched question in the given question-answering database. BERT (i.e. Bidirectional Encoder Representations from Transformers) is proved to be state-of- the-art model in natural language processing (NLP) tasks, such as binary classification and sentence matching. As a light model of BERT, ALBERT is proposed to address the huge parameters and low training speed problems of BERT. Both of BERT and ALBERT can be used to address the QM problem. OBJECTIVE In this study, we aim to develop an ALBERT based method for Chinese health related question matching. METHODS Our proposed method, named as ALBERT-QM, consists of three components. (1)Data augmenting. Similar health question pairs were augmented for training preparation. (2)ALBERT model training. Given the augmented training pairs, three ALBERT models were trained and fine-tuned. (3)Similarity combining. Health question similarity score were calculated by combining ALBRT model outputs with text similarity. To evaluate our ALBERT-QM performance on similar question identification, we used an open dataset with 20,000 labeled Chinese health question pairs. RESULTS Our ALBERT-QM is able to identify similar Chinese health questions, achieving the precision of 86.69%, recall of 86.70% and F1 of 86.69%. Comparing with baseline method (text similarity algorithm), ALBERT-QM enhanced the F1-score by 20.73%. Comparing with other BERT series models, our ALBERT-QM is much lighter with the files size of 64.8MB which is 1/6 times that other BERT models. We made our ALBERT-QM open accessible at https://github.com/trueto/albert_question_match. CONCLUSIONS In this study, we developed an open source algorithm, ALBERT-QM, contributing to similar Chinese health questions identification in a health QA system. Our ALBERT-QM achieved better performance in question matching with lower memory usage, which is beneficial to the web-based or mobile-based QA applications.

Reports of the Workshops Held at the 2019 AAAI Conference on Artificial Intelligence

AI Magazine ◽  
2019 ◽  
Vol 40 (3) ◽  
pp. 67-78
Author(s):  
Guy Barash ◽  
Mauricio Castillo-Effen ◽  
Niyati Chhaya ◽  
Peter Clark ◽  
Huáscar Espinoza ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Language Processing ◽  
Cyber Security ◽  
Question Answering ◽  
Intent Recognition ◽  
Affective Content ◽  
Learning Plan ◽  
Dialog System ◽  
Affective Content Analysis ◽  
Games And Simulations

The workshop program of the Association for the Advancement of Artificial Intelligence’s 33rd Conference on Artificial Intelligence (AAAI-19) was held in Honolulu, Hawaii, on Sunday and Monday, January 27–28, 2019. There were fifteen workshops in the program: Affective Content Analysis: Modeling Affect-in-Action, Agile Robotics for Industrial Automation Competition, Artificial Intelligence for Cyber Security, Artificial Intelligence Safety, Dialog System Technology Challenge, Engineering Dependable and Secure Machine Learning Systems, Games and Simulations for Artificial Intelligence, Health Intelligence, Knowledge Extraction from Games, Network Interpretability for Deep Learning, Plan, Activity, and Intent Recognition, Reasoning and Learning for Human-Machine Dialogues, Reasoning for Complex Question Answering, Recommender Systems Meet Natural Language Processing, Reinforcement Learning in Games, and Reproducible AI. This report contains brief summaries of the all the workshops that were held.

Simulation of bright–dark soliton solutions of the Lonngren wave equation arising the model of transmission lines

2021 ◽  
pp. 2150484
Author(s):  
Asif Yokuş
Keyword(s):  
Wave Equation ◽  
Soliton Solutions ◽  
Stationary Wave ◽  
Dark Soliton ◽  
Traveling Wave Solution ◽  
Bright Soliton ◽  
Auxiliary Equation ◽  
Discussion Section ◽  
Auxiliary Equation Method ◽  
Advantages And Disadvantages

In this study, the auxiliary equation method is applied successfully to the Lonngren wave equation. Bright soliton, bright–dark soliton solutions are produced, which play an important role in the distribution and distribution of electric charge. In the conclusion and discussion section, the effect of nonlinearity term on wave behavior in bright soliton traveling wave solution is examined. The advantages and disadvantages of the method are discussed. While graphs representing the stationary wave are obtained, special values are given to the constants in the solutions. These graphs are presented as 3D, 2D and contour.

A content spectral-based text representation

2021 ◽  
pp. 1-12
Author(s):  
Melesio Crespo-Sanchez ◽  
Ivan Lopez-Arevalo ◽  
Edwin Aldana-Bobadilla ◽  
Alejandro Molina-Villegas
Keyword(s):  
Machine Learning ◽  
Text Analysis ◽  
Question Answering ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Text Representation ◽  
Feature Vectors ◽  
Learning Tasks ◽  
Semantic Component ◽  
Vector Representations

In the last few years, text analysis has grown as a keystone in several domains for solving many real-world problems, such as machine translation, spam detection, and question answering, to mention a few. Many of these tasks can be approached by means of machine learning algorithms. Most of these algorithms take as input a transformation of the text in the form of feature vectors containing an abstraction of the content. Most of recent vector representations focus on the semantic component of text, however, we consider that also taking into account the lexical and syntactic components the abstraction of content could be beneficial for learning tasks. In this work, we propose a content spectral-based text representation applicable to machine learning algorithms for text analysis. This representation integrates the spectra from the lexical, syntactic, and semantic components of text producing an abstract image, which can also be treated by both, text and image learning algorithms. These components came from feature vectors of text. For demonstrating the goodness of our proposal, this was tested on text classification and complexity reading score prediction tasks obtaining promising results.

A survey of question answering in natural language processing

Poetics ◽  
1990 ◽  
Vol 19 (1-2) ◽  
pp. 99-120
Author(s):  
Stefan Wermter ◽  
Wendy G. Lehnert
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering

scholarly journals Enhancing Natural Language Inference Using New and Expanded Training Data Sets and New Learning Models

2020 ◽  
Vol 34 (05) ◽  
pp. 8504-8511
Author(s):  
Arindam Mitra ◽  
Ishan Shrivastava ◽  
Chitta Baral
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Training Data ◽  
Data Sets ◽  
Learning Models ◽  
New Learning ◽  
Word Attention ◽  
Attention Function

Natural Language Inference (NLI) plays an important role in many natural language processing tasks such as question answering. However, existing NLI modules that are trained on existing NLI datasets have several drawbacks. For example, they do not capture the notion of entity and role well and often end up making mistakes such as “Peter signed a deal” can be inferred from “John signed a deal”. As part of this work, we have developed two datasets that help mitigate such issues and make the systems better at understanding the notion of “entities” and “roles”. After training the existing models on the new dataset we observe that the existing models do not perform well on one of the new benchmark. We then propose a modification to the “word-to-word” attention function which has been uniformly reused across several popular NLI architectures. The resulting models perform as well as their unmodified counterparts on the existing benchmarks and perform significantly well on the new benchmarks that emphasize “roles” and “entities”.

scholarly journals Analyzing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Changchang Zeng ◽  
Shaobo Li
Keyword(s):  
Reading Comprehension ◽  
Language Processing ◽  
Question Answering ◽  
Multiple Choice ◽  
Length Distribution ◽  
Research Field ◽  
Evaluation Framework ◽  
Language Models ◽  
Training Objective ◽  
Machine Reading

Machine reading comprehension (MRC) is a challenging natural language processing (NLP) task. It has a wide application potential in the fields of question answering robots, human-computer interactions in mobile virtual reality systems, etc. Recently, the emergence of pretrained models (PTMs) has brought this research field into a new era, in which the training objective plays a key role. The masked language model (MLM) is a self-supervised training objective widely used in various PTMs. With the development of training objectives, many variants of MLM have been proposed, such as whole word masking, entity masking, phrase masking, and span masking. In different MLMs, the length of the masked tokens is different. Similarly, in different machine reading comprehension tasks, the length of the answer is also different, and the answer is often a word, phrase, or sentence. Thus, in MRC tasks with different answer lengths, whether the length of MLM is related to performance is a question worth studying. If this hypothesis is true, it can guide us on how to pretrain the MLM with a relatively suitable mask length distribution for MRC tasks. In this paper, we try to uncover how much of MLM’s success in the machine reading comprehension tasks comes from the correlation between masking length distribution and answer length in the MRC dataset. In order to address this issue, herein, (1) we propose four MRC tasks with different answer length distributions, namely, the short span extraction task, long span extraction task, short multiple-choice cloze task, and long multiple-choice cloze task; (2) four Chinese MRC datasets are created for these tasks; (3) we also have pretrained four masked language models according to the answer length distributions of these datasets; and (4) ablation experiments are conducted on the datasets to verify our hypothesis. The experimental results demonstrate that our hypothesis is true. On four different machine reading comprehension datasets, the performance of the model with correlation length distribution surpasses the model without correlation.

scholarly journals ENTITY EXTRACTION FROM UNSTRUCTURED MEDICAL TEXT

2020 ◽  
Vol 5 (8) ◽  
Author(s):  
G Deepank ◽  
R Tharun Raj ◽  
Aditya Verma
Keyword(s):  
Language Processing ◽  
Medical Records ◽  
Question Answering ◽  
Entity Extraction ◽  
Data Repositories ◽  
Efficient Treatment ◽  
Learning Techniques ◽  
Treatment Plans ◽  
Data Elements ◽  
Rich Data

Electronic medical records represent rich data repositories loaded with valuable patient information. As artificial intelligence and machine learning in the field of medicine is becoming more popular by the day, ways to integrate it are always changing. One such way is processing the clinical notes and records, which are maintained by doctors and other medical professionals. Natural language processing can record this data and read more deeply into it than any human. Deep learning techniques such as entity extraction which involves identifying and returning of key data elements from an electronic medical record, and other techniques involving models such as BERT for question answering, when applied to all these medical records can create bespoke and efficient treatment plans for the patients, which can help in a swift and carefree recovery.

scholarly journals Emerging trends: A gentle introduction to fine-tuning

2021 ◽  
Vol 27 (6) ◽  
pp. 763-778
Author(s):  
Kenneth Ward Church ◽  
Zeyu Chen ◽  
Yanjun Ma
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
General Purpose ◽  
Fine Tuning ◽  
Language Engineering ◽  
Training Models ◽  
Emerging Trends ◽  
Foundation Model ◽  
Programming Skills

AbstractThe previous Emerging Trends article (Church et al., 2021. Natural Language Engineering27(5), 631–645.) introduced deep nets to poets. Poets is an imperfect metaphor, intended as a gesture toward inclusion. The future for deep nets will benefit by reaching out to a broad audience of potential users, including people with little or no programming skills, and little interest in training models. That paper focused on inference, the use of pre-trained models, as is, without fine-tuning. The goal of this paper is to make fine-tuning more accessible to a broader audience. Since fine-tuning is more challenging than inference, the examples in this paper will require modest programming skills, as well as access to a GPU. Fine-tuning starts with a general purpose base (foundation) model and uses a small training set of labeled data to produce a model for a specific downstream application. There are many examples of fine-tuning in natural language processing (question answering (SQuAD) and GLUE benchmark), as well as vision and speech.

Sign in / Sign up

Export Citation Format

Share Document