Automatic clustering of part-of-speech for vocabulary divided PLSA language model

2008 ◽  
Author(s):  
Motoyuki Suzuki ◽  
Naoto Kuriyama ◽  
Akinori Ito ◽  
Shozo Makino
Keyword(s):  
Language Model ◽  
Part Of Speech ◽  
Automatic Clustering

scholarly journals Fracture Mechanics Method for Word Embedding Generation of Neural Probabilistic Linguistic Model

2016 ◽  
Vol 2016 ◽  
pp. 1-11
Author(s):  
Size Bi ◽  
Xiao Liang ◽  
Ting-lei Huang
Keyword(s):  
Fracture Mechanics ◽  
Language Model ◽  
Named Entity Recognition ◽  
Word Embedding ◽  
Entity Recognition ◽  
Semantic Role Labeling ◽  
Part Of Speech ◽  
Linguistic Model ◽  
Traditional Language ◽  
Long Time

Word embedding, a lexical vector representation generated via the neural linguistic model (NLM), is empirically demonstrated to be appropriate for improvement of the performance of traditional language model. However, the supreme dimensionality that is inherent in NLM contributes to the problems of hyperparameters and long-time training in modeling. Here, we propose a force-directed method to improve such problems for simplifying the generation of word embedding. In this framework, each word is assumed as a point in the real world; thus it can approximately simulate the physical movement following certain mechanics. To simulate the variation of meaning in phrases, we use the fracture mechanics to do the formation and breakdown of meaning combined by a 2-gram word group. With the experiments on the natural linguistic tasks of part-of-speech tagging, named entity recognition and semantic role labeling, the result demonstrated that the 2-dimensional word embedding can rival the word embeddings generated by classic NLMs, in terms of accuracy, recall, and text visualization.

scholarly journals Neural text normalization with adapted decoding and POS features

2019 ◽  
Vol 25 (5) ◽  
pp. 585-605
Author(s):  
T. Ruzsics ◽  
M. Lusetti ◽  
A. Göhring ◽  
T. Samardžić ◽  
E. Stark
Keyword(s):  
Machine Translation ◽  
Computer Mediated Communication ◽  
Statistical Machine Translation ◽  
Language Model ◽  
Translation System ◽  
Swiss German ◽  
Part Of Speech ◽  
Word Level ◽  
Speech Transcription ◽  
Text Normalization

AbstractText normalization is the task of mapping noncanonical language, typical of speech transcription and computer-mediated communication, to a standardized writing. This task is especially important for languages such as Swiss German, with strong regional variation and no written standard. In this paper, we propose a novel solution for normalizing Swiss German WhatsApp messages using the encoder–decoder neural machine translation (NMT) framework. We enhance the performance of a plain character-level NMT model with the integration of a word-level language model and linguistic features in the form of part-of-speech (POS) tags. The two components are intended to improve the performance by addressing two specific issues: the former is intended to improve the fluency of the predicted sequences, whereas the latter aims at resolving cases of word-level ambiguity. Our systematic comparison shows that our proposed solution results in an improvement over a plain NMT system and also over a comparable character-level statistical machine translation system, considered the state of the art in this task till recently. We perform a thorough analysis of the compared systems’ output, showing that our two components produce indeed the intended, complementary improvements.

Learning Syllables Using Conv-LSTM Model for Swahili Word Representation and Part-of-speech Tagging

2021 ◽  
Vol 20 (4) ◽  
pp. 1-25
Author(s):  
Casper Shikali Shivachi ◽  
Refuoe Mokhosi ◽  
Zhou Shijie ◽  
Liu Qihe
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Language Model ◽  
Word Embeddings ◽  
Short Term ◽  
Term Memory ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Word Representation ◽  
Long Short Term Memory

The need to capture intra-word information in natural language processing (NLP) tasks has inspired research in learning various word representations at word, character, or morpheme levels, but little attention has been given to syllables from a syllabic alphabet. Motivated by the success of compositional models in morphological languages, we present a Convolutional-long short term memory (Conv-LSTM) model for constructing Swahili word representation vectors from syllables. The unified architecture addresses the word agglutination and polysemous nature of Swahili by extracting high-level syllable features using a convolutional neural network (CNN) and then composes quality word embeddings with a long short term memory (LSTM). The word embeddings are then validated using a syllable-aware language model ( 31.267 ) and a part-of-speech (POS) tagging task ( 98.78 ), both yielding very competitive results to the state-of-art models in their respective domains. We further validate the language model using Xhosa and Shona, which are syllabic-based languages. The novelty of the study is in its capability to construct quality word embeddings from syllables using a hybrid model that does not use max-over-pool common in CNN and then the exploitation of these embeddings in POS tagging. Therefore, the study plays a crucial role in the processing of agglutinative and syllabic-based languages by contributing quality word embeddings from syllable embeddings, a robust Conv–LSTM model that learns syllables for not only language modeling and POS tagging, but also for other downstream NLP tasks.

scholarly journals External features enriched model for biomedical question answering

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Gezheng Xu ◽  
Wenge Rong ◽  
Yanmeng Wang ◽  
Yuanxin Ouyang ◽  
Zhang Xiong
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Question Answering ◽  
Language Model ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Small Scale ◽  
Original Text ◽  
Specific Domain ◽  
Part Of Speech

Abstract Background Biomedical question answering (QA) is a sub-task of natural language processing in a specific domain, which aims to answer a question in the biomedical field based on one or more related passages and can provide people with accurate healthcare-related information. Recently, a lot of approaches based on the neural network and large scale pre-trained language model have largely improved its performance. However, considering the lexical characteristics of biomedical corpus and its small scale dataset, there is still much improvement room for biomedical QA tasks. Results Inspired by the importance of syntactic and lexical features in the biomedical corpus, we proposed a new framework to extract external features, such as part-of-speech and named-entity recognition, and fused them with the original text representation encoded by pre-trained language model, to enhance the biomedical question answering performance. Our model achieves an overall improvement of all three metrics on BioASQ 6b, 7b, and 8b factoid question answering tasks. Conclusions The experiments on BioASQ question answering dataset demonstrated the effectiveness of our external feature-enriched framework. It is proven by the experiments conducted that external lexical and syntactic features can improve Pre-trained Language Model’s performance in biomedical domain question answering task.

Improvement of Lecture Speech Recognition by Using Unsupervised Adaptation

2011 ◽  
pp. 189-202
Author(s):  
Tetsuo Kosaka ◽  
Takashi Kusama ◽  
Masaharu Kato ◽  
Masaki Kohda
Keyword(s):  
Large Scale ◽  
Markov Models ◽  
Recognition Performance ◽  
Language Model ◽  
Spontaneous Speech ◽  
Part Of Speech ◽  
Speech Database ◽  
Adaptation Procedure ◽  
Lm Adaptation ◽  
Optimal Approach

The aim of this work is to improve the recognition performance of spontaneous speech. In order to achieve the purpose, the authors of this chapter propose new approaches of unsupervised adaptation for spontaneous speech and evaluate the methods by using diagonal-covariance and full-covariance hidden Markov models. In the adaptation procedure, both methods of language model (LM) adaptation and acoustic model (AM) adaptation are used iteratively. Several combination methods are tested to find the optimal approach. In the LM adaptation, a word trigram model and a part-of-speech (POS) trigram model are combined to build a more task-specific LM. In addition, the authors propose an unsupervised speaker adaptation technique based on adaptation data weighting. The weighting is performed depending on POS class. In Japan, a large-scale spontaneous speech database “Corpus of Spontaneous Japanese (CSJ)” has been used as the common evaluation database for spontaneous speech and the authors used it for their recognition experiments. From the results, the proposed methods demonstrated a significant advantage in that task.

Part-of-Speech Tagging

2012 ◽  
Author(s):  
Atro Voutilainen
Keyword(s):  
Markov Models ◽  
Language Model ◽  
Language Models ◽  
Symbolic Language ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Text Corpora ◽  
History Of ◽  
General Architecture ◽  
Speech Tagging

This article outlines the recently used methods for designing part-of-speech taggers; computer programs for assigning contextually appropriate grammatical descriptors to words in texts. It begins with the description of general architecture and task setting. It gives an overview of the history of tagging and describes the central approaches to tagging. These approaches are: taggers based on handwritten local rules, taggers based on n-grams automatically derived from text corpora, taggers based on hidden Markov models, taggers using automatically generated symbolic language models derived using methods from machine tagging, taggers based on handwritten global rules, and hybrid taggers, which combine the advantages of handwritten and automatically generated taggers. This article focuses on handwritten tagging rules. Well-tagged training corpora are a valuable resource for testing and improving language model. The text corpus reminds the grammarian about any oversight while designing a rule.

Sign in / Sign up

Export Citation Format

Share Document