An Approach to NMT Re-Ranking Using Sequence-Labeling for Grammatical Error Correction

2020 ◽  
Vol 24 (4) ◽  
pp. 557-567
Author(s):  
Bo Wang ◽  
◽  
Kaoru Hirota ◽  
Chang Liu ◽  
Yaping Dai ◽  
...  
Keyword(s):  
Error Correction ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Short Term ◽  
Neural Machine Translation ◽  
Term Memory ◽  
Sequence Labeling ◽  
Grammatical Error ◽  
Small Set ◽  
Long Short Term Memory

An approach to N-best hypotheses re-ranking using a sequence-labeling model is applied to resolve the data deficiency problem in Grammatical Error Correction (GEC). Multiple candidate sentences are generated using a Neural Machine Translation (NMT) model; thereafter, these sentences are re-ranked via a stacked Transformer following a Bidirectional Long Short-Term Memory (BiLSTM) with Conditional Random Field (CRF). Correlations within the sentences are extracted using the sequence-labeling model based on the Transformer, which is particularly suitable for long sentences. Meanwhile, the knowledge from a large amount of unlabeled data is acquired through the pre-trained structure. Thus, completely revised sentences are adopted instead of partially modified sentences. Compared with conventional NMT, experiments on the NUCLE and FCE datasets demonstrate that the model improves the F0.5 score by 8.22% and 2.09%, respectively. As an advantage, the proposed re-ranking method has the advantage of only requires a small set of easily computed features that do not need linguistic inputs.

scholarly journals Ekstraksi Event Berbasis Paragraf dari Artikel Berita Bahasa Indonesia

2018 ◽  
Author(s):  
Masayu Leylia Khodra ◽  
Yudi Wibisono
Keyword(s):  
Random Fields ◽  
Conditional Random Fields ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Sequence Labeling ◽  
Long Short Term Memory ◽  
Bahasa Indonesia

Dengan banyaknya artikel berita online yang terbit setiap saat, sistem ekstraksi event dapat membantu pembaca berita dengan memberikan informasi terstruktur dari setiap artikel berita. Ekstraksi event dari artikel berita merupakan proses mendapatkan informasi terstruktur 5W1H yaitu siapa (who) melakukan apa (what), kapan (when), dimana (where), mengapa (why), dan bagaimana (how). Ekstraksi 5W1H ini merupakan salah satu jenis ekstraksi informasi. Model ekstraksi 5W1H dibangun dengan pendekatan berbasis sequence labeling berbasis skema BIO (Begin Inside Outside). Karena setiap paragraf berisi satu pokok pikiran, idealnya satu instans frame 5W1H dihasilkan dari satu paragraf, dan satu artikel berita direpresentasikan dengan sejumlah instans frame 5W1H. Oleh karena itu, makalah ini membahas pembangunan model ekstraksi event 5W1H berbasis paragraf. Pemodelan dilakukan dengan menggunakan korpus 610 teks paragraf yang diambil dari 57 artikel berita yang telah dianotasi secara manual dengan informasi 5W1H. Pemodelan memanfaatkan arsitektur bidirectional LSTMs (long short term memory) dan CRF (conditional random fields). Pada tahap evaluasi, kinerja model yang dicapai adalah F1 0.62

scholarly journals Pengenalan Entitas Bernama Otomatis untuk Bahasa Indonesia dengan Pendekatan Pembelajaran Mesin

2018 ◽  
Author(s):  
Yudi Wibisono ◽  
Masayu Leylia Khodra
Keyword(s):  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Short Term ◽  
Term Memory ◽  
Named Entity ◽  
Long Short Term Memory ◽  
F Measure ◽  
Bahasa Indonesia

Pengenalan entitas bernama (named-entity recognition atau NER) adalah proses otomatis mengekstraksi entitas bernama yang dianggap penting di dalam sebuah teks dan menentukan kategorinya ke dalam kategori terdefinisi. Sebagai contoh, untuk teks berita, NER dapat mengekstraksi nama orang, nama organisasi, dan nama lokasi. NER bermanfaat dalam berbagai aplikasi analisis teks, misalnya pencarian, sistem tanya jawab, peringkasan teks dan mesin penerjemah. Tantangan utama NER adalah penanganan ambiguitas makna karena konteks kata pada kalimat, misalnya kata “Cendana” dapat merupakan nama lokasi (Jalan Cendana), atau nama organisasi (Keluarga Cendana), atau nama tanaman. Tantangan lainnya adalah penentuan batas entitas, misalnya “[Istora Senayan] [Jakarta]”. Berbagai kakas NER telah dikembangkan untuk berbagai bahasa terutama Bahasa Inggris dengan kinerja yang baik, tetapi kakas NER bahasa Indonesia masih memiliki kinerja yang belum baik. Makalah ini membahas pendekatan berbasis pembelajaran mesin untuk menghasilkan model NER bahasa Indonesia. Pendekatan ini sangat bergantung pada korpus yang menjadi sumber belajar, dan teknik pembelajaran mesin yang digunakan. Teknik yang akan digunakan adalah LSTM - CRF (Long Short Term Memory – Conditional Random Field). Hasil terbaik (F-measure = 0.72) didapatkan dengan menggunakan word embedding GloVe Wikipedia Bahasa Indonesia.

Semantic Sequence Labeling Model of Power Dispatching Based on Deep Long Short Term Memory Network

2021 ◽  
Author(s):  
Hu Feifei ◽  
Zeng Shibo ◽  
Hong Danke ◽  
Zhang Situo ◽  
Song yongwei ◽  
...  
Keyword(s):  
Decision Making ◽  
Short Term Memory ◽  
Mechanism Analysis ◽  
Short Term ◽  
Term Memory ◽  
Control Center ◽  
Fine Grained ◽  
Sequence Labeling ◽  
Long Short Term Memory ◽  
Power Dispatching

As the decision-making brain for power system operation, grid regulation and operation is a comprehensive decision-making control that combines a large amount of data, mechanism analysis, operating procedures and professional experience, and a new generation of artificial intelligence development ideas and evolution characterized by data-driven and knowledge-guided. The directions are very close. However, the current scheduling control is still based on experience and manual analysis. The massive and diverse data of the control center and the lack of logical models between the plans require a large amount of experience and knowledge associations by the control personnel. There are more repetitive human brain labor and relatively low intelligence. Therefore, deep learning is applied to the learning of power control knowledge, and a semantic understanding network based on deep Long Short Term Memory is proposed. It uses sequence labeling to extract in-depth semantic related information of different keywords and query questions, and finds key information about language problems in order to achieve fine-grained and precise query. Experiments show that the proposed network model is superior to the previous methods, and it achieves better performance in the joint extraction of fine-grained evaluation words and evaluation objects, extracts the key information and deep semantic information of query problems and corresponding cases, and realizes power scheduling based on voice interaction The model can be effectively applied in the field of power dispatching and solve a large number of problems in power dispatching and control.

scholarly journals Post Text Processing of Chinese Speech Recognition Based on Bidirectional LSTM Networks and CRF

Electronics ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. 1248 ◽  
Author(s):  
Li Yang ◽  
Ying Li ◽  
Jin Wang ◽  
Zhuo Tang
Keyword(s):  
Speech Recognition ◽  
Error Detection ◽  
Short Term Memory ◽  
Text Processing ◽  
Conditional Random Field ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Two Stages

With the rapid development of Internet of Things Technology, speech recognition has been applied more and more widely. Chinese Speech Recognition is a complex process. In the process of speech-to-text conversion, due to the influence of dialect, environmental noise, and context, the accuracy of speech-to-text in multi-round dialogues and specific contexts is still not high. After the general speech recognition technology, the text after speech recognition can be detected and corrected in the specific context, which is helpful to improve the robustness of text comprehension and is a beneficial supplement to the speech recognition technology. In this paper, a text processing model after Chinese Speech Recognition is proposed, which combines a bidirectional long short-term memory (LSTM) network with a conditional random field (CRF) model. The task is divided into two stages: text error detection and text error correction. In this paper, a bidirectional long short-term memory (Bi-LSTM) network and conditional random field are used in two stages of text error detection and text error correction respectively. Through verification and system test on the SIGHAN 2013 Chinese Spelling Check (CSC) dataset, the experimental results show that the model can effectively improve the accuracy of text after speech recognition.

scholarly journals Extracting entities with attributes in clinical text via joint deep learning

2019 ◽  
Vol 26 (12) ◽  
pp. 1584-1591 ◽  
Author(s):  
Xue Shi ◽  
Yingping Yi ◽  
Ying Xiong ◽  
Buzhou Tang ◽  
Qingcai Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Relation Extraction ◽  
Entity Recognition ◽  
Short Term ◽  
Term Memory ◽  
Clinical Text ◽  
Long Short Term Memory

Abstract Objective Extracting clinical entities and their attributes is a fundamental task of natural language processing (NLP) in the medical domain. This task is typically recognized as 2 sequential subtasks in a pipeline, clinical entity or attribute recognition followed by entity-attribute relation extraction. One problem of pipeline methods is that errors from entity recognition are unavoidably passed to relation extraction. We propose a novel joint deep learning method to recognize clinical entities or attributes and extract entity-attribute relations simultaneously. Materials and Methods The proposed method integrates 2 state-of-the-art methods for named entity recognition and relation extraction, namely bidirectional long short-term memory with conditional random field and bidirectional long short-term memory, into a unified framework. In this method, relation constraints between clinical entities and attributes and weights of the 2 subtasks are also considered simultaneously. We compare the method with other related methods (ie, pipeline methods and other joint deep learning methods) on an existing English corpus from SemEval-2015 and a newly developed Chinese corpus. Results Our proposed method achieves the best F1 of 74.46% on entity recognition and the best F1 of 50.21% on relation extraction on the English corpus, and 89.32% and 88.13% on the Chinese corpora, respectively, which outperform the other methods on both tasks. Conclusions The joint deep learning–based method could improve both entity recognition and relation extraction from clinical text in both English and Chinese, indicating that the approach is promising.

A Framework for Indonesian Grammar Error Correction

2021 ◽  
Vol 20 (4) ◽  
pp. 1-12
Author(s):  
Nankai Lin ◽  
Boyu Chen ◽  
Xiaotian Lin ◽  
Kanoksak Wattanachote ◽  
Shengyi Jiang
Keyword(s):  
Error Correction ◽  
Language Processing ◽  
Short Term Memory ◽  
Short Term ◽  
Low Resource ◽  
Part Of Speech ◽  
Grammatical Error ◽  
Evaluation Dataset ◽  
Long Short Term Memory ◽  
Multi Classification

Grammatical Error Correction (GEC) is a challenge in Natural Language Processing research. Although many researchers have been focusing on GEC in universal languages such as English or Chinese, few studies focus on Indonesian, which is a low-resource language. In this article, we proposed a GEC framework that has the potential to be a baseline method for Indonesian GEC tasks. This framework treats GEC as a multi-classification task. It integrates different language embedding models and deep learning models to correct 10 types of Part of Speech (POS) error in Indonesian text. In addition, we constructed an Indonesian corpus that can be utilized as an evaluation dataset for Indonesian GEC research. Our framework was evaluated on this dataset. Results showed that the Long Short-Term Memory model based on word-embedding achieved the best performance. Its overall macro-average F 0.5 in correcting 10 POS error types reached 0.551. Results also showed that the framework can be trained on a low-resource dataset.

Sign in / Sign up

Export Citation Format

Share Document