Enhancing LSTM-based Word Segmentation Using Unlabeled Data

2017 ◽  
pp. 60-70 ◽  
Author(s):  
Bo Zheng ◽  
Wanxiang Che ◽  
Jiang Guo ◽  
Ting Liu
Keyword(s):  
Word Segmentation ◽  
Unlabeled Data

scholarly journals Unsupervised Learning Helps Supervised Neural Word Segmentation

2019 ◽  
Vol 33 ◽  
pp. 7200-7207 ◽  
Author(s):  
Xiaobin Wang ◽  
Deng Cai ◽  
Linlin Li ◽  
Guangwei Xu ◽  
Hai Zhao ◽  
...  
Keyword(s):  
Recall Rate ◽  
Word Segmentation ◽  
Unlabeled Data ◽  
Data Sets ◽  
Chinese Word Segmentation ◽  
Unsupervised Segmentation ◽  
Data Set ◽  
Standard Data ◽  
Effective Strategies ◽  
Training Approach

By exploiting unlabeled data for further performance improvement for Chinese word segmentation, this work makes the first attempt at exploring adding unsupervised segmentation information into neural supervised segmenter. We survey various effective strategies, including extending the character embedding, augmenting the word score and applying multi-task learning, for leveraging unsupervised information derived from abundant unlabeled data. Experiments on standard data sets show that the explored strategies indeed improve the recall rate of out-of-vocabulary words and thus boost the segmentation accuracy. Moreover, the model enhanced by the proposed methods outperforms state-of-theart models in closed test and shows promising improvement trend when adopting three different strategies with the help of a large unlabeled data set. Our thorough empirical study eventually verifies the proposed approach outperforms the widelyused pre-training approach in terms of effectively making use of freely abundant unlabeled data.

scholarly journals Improving Neural Chinese Word Segmentation Using Unlabeled Data

2018 ◽  
Vol 435 ◽  
pp. 012032
Author(s):  
Yanna Zhang ◽  
Jinan Xu ◽  
Guoyi Miao ◽  
Yufeng Chen ◽  
Yujie Zhang
Keyword(s):  
Word Segmentation ◽  
Unlabeled Data ◽  
Chinese Word ◽  
Chinese Word Segmentation

scholarly journals Neural Chinese Word Segmentation with Lexicon and Unlabeled Data via Posterior Regularization

2019 ◽  
Author(s):  
Junxin Liu ◽  
Fangzhao Wu ◽  
Chuhan Wu ◽  
Yongfeng Huang ◽  
Xing Xie
Keyword(s):  
Word Segmentation ◽  
Unlabeled Data ◽  
Chinese Word ◽  
Chinese Word Segmentation

Word Segmentation in the "Real World" of Conversational Speech

2007 ◽  
Author(s):  
Joseph D. W. S tephens ◽  
Mark A. Pitt
Keyword(s):  
Real World ◽  
Word Segmentation ◽  
Conversational Speech ◽  
The Real

Does Prior Orthographic Experience Influence L1 and L2 Word Segmentation?

2004 ◽  
Author(s):  
Jyotsna Vaid ◽  
Hsin-Chin Chen ◽  
Francisco E. Martinez ◽  
Chaitra Rao
Keyword(s):  
Word Segmentation ◽  
L1 And L2

Gated Recursive Neural Network for Chinese Word Segmentation

2015 ◽  
Author(s):  
Xinchi Chen ◽  
Xipeng Qiu ◽  
Chenxi Zhu ◽  
Xuanjing Huang
Keyword(s):  
Neural Network ◽  
Word Segmentation ◽  
Chinese Word ◽  
Chinese Word Segmentation ◽  
Recursive Neural Network

Tibetan Unknown Word Identification from News Corpora for Supporting Lexicon-based Tibetan Word Segmentation

2015 ◽  
Author(s):  
Minghua Nuo ◽  
Huidan Liu ◽  
Congjun Long ◽  
Jian Wu
Keyword(s):  
Word Identification ◽  
Word Segmentation ◽  
Unknown Word

A mixed approach for Chinese word segmentation

2014 ◽  
Author(s):  
Zhen Wang
Keyword(s):  
Word Segmentation ◽  
Chinese Word ◽  
Chinese Word Segmentation ◽  
Mixed Approach
Sign in / Sign up

Export Citation Format

Share Document