A Chinese word segmentation algorithm based on maximum entropy

Author(s):  
Li-Yan Zhang ◽  
Min Qin ◽  
Xue-Mei Zhang ◽  
Hong-Xia Ma
2014 ◽  
Vol 687-691 ◽  
pp. 1536-1539
Author(s):  
Dong Xue Chu

Today's very popular search engine technology, which is conducive to further analysis and full-text retrieval technology for Chinese word segmentation technology, and Chinese word segmentation is an important technology of Chinese information, the quality of Chinese word segmentation will have a direct impact on the efficiency of Chinese information. Therefore, the related concepts of the Chinese algorithm are discussed in this paper, some specific algorithm for Chinese, like algorithms based on rules and dictionary, statistical algorithms based on large-scale corpus, unity algorithm of statistics and the rule, artificial intelligence word segmentation algorithms and so on, and finally it describes the evaluated basis and difficulty of Chinese word segmentation algorithm.


2013 ◽  
Vol 411-414 ◽  
pp. 313-316
Author(s):  
Chang Liu

In order to improve the speed of Chinese full-text retrieval in the premise of ensuring Chinese ambiguity inclusion and length limitation, this paper introduces the application methods of Chinese full-text retrieval system and the current application situation of Chinese word segmentation technology. Based on the existed word segmentation algorithms, this paper proposed an improved Chinese word segmentation algorithm. In the proposed method, the procedure of indexing is to construct the map between the relative words in the context and the dictionary. This paper improves the diction to realize better mapping with relative words, so as to realize Chinese words segmentation. The experiments demonstrate that the proposed Chinese full-text words segmentation algorithm is more effective than the existing methods.


Sign in / Sign up

Export Citation Format

Share Document