Improving Chinese Word Segmentation with Description Length Gain

Keywords

Computer science, Artificial intelligence, Conditional random field, Segmentation, Natural language processing

References

• Designing Special Post-Processing Rules for SVM-Based Chinese Word SegmentationMuhua Zhu, Yilin Wang, Zhenxing Wang — 2006
• A Systematic Cross-Comparison of Sequence ClassifiersBinyamin Rozenfeld, Ronen Feldman, Moshe Fresko — 2006
• Chinese Word Segmentation with Maximum Entropy and N-gram Language ModelXinhao Wang, Xiaojun Lin, Dianhai Yu — 2006
• Information Theory: 1948-1998 - Guest EditorialSergio Verdú — 1998
• Unsupervised Learning of Word Boundary with Description Length GainChunyu Kit, Yorick Wilks — 1999
• Stochastic Complexity in Statistical InquiryJ. Rissanen — 1989
• Unsupervised lexical learning as inductive inference.Kit Cy — 2000
• PAT-tree-based keyword extraction for Chinese information retrievalLee‐Feng Chien — 1997
• Using Suffix Arrays to Compute Term Frequency and Document Frequency for All Substrings in a CorpusMikio Yamamoto, Kenneth Church — 2001
• Extraction of Chinese Compound Words - An Experimental Study on a Very Large CorpusJian Zhang, Jianfeng Gao, Ming Zhou — 2000
• The First International Chinese Word Segmentation BakeoffRichard Sproat, Thomas E. Emerson — 2003
• Chinese Segmentation and New Word Detection using Conditional Random FieldsFuchun Peng, Fangfang Feng, Andrew McCallum — 2004
• Modeling By Shortest Data Description*J. Rissanen — 1978
• An Efficient, Probabilistically Sound Algorithm for Segmentation and Word DiscoveryMichael R. Brent — 1999
• Contextual Dependencies in Unsupervised Word SegmentationSharon Goldwater, Thomas L. Griffiths, Mark Johnson — 2006
• Chinese Word Segmentation without Using Lexicon and Hand-crafted Training DataMaosong Sun, Shen Dayang, Benjamin K. Tsou — 1998
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence DataJohn Lafferty, Andrew McCallum, Fernando C. N. Pereira — 2001
• Chinese Word Segmentation as Character TaggingNianwen Xue — 2003
• Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field ModelingHai Zhao, Changning Huang, Mu Li — 2006
• Chinese Word Segmentation and Named Entity Recognition Based on a Context-Dependent Mutual Information Independence ModelMin Zhang, Guodong Zhou, Lingpeng Yang — 2006

Cited by

• Integrating unsupervised and supervised word segmentation: The role of goodness measuresHai Zhao, Chunyu Kit — 2011
• Entropy-based Training Data Selection for Domain AdaptationYan Song, Prescott Klassen, Fei Xia — 2012
• Unsupervised Segmentation Helps Supervised Learning of Character Tagging for Word Segmentation and Named Entity RecognitionHai Zhao, Chunyu Kit — 2008
• ii PrefaceGuang-yi Jin — 2007

Related papers

• Overview of the Chinese Word Segmentation AlgorithmGan Qiu-yu — 2013
• The Review of Chinese Word Segmentation TechnologyZheng Zhang — 2010
• Chinese Word Segmentation Using Minimal Linguistic KnowledgeAitao Chen — 2003
• An Improved Mechanism on the Chinese Word SegmentationYi Guo — 2008
• A Research of Chinese Word Segmentation System Based on Automatic LearningDeng Shuguang — 2006
• A Probe into "Words" and Word Segmentation Norm in Chinese Information Processing——A Comparison of Word Segmentation Norm and OrthographyBo Zheng — 2011
• Chinese Word Segmentation Based on Contextual EntropyJin Huang, David Powers — 2003
• Neural Chinese word segmentation with dictionaryJunxin Liu, Fangzhao Wu, Chuhan Wu — 2019
• A Survey of Chinese Word Segmentation and the Application in Information RetrievalJingui Pan — 2004