A Hierarchical Bayesian Language Model Based On Pitman-Yor Processes

Keywords

Dirichlet distribution, Computer science, Language model, Smoothing, Bayesian probability

References

• Combinatorial Stochastic ProcessesJim Pitman, Jean Picard — 2006
• Improved backing-off for M-gram language modelingReinhard Kneser, Hermann Ney — 1995
• Gibbs Sampling Methods for Stick-Breaking PriorsHemant Ishwaran, Lancelot F. James — 2001
• The two-parameter Poisson-Dirichlet distribution derived from a stable subordinatorJim Pitman, Marc Yor — 1997
• An Empirical Study of Smoothing Techniques for Language ModelingStanley F. Chen, Joshua Goodman — 1996
• Two decades of statistical language modeling: where do we go from here?Roni Rosenfeld — 2000
• A bit of progress in language modelingJoshua Goodman — 2001
• A hierarchical Dirichlet language modelDavid Mackay, Linda C. Bauman Peto — 1995
• Hierarchical Dirichlet ProcessesYee Whye Teh, Michael I. Jordan, Matthew J. Beal — 2006
• Interpolating between types and tokens by estimating power-law generatorsSharon Goldwater, Mark Johnson, Thomas L. Griffiths — 2005
• Estimation of probabilities in the language model of the IBM speech recognition systemArthur Nádas — 1984
• A Neural Probabilistic Language ModelBengioYoshua, DucharmeRéjean, VincentPascal — 2003
• A Bayesian Interpretation of Interpolated Kneser-NeyYW Teh — 2006
• An empirical study of smoothing techniques for language modelingStanley F. Chen, Joshua Goodman — 1999
• Exponential Priors for Maximum Entropy ModelsJoshua Goodman — 2004

Cited by

• Forgetting Counts: Constant Memory Inference for a Dependent Hierarchical Pitman-Yor ProcessNicholas Bartlett, David Pfau, Frank Wood — 2010
• Given Bilingual Terminology in Statistical Machine Translation: MWE-Sensitve Word Alignment and Hierarchical Pitman-Yor Process-Based Translation Model SmoothingTsuyoshi Okita, Andy Way — 2011
• Word alignment and smoothing methods in statistical machine translation: Noise, prior knowledge and overfittingTsuyoshi Okita — 2012
• Hyper Markov Non-Parametric Processes for Mixture Modeling and Model SelectionDaniel Heinz — 2010
• Pitfalls in the use of Parallel Inference for the Dirichlet ProcessYarin Gal, Zoubin Ghahramani — 2014
• Approximation and Relaxation Approaches for Parallel and Distributed Machine LearningStephen Tyree — 2014
• Training Continuous Space Language Models: Some Practical IssuesHai Son Le, Alexandre Allauzen, Guillaume Wisniewski — 2010
• Knowledge integration into language models: a random forest approachFrederick Jelinek, Yi Su — 2009
• Smoothing for Bracketing InductionXiangyu Duan, Min Zhang, Wenliang Chen — 2013
• Clustering Words by Projection EntropyIşık Barış Fidaner, Ali Taylan Cemgil — 2014
• Unsupervised Part of Speech Inference with Particle FiltersGregory Dubbin, Phil Blunsom — 2012
• An Introduction to Bayesian Nonparametric ModellingYee Whye Teh — 2009
• Phrase Alignment Models for Statistical Machine TranslationJohn DeNero — 2010
• Cooperative Semantic Information Processing for Literature-Based Biomedical Knowledge DiscoveryZhiguo Yu — 2013
• Measuring the Influence of Long Range Dependencies with Neural Network Language ModelsHai-Son Le, Alexandre Allauzen, François Yvon — 2012
• Computational Linguistic Models of Deceptive Opinion SpamMyle Ott — 2013
• Probability of Belonging to a LanguageKevin Cook — 2013
• Joint Phrase Alignment and Extraction for Statistical Machine TranslationGraham Neubig, Taro Watanabe, Eiichiro Sumita — 2012
• Parallel Markov Chain Monte Carlo for Pitman-Yor Mixture ModelsAvinava Dubey, Sinead A. Williamson, Eric P. Xing — 2014
• 可変長階層 Pitman-Yor 言語モデルを用いたメロディー生成手法の提案Akira SHIRAI, Tadahiro Taniguchi — 2013

Related papers

• Effectiveness of Two-Step Smoothing in Digital Control ComputersRobert Spero — 1953
• Algorithms for Surface Smoothing in Reverse EngineeringPing Xi, Yong Liu — 2002
• Noise reduction for mesh smoothing of 3D mesh dataDae-Hwan Hyeon, Taeg Keun Whangbo — 2009
• Parabolic smoothing simplifiedGianluigi Giannelli, O. Altamura — 1976
• The Smoothing Method for Finding PRIs from the PRI Transform SpectrumShuhang Wu, Weiji Su, Lin Zhang — 2008
• Applications of a nonlinear smoothing algorithm to speech processingL. R. Rabiner, M. R. Sambur, Christiane Schmidt — 1975
• A study of smoothing methods for language models applied to information retrievalChengXiang Zhai, John Lafferty — 2004
• A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information RetrievalChengXiang Zhai, John Lafferty — 2001
• Axiomatic Analysis of Smoothing Methods in Language Models for Pseudo-Relevance FeedbackHussein Hazimeh, ChengXiang Zhai — 2015