Re-evaluating the Role of Bleu in Machine Translation Research

Keywords

BLEU, Machine translation, Computer science, Translation (biology), Metric (unit)

References

• Europarl: A Parallel Corpus for Statistical Machine TranslationPhilipp Koehn — 2005
• A Smorgasbord of Features for Statistical Machine TranslationFranz Josef Och, Daniel Gildea, Sanjeev Khudanpur — 2004
• Correlating automated and human assessments of machine translation qualityDeborah A. Coughlin — 2003
• Linear B System Description for the 2005 NIST MT Evaluation ExerciseChris Callison-Burch — 2005
• Automatic evaluation of machine translation quality using n-gram co-occurrence statisticsGeorge R. Doddington — 2002
• Precision and Recall of Machine TranslationI. Dan Melamed, Ryan Green, Joseph Turian — 2003
• Bleu: a Method for Automatic Evaluation of Machine TranslationKishore Papineni, Salim Roukos, Todd J. Ward — 2002
• METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human JudgmentsSatanjeev Banerjee, Alon Lavie — 2005
• Extending the BLEU MT Evaluation Method with Frequency WeightingsBogdan Babych, Anthony Hartley — 2004
• Word Sense Disambiguation vs. Statistical Machine TranslationMarine Carpuat, Dekai Wu — 2005
• Minimum Error Rate Training in Statistical Machine TranslationFranz Josef Och — 2003
• Automatic Evaluation of Summaries Using N-gram Co-occurrence StatisticsChin-Yew Lin, Eduard Hovy — 2003
• Discriminative Training and Maximum Entropy Models for Statistical Machine TranslationFranz Josef Och, Hermann Ney — 2002
• A Unified Framework For Automatic Evaluation Using 4-Gram Co-occurrence StatisticsRadu Soricut, Eric Brill — 2004
• Holy and unholy grailsEduard Hovy, Deepak Ravichandran — 2003
• Pharaoh: a beam search decoder for phrase-based statistical machine translation modelsPhilipp Koen — 2004
• Pharaoh: A Beam Search Decoder for Phrase-Based Statistical Machine Translation ModelsPhilipp Koehn — 2004
• Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics2002
• NIST 2005 machine translation evaluation official resultsAudrey J Lee, Mark A. Przybocki — 2005
• Automatic Evaluation of Translation Quality : Outline of Methodology and Report on Pilot ExperimentH. Thompson — 1991

Cited by

• Automatic generation of natural language summariesΔημήτριος Γαλάνης — 2012
• -ing words in RBMT: multilingual evaluation and exploration of pre- and post-processing solutionsNora Aranberri — 2010
• Génération de phrases multilingues par apprentissage automatique de modèles de phrases. (Multilingual Natural Language Generation using sentence models learned from corpora)Éric Charton — 2010
• Comparing rule-based and data-driven approaches to Spanish-to-Basque machine translationGorka Labaka, Nicolas Stroppa, Andy Way — 2007
• Recycling texts: human evaluation of example-based machine translation subtitles for DVDMarian Flanagan — 2009
• TMX Markup: A Challenge When Adapting SMT to the Localisation EnvironmentJinhua Du, Johann Roturier, Andy Way — 2010
• An Investigation into Automatic Translation of Prepositions in IT Technical Documentation from English to ChineseYanli Sun — 2010
• Acquisition automatique de sens pour la désambiguïsation et la sélection lexicale en traductionMarianna Apidianaki — 2008
• TransBooster:black box optimisation of machine translation systemsBart Mellebeek — 2007
• Sensitivity of Automated MT Evaluation Metrics on Higher Quality MT Output: BLEU vs Task-Based Evaluation MethodsBogdan Babych, Anthony Hartley — 2008
• Applying Automated Metrics to Speech Translation DialogsSherri Condon, Jon Phillips, Christy Doran — 2008
• Evaluating the impact of variation in automatically generated embodied object descriptionsMary Ellen Foster — 2007
• Reference-based vs. task-based evaluation of human language technologyAndréi Popescu-Belis — 2008
• Compound Processing for Phrase-Based Statistical Machine TranslationSara Stymne — 2009
• Investigating the effects of controlled language on the reading and comprehension of machine translated texts: A mixed-methods approachStephen Doherty — 2012
• Text Content and Task Performance in the Evaluation of a Natural Language Generation SystemAlbert Gatt — 2009
• Lexical syntax for statistical machine translationHany Hassan — 2009
• Learning Labelled Dependencies in Machine Translation EvaluationYifan He, Andy Way — 2009
• Towards Heterogeneous Automatic MT Error AnalysisJesús Giménez, Lluı́s Màrquez — 2008
• Using F-structures in machine translation evaluationKarolina Owczarzak, Josef van Genabith, Yvette Graham — 2007

Related papers

• Statistical Significance Tests for Machine Translation EvaluationPhilipp Koehn — 2004
• Europarl: A Parallel Corpus for Statistical Machine TranslationPhilipp Koehn — 2005
• Further Meta-Evaluation of Machine TranslationChris Callison-Burch, Cameron Shaw Fordyce, Philipp Koehn — 2008
• Syntactic Features for Evaluation of Machine TranslationDing Liu, Daniel Gildea — 2005
• A Systematic Comparison of Various Statistical Alignment ModelsFranz Josef Och, Hermann Ney — 2003
• ROUGE: A Package for Automatic Evaluation of SummariesChin-Yew Lin — 2004
• Discriminative Training and Maximum Entropy Models for Statistical Machine TranslationFranz Josef Och, Hermann Ney — 2002
• Statistical Phrase-Based TranslationPhilipp Koehn, Franz Josef Och, Daniel Marcu — 2003
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATIONMatthew Snover, Bonnie J. Dorr, Richard Schwartz — 2005
• (Meta-) Evaluation of Machine TranslationChris Callison-Burch, Cameron Shaw Fordyce, Philipp Koehn — 2007