Distributed Asynchronous Online Learning for Natural Language Processing

Keywords

Computer science, Asynchronous communication, Exploit, Focus (optics), Artificial intelligence

References

• On-line EM Algorithm for the Normalized Gaussian NetworkJames McSherry — 2000
• Why Doesn’t EM Find Good HMM POS-Taggers?Mark Johnson — 2007
• Building a Large Annotated Corpus of English: The Penn TreebankMitchell P. Marcus — 1993
• Log-Linear Models for Wide-Coverage CCG ParsingStephen Clark, James Curran — 2003
• The Mathematics of Statistical Machine Translation: Parameter EstimationPeter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra — 1993
• Online EM for Unsupervised ModelsPercy Liang, Dan Klein — 2009
• Learning structured prediction models: a large margin approachBen Taskar, Vassil Chatalbashev, Daphne Koller — 2005
• Large Margin Methods for Structured and Interdependent Output VariablesIoannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann — 2005
• Online Large-Margin Training of Syntactic and Structural Translation FeaturesDavid Chiang, Yuval Marton, Philip Resnik — 2008
• Fully distributed EM for very large datasetsJason Wolfe, Aria Haghighi, Dan Klein — 2008
• Gradient-based learning applied to document recognitionYann LeCun, Léon Bottou, Yoshua Bengio — 1998
• Fast, Easy, and Cheap: Construction of Statistical Machine Translation Models with MapReduceChristopher J. Dyer, Aarón Córdova, Alex Mont — 2008
• New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted PerceptronMichael Collins, Nigel Duffy — 2002
• A New Perceptron Algorithm for Sequence Labeling with Non-Local FeaturesJun’ichi Kazama, Kentaro Torisawa — 2007
• Efficient, Feature-based, Conditional Random Field ParsingJenny Rose Finkel, Alex Kleeman, Christopher D. Manning — 2008
• An Evaluation Exercise for Word AlignmentRada Mihalcea, Ted Pedersen — 2003
• Ranking Algorithms for Named Entity Extraction: Boosting and the VotedPerceptronMichael Collins — 2002
• Efficient Large-Scale Distributed Training of Conditional Maximum Entropy ModelsRyan McDonald, Mehryar Mohri, Nathan Silberman — 2009
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence DataJohn Lafferty, Andrew McCallum, Fernando C. N. Pereira — 2001
• Statistical Phrase-Based TranslationPhilipp Koehn, Franz Josef Och, Daniel Marcu — 2003

Cited by

• Predicting Linguistic Structure with Incomplete and Cross-Lingual SupervisionOscar Täckström — 2013
• Iterative parameter mixing for distributed large-margin training of structured predictors for natural language processingGregory Coppola — 2015
• A Named Entity Recognition Method based on Decomposition and Concatenation of Word ChunksTomoya Iwakura, Hiroya Takamura, Manabu Okumura — 2011
• Efficient mini-batch training for stochastic optimizationMu Li, Tong Zhang, Yuqiang Chen — 2014
• Minibatch and Parallelization for Online Large Margin Structured LearningKai Zhao, Liang Huang — 2013
• Fast and Adaptive Online Training of Feature-Rich Translation ModelsSpence Green, Sida Wang, Daniel Cer — 2013
• Optimal Distributed Online Prediction Using Mini-BatchesOfer Dekel, Ran Gilad-Bachrach, Ohad Shamir — 2010
• Optimal Distributed Online PredictionOfer Dekel, Ran Gilad-Bachrach, Ohad Shamir — 2011
• Hope and Fear for Discriminative Training of Statistical Translation ModelsDavid Chiang — 2012
• Fast Distributed Asynchronous SGD with Variance ReductionRuiliang Zhang, Shuai Zheng, James T. Kwok — 2015
• A zealous parallel gradient descent algorithmGilles Louppe, Pierre Geurts — 2010
• A system analysis of improvements in machine learningSabin M. Thomas — 2014
• MapReduce/Bigtable for Distributed OptimizationKeith Hall, Scott Gilpin, Gideon Mann — 2010
• Word Sense Disambiguation: A Structured Learning PerspectiveYun Zhou, Ting Wang, Zhiyuan Wang — 2015
• Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense FeaturesXu Sun — 2016
• Small Batch or Large Batch?: Gaussian Walk with Rebound Can TeachPeifeng Yin, Ping Luo, Taiga Nakamura — 2017
• Iteration acceleration for distributed learning systemsJunxiong Wang, Hongzhi Wang, Chenxu Zhao — 2018
• Random Barzilai-Borwein step size for mini-batch algorithmsZhuang Yang, Cheng Wang, Zhemin Zhang — 2018
• Robust Fully Distributed Minibatch Gradient Descent with Privacy PreservationGábor Danner, Árpád Berta, István Hegedűs — 2018
• Efficient Privacy-preserving Machine Learning in Hierarchical Distributed SystemQi Jia, Linke Guo, Yuguang Fang — 2018

Related papers

• Large Scale Distributed Deep NetworksJay B. Dean, Greg S. Corrado, Rajat Monga — 2012
• Adaptive Subgradient Methods for Online Learning and Stochastic OptimizationJohn C. Duchi, Elad Hazan, Yoram Singer — 2010
• Slow Learners are FastMartin Zinkevich, John Langford, Alex Smola — 2009
• Optimal Distributed Online Prediction Using Mini-BatchesOfer Dekel, Ran Gilad-Bachrach, Ohad Shamir — 2010
• Parallelized Stochastic Gradient DescentMartin Zinkevich, Markus Weimer, Lihong Li — 2010
• Distributed Training Strategies for the Structured PerceptronRyan McDonald, Keith Hall, Gideon Mann — 2010
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence DataJohn Lafferty, Andrew McCallum, Fernando C. N. Pereira — 2001
• Accelerating Stochastic Gradient Descent using Predictive Variance ReductionRie Johnson, Tong Zhang — 2013
• Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron AlgorithmsMichael Collins — 2002
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersStephen Boyd — 2011