Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
Explore this paper's citation graph
Summary
A novel decoding strategy motivated by an earlier observation that nonlinear hidden layers of a deep neural network stretch the data manifold is proposed, which is embarrassingly parallelizable without any communication overhead, while improving an existing decoding algorithm.
- Type
- preprint
- Published
- 2016-05-12
- Cited by
- 68
- References
- 32
- Access
- Open access
- OpenAlex
- https://openalex.org/W2353655624
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:14830778
Keywords
Decoding methods, Computer science, Language model, Artificial intelligence, Natural language processing
References
- ADADELTA: An Adaptive Learning Rate Method
- Recurrent neural network based language model
- Generating Text with Recurrent Neural Networks
- A Recurrent Latent Variable Model for Sequential Data
- Foundations of Statistical Natural Language Processing
- Better Mixing via Deep Representations
- Learning representations by back-propagating errors
- Book Reviews: Foundations of Statistical Natural Language Processing
- Neural Machine Translation of Rare Words with Subword Units
- Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks
- Auto-Encoding Variational Bayes
- From Feedforward to Recurrent LSTM Neural Networks for Language Modeling
- Long Short-Term Memory
- A simple recursive numerical method for Bermudan option pricing under Lévy processes
- Learning long-term dependencies with gradient descent is difficult
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
- Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models
- Task Loss Estimation for Sequence Prediction
- Mutual Information and Diverse Decoding Improve Neural Machine Translation
- Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Cited by
- A Simple, Fast Diverse Decoding Algorithm for Neural Generation
- Neural Combinatorial Optimization with Reinforcement Learning
- Decoding as Continuous Optimization in Neural Machine Translation
- Learning to Decode for Future Success
- Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
- Towards Decoding as Continuous Optimisation in Neural Machine Translation
- Neural Machine Translation
- Learning a Generative Model for Validity in Complex Discrete Structures
- SIC-GAN: A Self-Improving Collaborative GAN for Decoding Sketch RNNs
- Analyzing Uncertainty in Neural Machine Translation
- Levenshtein Transformer
- Multi-Turn Beam Search for Neural Dialogue Modeling
- Biological applications, visualizations, and extensions of the long short-term memory network
- Comparison of Diverse Decoding Methods from Conditional Language Models
- Can Unconditional Language Models Recover Arbitrary Sentences?
- On the use of prior and external knowledge in neural sequence models
- Trainable Greedy Decoding for Neural Machine Translation
- Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
- A Stable and Effective Learning Strategy for Trainable Greedy Decoding
- Non-Autoregressive Neural Machine Translation
Related papers
- Remarks on Algorithm 2, Algorithm 3, Algorithm 15, Algorithm 25 and Algorithm 26
- Remarks on Algorithm 332: Jacobi polynomials: Algorithm 344: student's t-distribution: Algorithm 351: modified Romberg quadrature: Algorithm 359: factoral analysis of variance
- An optimized Inactivation Decoding of BATS Codes
- Reduction of sphere decoding complexity using an adaptive SD-OSIC system
- An Improved Decoding Method for IS-OFDM
- A Study of Joint Equalization and TCM Decoding with Multiple Decoding Depths
- On Achievable Rates for Relay Channels
- An enhanced BP secondary decoding algorithm
- A simplified multistage decoding method using threshold technique for multilevel coding