BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Explore this paper's citation graph
Summary
BART is presented, a denoising autoencoder for pretraining sequence-to-sequence models, which matches the performance of RoBERTa on GLUE and SQuAD, and achieves new state-of-the-art results on a range of abstractive dialogue, question answering, and summarization tasks.
- Type
- preprint
- Published
- 2019-10-29
- Cited by
- 13,168
- References
- 36
- Access
- Open access
- OpenAlex
- https://openalex.org/W2982399380
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:204960716
Keywords
Machine translation, Computer science, Artificial intelligence, Natural language processing, Translation (biology)
References
- Automatically Constructing a Corpus of Sentential Paraphrases
- Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment
- Teaching Machines to Read and Comprehend
- Efficient Estimation of Word Representations in Vector Space
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
- Edinburgh Neural Machine Translation Systems for WMT 16
- The PASCAL Recognising Textual Entailment Challenge
- Get To The Point: Summarization with Pointer-Generator Networks
- A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
- Deep Contextualized Word Representations
- Neural Network Acceptability Judgments
- Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization
- Gaussian Error Linear Units (GELUs)
- Cross-lingual Language Model Pretraining
- Pre-trained language model representations for language generation
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- ELI5: Long Form Question Answering
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- SpanBERT: Improving Pre-training by Representing and Predicting Spans
Cited by
- Natural language processing
- An Attentive Survey of Attention Models
- Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
- c-TextGen: Conditional Text Generation for Harmonious Human-Machine Interaction
- Transformers: State-of-the-Art Natural Language Processing
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
- CommonGen: A Constrained Text Generation Dataset Towards Generative Commonsense Reasoning
- Machines Getting with the Program: Understanding Intent Arguments of Non-Canonical Directives
- Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline
- A Survey on Document-level Machine Translation: Methods and Evaluation
- RobBERT: a Dutch RoBERTa-based Language Model
- Neural Abstractive Text Summarization with Sequence-to-Sequence Models
- Multilingual Denoising Pre-training for Neural Machine Translation
- ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation
- A Multilingual View of Unsupervised Machine Translation
- UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
- Learning by Semantic Similarity Makes Abstractive Summarization Better
- Lite Transformer with Long-Short Range Attention
- A Primer in BERTology: What We Know About How BERT Works