Fine-Tuning Language Models from Human Preferences
Explore this paper's citation graph
Summary
This paper builds on advances in generative pretraining of language models to apply reward learning to four natural language tasks: continuing text with positive sentiment or physically descriptive language, and summarization tasks on the TL;DR and CNN/Daily Mail datasets.
- Type
- preprint
- Published
- 2019-09-18
- Cited by
- 2,699
- References
- 53
- Access
- Open access
- OpenAlex
- https://openalex.org/W2973379954
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:202660943
Keywords
Computer science, Natural language processing, Linguistics, Philosophy
References
- Teaching Machines to Read and Comprehend
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books
- Neural Machine Translation of Rare Words with Subword Units
- Image-Based Recommendations on Styles and Substitutes
- Discriminative Batch Mode Active Learning
- Semi-supervised Sequence Learning
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Dialogue Learning With Human-In-The-Loop
- Active Learning for Speech Recognition: the Power of Gradients
- Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
- Learning to Generate Reviews and Discovering Sentiment
- Get To The Point: Summarization with Pointer-Generator Networks
- Proximal Policy Optimization Algorithms
- Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
- TL;DR: Mining Reddit to Learn Automatic Summarization
- Deep Contextualized Word Representations
- Universal Language Model Fine-tuning for Text Classification
- AI safety via debate
- Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning
- Bottom-Up Abstractive Summarization
Cited by
- Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
- Plug and Play Language Models: A Simple Approach to Controlled Text Generation
- Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text
- Learning Norms from Stories: A Prior for Value Aligned Agents
- Fill in the BLANC: Human-free quality estimation of document summaries
- Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning
- MLE-guided parameter search for task loss minimization in neural sequence modeling
- ColdGANs: Taming Language GANs with Cautious Sampling Strategies
- CoCon: A Self-Supervised Approach for Controlled Text Generation
- Quantifying Differences in Reward Functions
- Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation
- Investigation of Sentiment Controllable Chatbot
- DeepClone: Modeling Clones to Generate Code Predictions
- SummEval: Re-evaluating Summarization Evaluation
- Neural Language Generation: Formulation, Methods, and Evaluation
- ETC-NLG: End-to-end Topic-Conditioned Natural Language Generation
- GeDi: Generative Discriminator Guided Sequence Generation
- Controllable Text Generation with Focused Variation
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- Plug-and-Play Conversational Models
Related papers
- ИСПОЛЬЗОВAНИЕ ПОТЕНЦИAЛA СОЦИAЛЬНЫХ ПAРТНЕРОВ В ПОДГОТОВКЕ БУДУЩИХ ПЕДAГОГОВ
- Using DataGrid Control to Realize DataBase of Querying in VB6.0
- Study and Two Types of Typical Usage of DataGrid Web Server Control
- PACWON: A parallelizing compiler for workstations on a network
- Bidirectional Sort and Choosing a Row to Update or Delete by Click Any Cell in DataGrid
- OpenCL-accelerated object classification in video streams using Spatial Pooler of Hierarchical Temporal Memory
- ESKVS: efficient and secure approach for keyframes-based video summarization framework
- Flexible Application of VSFlexGrid