On First-Order Meta-Learning Algorithms
Explore this paper's citation graph
Summary
A family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates, including Reptile, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task.
- Type
- preprint
- Published
- 2018-03-08
- Cited by
- 2,652
- References
- 22
- Access
- Open access
- OpenAlex
- https://openalex.org/W2795900505
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:4587331
Keywords
Initialization, Meta learning (computer science), Task (project management), Computer science, Artificial intelligence
References
- The CMA Evolution Strategy: A Comparing Review
- Meaning and compositionality as statistical induction of categories and constraints
- Using fast weights to deblur old memories
- ImageNet: A large-scale hierarchical image database
- One-Shot Learning with a Hierarchical Nonparametric Bayesian Model
- Parallelized Stochastic Gradient Descent
- Dueling Network Architectures for Deep Reinforcement Learning
- Human-level concept learning through probabilistic program induction
- One shot learning of simple visual concepts
- RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning
- Meta-Learning with Memory-Augmented Neural Networks
- Optimization as a Model for Few-Shot Learning
- Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Matching Networks for One Shot Learning
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- Part-Based R-CNNs for Fine-Grained Category Detection
- Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
- Learning to learn by gradient descent by gradient descent
Cited by
- On Learning Intrinsic Rewards for Policy Gradient Methods
- Gotta Learn Fast: A New Benchmark for Generalization in RL
- Transductive Propagation Network for Few-shot Learning
- Multi-task Maximum Entropy Inverse Reinforcement Learning
- Contextual Policy Optimisation
- Task Agnostic Meta-Learning for Few-Shot Learning
- Auto-Meta: Automated Gradient Based Meta Learner Search
- Meta Continual Learning
- Transfer Learning for Estimating Causal Effects using Neural Networks
- A Meta-Learning Approach for Custom Model Training
- Model-Based Reinforcement Learning via Meta-Policy Optimization
- VPE: Variational Policy Embedding for Transfer Reinforcement Learning
- Generalization and Regularization in DQN
- Multi-Level Semantic Feature Augmentation for One-Shot Learning
- Meta-Learning: A Survey
- Gradient Agreement as an Optimization Objective for Meta-Learning
- Towards Faster Development of Deep Learning Models Using Meta-Learning
- Recurrent Adaptation Networks for Online Signature Verification
- On the reproducibility of gradient-based Meta-Reinforcement Learning baselines
- Deep Comparison: Relation Columns for Few-Shot Learning