Continual Learning with Deep Generative Replay
Explore this paper's citation graph
Summary
The Deep Generative Replay is proposed, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"), with only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task.
- Type
- preprint
- Published
- 2017-05-01
- Cited by
- 2,563
- References
- 36
- Access
- Open access
- OpenAlex
- https://openalex.org/W2618767506
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:1888776
Keywords
Computer science, Forgetting, Generative grammar, Task (project management), Artificial intelligence
References
- Deep Boltzmann Machines
- Using fast weights to deblur old memories
- Multiple Classifier Systems
- Auto-Encoding Variational Bayes
- Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall
- Creating a False Memory in the Hippocampus
- Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.
- Catastrophic forgetting in connectionist networks.
- Long-term dendritic spine stability in the adult cortex
- A biologically inspired dual-network memory model for reduction of catastrophic forgetting
- Dropout: a simple way to prevent neural networks from overfitting
- Play it again: reactivation of waking experience and memory.
- Compete to Compute
- Evidence for large long-term memory capacities in baboons and pigeons and its implications for learning and the evolution of cognition
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
- Gradient-based learning applied to document recognition
- Catastrophic Forgetting, Rehearsal and Pseudorehearsal
- THE DEVELOPMENT OF OBJECT CATEGORIZATION IN YOUNG CHILDREN: HIERARCHICAL INCLUSIVENESS, AGE, PERCEPTUAL ATTRIBUTE, AND GROUP VERSUS INDIVIDUAL ANALYSES
- Memory retention--the synaptic stability versus plasticity dilemma.
- Human-level control through deep reinforcement learning
Cited by
- DynMat, a network that can learn after learning
- Lifelong Generative Modeling
- Transfer Learning for Cross-Dataset Recognition: A Survey
- Deep Generative Dual Memory Network for Continual Learning
- Less-forgetful Learning for Domain Expansion in Deep Neural Networks
- Overcoming catastrophic forgetting with hard attention to the task
- Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
- Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
- Continual Lifelong Learning with Neural Networks: A Review
- Scalable Recollections for Continual Lifelong Learning
- Bayesian Gradient Descent: Online Variational Bayes Learning with Increased Robustness to Catastrophic Forgetting and Weight Pruning
- Born Again Neural Networks
- Towards Robust Evaluations of Continual Learning
- SupportNet: solving catastrophic forgetting in class incremental learning with support data
- Continual Learning for Deep Dense Prediction
- Doubly Nested Network for Resource-Efficient Inference
- Meta Continual Learning
- On catastrophic forgetting and mode collapse in Generative Adversarial Networks
- Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
- Memory Replay GANs: learning to generate images from new categories without forgetting