Semi-Amortized Variational Autoencoders
Explore this paper's citation graph
Summary
This work proposes a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them, which enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation.
- Type
- preprint
- Published
- 2018-02-07
- Cited by
- 268
- References
- 69
- Access
- Open access
- OpenAlex
- https://openalex.org/W2785734416
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:334803
Keywords
Inference, Generative grammar, Computer science, Differentiable function, Artificial intelligence
References
- Efficient Learning of Deep Boltzmann Machines
- A Recurrent Latent Variable Model for Sequential Data
- An Introduction to Variational Methods for Graphical Models
- Accurate and conservative estimates of MRF log-likelihood using reverse annealing
- Minimizing finite sums with the stochastic average gradient
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Fast Exact Multiplication by the Hessian
- Propagation Algorithms for Variational Bayesian Learning
- Multilayer feedforward networks are universal approximators
- Automatic Learning Rate Maximization by On-Line Estimation of the Hessian's Eigenvectors
- Training energy-based models for time-series imputation
- Stochastic variational inference
- Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure
- Generic Methods for Optimization-Based Modeling
- Human-level concept learning through probabilistic program induction
- Deep Residual Learning for Image Recognition
- Generating Sentences from a Continuous Space
- Variational Inference: A Review for Statisticians
- Hierarchical Variational Models
- A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Cited by
- Decision-Theoretic Meta-Learning: Versatile and Efficient Amortization of Few-Shot Learning
- Adaptive path-integral autoencoder: representation learning and planning for dynamical systems
- Item Recommendation with Variational Autoencoders and Heterogeneous Priors
- A Review of Learning with Deep Generative Models from perspective of graphical modeling
- Approximate inference: new visions
- Coupled Variational Bayes via Optimization Embedding
- VoxelMorph: A Learning Framework for Deformable Medical Image Registration
- Fairness through Causal Awareness: Learning Causal Latent-Variable Models for Biased Data
- Meta-Learning Probabilistic Inference for Prediction
- A Tutorial on Deep Latent Variable Models of Natural Language
- Conditional Variational Autoencoder for Neural Machine Translation
- Initialized Equilibrium Propagation for Backprop-Free Training
- Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions
- Amortized Bayesian Meta-Learning
- Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
- Latent Normalizing Flows for Discrete Sequences
- Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
- Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
- Benchmarking Approximate Inference Methods for Neural Structured Prediction
- Effective Estimation of Deep Generative Language Models