Categorical Reparameterization with Gumbel-Softmax
Explore this paper's citation graph
Summary
It is shown that the Gumbel-Softmax estimator outperforms state-of-the-art gradient estimators on structured output prediction and unsupervised generative modeling tasks with categorical latent variables, and enables large speedups on semi-supervised classification.
- Type
- preprint
- Published
- 2016-11-03
- Cited by
- 6,647
- References
- 33
- Access
- Open access
- OpenAlex
- https://openalex.org/W2547875792
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:2428314
Keywords
Categorical variable, Gumbel distribution, Softmax function, Estimator, Latent variable
References
- Techniques for Learning Binary Stochastic Feedforward Neural Networks
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Gradient Estimation Using Stochastic Computation Graphs
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Auto-Encoding Variational Bayes
- The conference paper
- Likelihood ratio gradient estimation for stochastic systems
- On the quantitative analysis of deep belief networks
- Deep AutoRegressive Networks
- Semi-supervised Learning with Deep Generative Models
- Variational Bayesian Inference with Stochastic Search
- The Neural Autoregressive Distribution Estimator
- Rethinking the Inception Architecture for Computer Vision
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- A* Sampling
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- Hierarchical Multiscale Recurrent Neural Networks
- Discrete Variational Autoencoders
- Hybrid computing using a neural network with dynamic external memory
- Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
Cited by
- A Learning Framework for Winner-Take-All Networks with Stochastic Synapses
- Learning to Compose and Reason with Language Tree Structures for Visual Grounding
- High Dimensional Inference With Random Maximum A-Posteriori Perturbations
- Deep Generative Models for Spectroscopic Analysis on Mars
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
- NIPS 2016 Tutorial: Generative Adversarial Networks
- Memory Augmented Neural Networks with Wormhole Connections
- Semi-parametric Network Structure Discovery Models
- Coarse Grained Exponential Variational Autoencoders
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images
- Differentiable Scheduled Sampling for Credit Assignment
- Semi-Supervised Generation with Cluster-aware Generative Models
- Amortized Inference and Learning in Latent Conditional Random Fields for Weakly-Supervised Semantic Image Segmentation
- Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning
- Stochastic Sequential Neural Networks with Structured Inference
- Lifelong Generative Modeling
- Black-Box Attacks against RNN based Malware Detection Algorithms
- A Regularized Framework for Sparse and Structured Neural Attention
- Model-Based Planning with Discrete and Continuous Actions
Related papers
- Evaluating Measurement Invariance in Categorical Data Latent Variable Models with the EPC-Interest
- Causal effect analysis in nonrandomized data with latent variables and categorical indicators: The implementation and benefits of EffectLiteR.
- New Approaches for Examining Associations with Latent Categorical Variables: Applications to Substance Abuse and Aggression
- On Examining the Underlying Normal Variable Assumption in Latent Variable Models With Categorical Indicators
- Latent Variable Interactions with Categorical Indicators: Continuous and Categorical Latent Moderated Structural Equations Approaches
- Tackling Challenges in Data Pooling: Missing Data Handling in Latent Variable Models with Continuous and Categorical Indicators
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution