Scalable Bayesian Optimization Using Deep Neural Networks
Explore this paper's citation graph
Summary
This work shows that performing adaptive basis function regression with a neural network as the parametric form performs competitively with state-of-the-art GP-based approaches, but scales linearly with the number of data rather than cubically, which allows for a previously intractable degree of parallelism.
- Type
- preprint
- Published
- 2015-02-19
- Cited by
- 1,152
- References
- 64
- Access
- Open access
- OpenAlex
- https://openalex.org/W1798702550
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:12604141
Keywords
Computer science, Bayesian optimization, Hyperparameter, Artificial intelligence, Artificial neural network
References
- Regularization of Neural Networks using DropConnect
- Collaborative hyperparameter tuning
- Adaptive MCMC with Bayesian Optimization
- Variational Learning of Inducing Variables in Sparse Gaussian Processes
- Anomaly Detection and Removal Using Non-Stationary Gaussian Processes
- An Experimental Evaluation of Bayesian Optimization on Bipedal Locomotion
- Pattern Recognition and Machine Learning
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Automated Machine Learning on Big Data using Stochastic Algorithm Tuning
- Recurrent Neural Network Regularization
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Manifold Gaussian Processes for regression
- Improving neural networks by preventing co-adaptation of feature detectors
- Auto-Encoding Variational Bayes
- Keeping the neural networks simple by minimizing the description length of the weights
- A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
- Long Short-Term Memory
- Marginalized Neural Network Mixtures for Large-Scale Regression
- Sparse Gaussian Processes using Pseudo-inputs
- Algorithms for Hyper-Parameter Optimization
Cited by
- GP-Select: Accelerating EM Using Adaptive Subspace Preselection
- Robust optimization of SVM hyperparameters in the classification of bioactive compounds
- Spectral Representations for Convolutional Neural Networks
- Constrained Bayesian Optimization and Applications
- Scalable Bayesian Kernel Models with Variable Selection
- Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
- GradNets: Dynamic Interpolation Between Neural Architectures
- Taking the Human Out of the Loop: A Review of Bayesian Optimization
- Distilling Reverse-Mode Automatic Differentiation (DrMAD) for Optimizing Hyperparameters of Deep Neural Networks
- Metric Learning with Adaptive Density Discrimination
- FLASH: Fast Bayesian Optimization for Data Analytic Pipelines
- Convolutional Tables Ensemble: classification in microseconds
- Do Deep Convolutional Nets Really Need to be Deep (Or Even Convolutional)?
- Object Recognition Based on Amounts of Unlabeled Data
- A Stratified Analysis of Bayesian Optimization Methods
- Deep Residual Networks with Exponential Linear Unit
- Probabilistic Integration
- Bayesian Approximate Kernel Regression with Variable Selection
- Multi-Bias Non-linear Activation in Deep Neural Networks
- Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
Related papers
- Stable Bayesian optimization
- SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
- Exploiting Configurations of MaxSAT Solvers
- Towards Assessing the Impact of Bayesian Optimization's Own Hyperparameters
- Automatic tuning of hyperparameters using Bayesian optimization
- Energy Efficient Hyperparameters Tuning through Augmented Gaussian Processes and Multi-information Source optimization
- Active Learning of Linear Embeddings for Gaussian Processes
- Extrapolative Bayesian Optimization with Gaussian Process and Neural Network Ensemble Surrogate Models