Parallel Architecture and Hyperparameter Search via Successive Halving and Classification
Explore this paper's citation graph
Summary
This work presents a simple and powerful algorithm for parallel black box optimization called Successive Halving and Classification (SHAC), which operates in stages of parallel function evaluations and trains a cascade of binary classifiers to iteratively cull the undesirable regions of the search space.
- Type
- preprint
- Published
- 2018-05-25
- Cited by
- 27
- References
- 52
- Access
- Open access
- OpenAlex
- https://openalex.org/W2803453473
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:44061260
Keywords
Hyperparameter, Computer science, Classifier (UML), Hyperparameter optimization, Binary number
References
- Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology
- Classification and Regression by randomForest
- Non-stochastic Best Arm Identification and Hyperparameter Optimization
- Designing Neural Networks using Genetic Algorithms
- Greedy function approximation: A gradient boosting machine.
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Scalable Bayesian Optimization Using Deep Neural Networks
- Portfolio Selection
- Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, by John H. Holland MIT Press (Bradford Books), Cambridge, Mass., 1992, xiv+211 pp. (Paperback £13.50, cloth £26.95)
- Dropout: a simple way to prevent neural networks from overfitting
- Going deeper with convolutions
- Algorithms for Hyper-Parameter Optimization
- Evolving Neural Networks through Augmenting Topologies
- Introductory Lectures on Convex Optimization - A Basic Course
- Practical Bayesian Optimization of Machine Learning Algorithms
- ImageNet classification with deep convolutional neural networks
- Gradient-Based Optimization of Hyperparameters
- Deep Residual Learning for Image Recognition
- XGBoost: A Scalable Tree Boosting System
- Designing Neural Network Architectures using Reinforcement Learning
Cited by
- AutoAugment: Learning Augmentation Policies from Data
- Pathological Voice Classification Using Mel-Cepstrum Vectors and Support Vector Machine
- Reducing the Search Space for Hyperparameter Optimization Using Group Sparsity
- AutoAugment: Learning Augmentation Strategies From Data
- Feature Partitioning for Efficient Multi-Task Architectures
- A Survey of Deep Learning Applications to Autonomous Vehicle Control
- Final Report for Creative Component
- TPOT-SH: A Faster Optimization Algorithm to Solve the AutoML Problem on Large Datasets
- Using a thousand optimization tasks to learn hyperparameter search strategies
- Reusing Trained Layers of Convolutional Neural Networks to Shorten Hyperparameters Tuning Time
- Hyperparameter Optimization in Neural Networks via Structured Sparse Recovery
- Towards a General Framework for ML-based Self-tuning Databases
- MOFA: Modular Factorial Design for Hyperparameter Optimization
- SOLO: Search Online, Learn Offline for Combinatorial Optimization Problems
- Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies
- Efficient Data Augmentation Policy for Electrocardiograms
- Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning
- DSP: A Deep Neural Network Approach for Serving Cell Positioning in Mobile Networks
- Predicting Sepsis Mortality Using Machine Learning Methods
- Prediction of sepsis mortality in ICU patients using machine learning methods
Related papers
- Hyperband: a novel bandit-based approach to hyperparameter optimization
- Practical Bayesian Optimization of Machine Learning Algorithms
- Long Short-Term Memory
- Deep Residual Learning for Image Recognition
- Algorithms for Hyper-Parameter Optimization
- Random search for hyper-parameter optimization
- CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification
- Reinforcement-based Method for Simultaneous Clustering Algorithm Selection and its Hyperparameters Optimization