The Effects of Hyperparameters on SGD Training of Neural Networks

Keywords

Hyperparameter, Artificial neural network, Artificial intelligence, Machine learning, Training (meteorology)

References

• Torch: a modular machine learning software libraryRonan Collobert, Samy Bengio, Johnny Mariéthoz — 2002
• Comparison and Combination of State-of-the-art Techniques for Handwritten Character Recognition: Topping the MNIST BenchmarkDaniel Keysers — 2007
• The mnist database of handwritten digitsYann LeCun, Corinna Cortes — 2005

Cited by

• No More Pesky Learning Rate Guessing GamesLeslie N. Smith — 2015
• FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute ClustersForrest Iandola, Matthew W. Moskewicz, Khalid Ashraf — 2015
• Optimizing deep learning hyper-parameters through an evolutionary algorithmSteven R. Young, Derek Rose, Thomas P. Karnowski — 2015
• Operational calculus on programming spaces and generalized tensor networksŽiga Sajovic, Martin Vuk — 2016
• Cyclical Learning Rates for Training Neural NetworksLeslie N. Smith — 2015
• Exploring the Design Space of Deep Convolutional Neural Networks at Large ScaleForrest Iandola — 2016
• Assessment and analysis of the applicability of recurrent neural networks to natural language understanding with a focus on the problem of coreference resolutionFranz David Kaumanns — 2016
• Operational Calculus for Differentiable ProgrammingŽiga Sajovic — 2016
• Deep learning analytics for diagnostic support of breast cancer disease managementTiancheng He, Mamta Puppala, Richard Ogunti — 2017
• Model Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic StudySuyog Gupta, Wei Zhang, Fei Wang — 2015
• Paleo: A Performance Model for Deep Neural NetworksHang Qi, Evan Sparks, Ameet Talwalkar — 2016
• A Resizable Mini-batch Gradient Descent based on a Multi-Armed BanditSeong Jin Cho, Sunghun Kang, Chang D. Yoo — 2017
• A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural NetworksTushar Semwal, Promod Yenigalla, Gaurav Mathur — 2018
• Protein Family-Specific Models Using Deep Neural Networks and Transfer Learning Improve Virtual Screening and Highlight the Need for More DataFergus Imrie, A.R. Bradley, Mihaela van der Schaar — 2018
• Measuring the Effects of Data Parallelism on Neural Network TrainingChristopher J. Shallue, Jaehoon Lee, Joseph M. Antognini — 2018
• Deep learning for supernovae detectionG. Amar — 2017
• Applicability of Recurrent Neural Networks to Player Data Analysis in Freemium Video GamesJonathan Tan — 2019
• A Comparative Study on Regularization Strategies for Embedding-based Neural NetworksHao Peng, Lili Mou, Ge Li — 2015
• Visual Ensemble Analysis to Study the Influence of Hyper-parameters on Training Deep Neural NetworksSagad Hamid, Adrian Derstroff, Sören Klemm — 2019
• A Sensitivity Analysis of Attention-Gated Convolutional Neural Networks for Sentence ClassificationYang Liu, Jianpeng Zhang, Chao Gao — 2019

Related papers

• Hyperparameter optimization to improve bug prediction accuracyHaidar Osman, Mohammad Ghafari, Oscar Nierstrasz — 2017
• To tune or not to tune? An Approach for Recommending Important HyperparametersMohamadjavad Bahmani, Radwa El Shawi, Nshan Potikyan — 2021
• Stealing Hyperparameters in Machine LearningBinghui Wang, Neil Zhenqiang Gong — 2018
• Bayesian Optimization Machine Learning Models for True and Fake News ClassificationGaohua Zhao, Shouyou Song, Hao Lin — 2023
• Hyperparameter Tuning for Overlapped Software Defect Prediction Data setsShivani Gupta, Kusum Lata Jain, Smaranika Mohapatra — 2020