An overview of gradient descent optimization algorithms
Explore this paper's citation graph
Summary
A technique is proposed, when a test sample is divided into N parts, on each of which the values of the metrics are calculated, which reduces the amount of computations needed, however, an experimental analysis of the binary cross-entropy metric for CTR prediction models showed that it is more rough than the first one.
- Type
- preprint
- Published
- 2016-09-15
- Cited by
- 7,037
- References
- 31
- Access
- Open access
- OpenAlex
- https://openalex.org/W2523246573
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:17485266
Keywords
Gradient descent, Descent (aeronautics), Algorithm, Computer science, Optimization algorithm
References
- ADADELTA: An Adaptive Learning Rate Method
- Learning to Execute
- A Suggestion for Using Powerful and Informative Tests of Normality
- On the momentum term in gradient descent learning algorithms
- A Stochastic Approximation Method
- Advances in optimizing recurrent networks
- On the training of recurrent neural networks
- The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test
- Delay-Tolerant Algorithms for Asynchronous Distributed Online Learning
- Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
- Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
- Large Scale Distributed Deep Networks
- GloVe: Global Vectors for Word Representation
- Training Recurrent Neural Networks
- Adding Gradient Noise Improves Learning for Very Deep Networks
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Curriculum learning
- Learning rate schedules for faster stochastic gradient search
- Incorporating Nesterov Momentum into Adam
- Efficient BackProp
Cited by
- Training improvements for ultrasound beamforming with deep neural networks
- Intelligent Deep Models Based on Scalograms of Electrocardiogram Signals for Biometrics
- Net-FLICS: fast quantitative wide-field fluorescence lifetime imaging with compressed sensing – a deep learning approach
- Cyclical Learning Rates for Training Neural Networks
- A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements
- Models of Neuronal Stimulus-Response Functions: Elaboration, Estimation, and Evaluation
- Higher Order Neural Networks and Neural Networks for Stream Learning
- Denoising High Resolution Multispectral Images Using Deep Learning Approach
- Language Independent Single Document Image Super-Resolution using CNN for improved recognition
- Mixture density networks for distribution and uncertainty estimation
- Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
- Deep Robust Kalman Filter
- A Study of Gradient-Based Algorithms
- Efficient parallel translating embedding for knowledge graphs
- Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning
- No-show Forecast Using Passenger Booking Data
- Deep Learning Based Regression and Multiclass Models for Acute Oral Toxicity Prediction with Automatic Chemical Feature Extraction
- Energy Harvesting for Wireless Sensor Networks
- Molecular Graph Encoding Convolutional Neural Networks for Automatic Chemical Feature Extraction
- Diagnosing Breast Cancer with a Neural Network
Related papers
- On the sufficient descent condition of the Hager-Zhang conjugate gradient methods
- A NOVEL HYBRID ALGORITHM BASED ON CROW SEARCH ALGORITHM AND WHALE OPTIMIZATION ALGORITHM FOR HIGH-DIMENSIONAL OPTIMIZATION AND FEATURE SELECTION
- A Hybrid Algorithm Based on Invasive Weed Optimization Algorithm and Grey Wolf Optimization Algorithm
- The Harris hawks optimization algorithm, salp swarm algorithm, grasshopper optimization algorithm and dragonfly algorithm for structural design optimization of vehicle components
- A Suggestion Algorithm Instituted on Invasive Weed Optimization Algorithm and Bat Optimization Algorithm
- Improving Moth-Flame Optimization Algorithm by using Slime-Mould Algorithm
- بهینه سازی انتخاب سبد سرمایه در شرایط ریسک با الگوریتم فراابتکاری ترکیبی ژنتیک (GA) و بهینه سازی شیر (LOA)
- A Hybrid Algorithm Based on Chaos Optimization and Steepest Descent Algorithm