On the momentum term in gradient descent learning algorithms

Keywords

Momentum (technical analysis), Term (time), Gradient descent, Convergence (economics), Algorithm

References

• Neurocomputing (vol. 2): directions for researchJames A. Anderson, A. Pellionisz, Edward Rosenfeld — 1990
• Introduction to MechanicsIrving Jule Levinson — 1968
• The computational brainPatricia Smith Churchland, Terrence J. Sejnowski — 1992
• Learning internal representationsJonathan Baxter — 1995
• Analysis of the accuracy and implications of simple methods for predicting the secondary structure of globular proteins.J. Garnier, David J. Osguthorpe, Barry Robson — 1978
• Optimal Brain DamageYann LeCun, John S. Denker, Sara A. Solla — 1989
• Predicting the secondary structure of globular proteins using neural network models.Ning Qian, Terrence J. Sejnowski — 1988
• Increased rates of convergence through learning rate adaptationRobert A. Jacobs — 1987
• Learning internal representations by error propagationDavid E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams — 1986
• Learning to Solve Random-Dot Stereograms of Dense and Transparent Surfaces with Recurrent BackpropagationNing Qian, T. Sejnowski — 1989
• Neurocomputing 2Stephen D. Collins — 1992

Cited by

• Scalability of using Restricted Boltzmann Machines for combinatorial optimizationMalte Probst, Franz Rothlauf, Jörn Grahl — 2014
• Variable habitat conditions drive species covariation in the human microbiotaGustavo Dias Fleck — 2017
• Hierarchical Neural Networks for Image InterpretationSven Behnke — 2003
• Numerical Solution of Bratu’s Problem Using Multilayer Perceptron Neural Network MethodManoj Kumar, Neha Yadav — 2015
• Explorations of Infinitesimal Inverse Spectral GeometryMikhail Panine — 2013
• Auto-derivation of in-shore and channel water depth via aerial images: a low-cost approach using artificial neural networksThi My Tuyen Nguyen, Peter Wheeler, Jim Peterson — 2009
• Complex-Weighted One-Unit ‘Rigid-Bodies’ Learning Rule for Independent Component AnalysisSimone Fiori — 2002
• Error assessment of laser cutting predictions by semi-supervised learningMustafa Zaidi, Imran Amin, Ahmad Hussain — 2014
• Frequency domain iterative tuning for active noiseand vibration controlJian Luo — 2008
• Iterative feedback/feedfoward tuning control in the frequency domain for AVNCJian Luo, Sándor M. Veres — 2007
• Artificial Neural Networks in Biological and Environmental AnalysisGrady Hanrahan — 2011
• Considerations on modeling for early detection of abnormalities in locally autonomous distributed systemsMartijn van Veelen — 2007
• Optimal gradient-based learning using importance weightsSepp Hochreiter, Klaus Obermayer — 2005
• Convergence of an inertial proximal method for l1-regularized least-squaresPatrick R. Johnstone, Pierre Moulin — 2015
• Locality sensitive batch feature extraction for high-dimensional dataJie Ding, Changyun Wen, Guoqi Li — 2016
• A fuzzy neural network approximator with fast terminal sliding mode and its applicationsShuanghe Yu, Xinghuo Yu, Zhihong Man — 2002
• Computational study of polymorphic structures of α- and β- chitin and chitosan in aqueous solution.Roberto Ribeiro Faria, Renan Faria Guerra, Lourival Rodrigues de Sousa Neto — 2016
• Unified control Liapunov function based design of neural networks that aim at global minimization of nonconvex functionsFernando Pazos, Amit Bhaya, Eugenius Kaszkurewicz — 2009
• Extended Hamiltonian algorithm for the solution of discrete algebraic Lyapunov equationsZhikun Luo, Huafei Sun — 2014
• Application of multi-dimensional scaling and artificial neural networks for biologically inspired opinion miningErik Cambria, Thomas Mazzocco, Amir Hussain — 2013

Related papers

• On Rectifying a Wrong Conclusion of Evaluating Double Limit by Directional LimitYue Xiao-hong — 2003
• Some Techniques on Lim Sup and Lim InfLijiang Zeng — 2017
• Regional Systems of Innovation and Regional Policy in EuropeGeorge M. Korres, Dionysios Chionis, Christos Staikouras — 2004
• Methods of Limit Proving by Using the Definition of LimitZhen Wen — 2002
• Several Ways For the Solution to the Limit of Multivariate FunctionZhu Lian-ya — 2014
• Provable Acceleration of Neural Net Training via Polyak's MomentumJun-Kun Wang, Jacob Abernethy — 2020
• Stability of steepest descent with momentum for quadratic functionsManabu Torii, Martin Hagan — 2002