Automatic differentiation in machine learning: a survey

Keywords

Computer science, Artificial intelligence, Relevance (law), Automatic differentiation, Machine learning

References

• Computer algebra handbookJohannes Grabmeier, Erich Kaltofen, Volker Weispfenning — 2003
• Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)Carl Edward Rasmussen, Christopher K. I. Williams — 2005
• Recurrent neural network based language modelTomáš Mikolov, Martin Karafiát, Lukáš Burget — 2010
• Probabilistic Inference Using Markov Chain Monte Carlo MethodsRadford M. Neal — 2011
• Application of PID Controller Based on BP Neural Network in Export Steam＇s Temperature Control System朱增辉, 孙慧影 — 2011
• Computer Algebra Handbook : Foundations · Applications · SystemsJohannes Grabmeier, Erich Kaltofen, Volker Weispfenning — 2003
• Inverse problems in image processing and image segmentation : some mathematical and numerical aspectsAntonin Chambolle — 2000
• Differential dynamic programmingD. Jacobson, D.Q. Mayne — 1972
• Torch7: A Matlab-like Environment for Machine LearningRonan Collobert, Koray Kavukcuoglu, Clément Farabet — 2011
• Automatic Differentiation for MatlabJohannes Willkomm — 2010
• Getting Started with ADOL-CAndrea Walther, Andreas Griewank — 2009
• Felzenszwalb-Baum-Welch: Event Detection by Changing AppearanceDaniel Paul Barrett, Jeffrey Mark Siskind — 2013
• Learning representations by back-propagating errorsDavid E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams — 1986
• Using Polyvariant Union-Free Flow Analysis to Compile aHigher-Order Functional-Programming Language with aFirst-Class Derivative Operator to Efficient Fortran-like CodeJeffrey Mark Siskind, Barak A. Pearlmutter — 2008
• Perturbation confusion in forward automatic differentiation of higher-order functionsOleksandr Manzyuk, Barak A. Pearlmutter, Alexey Radul — 2012
• Riemann manifold Langevin and Hamiltonian Monte Carlo methodsMark Girolami, Ben Calderhead — 2011
• Distributed Asynchronous Online Learning for Natural Language ProcessingKevin Gimpel, Dipanjan Das, Noah A. Smith — 2010
• Automatic Differentiation and Interval Arithmetic for Estimation of Disequilibrium ModelsMax E. Jerrell — 1997
• Partial evaluation and automatic program generationNeil D. Jones, Carsten K. Gomard, Peter Sestoft — 1993
• The Glasgow Haskell Compiler: a technical overviewSL Peyton Jones, Kevin Hammond, WD Partain — 1993

Cited by

• A Primer on Neural Network Models for Natural Language ProcessingYoav Goldberg — 2015
• Natural Language Understanding with Distributed RepresentationKyunghyun Cho — 2015
• Distilling Reverse-Mode Automatic Differentiation (DrMAD) for Optimizing Hyperparameters of Deep Neural NetworksJie Fu, Hongyin Luo, Jiashi Feng — 2016
• A Hitchhiker’s Guide to Automatic DifferentiationPhilipp H. W. Hoffmann — 2014
• DiffSharp: Automatic Differentiation LibraryAtılım Güneş Baydin, Barak A. Pearlmutter, Jeffrey Mark Siskind — 2015
• Differentiation of the Cholesky decompositionIain Murray — 2016
• Pymanopt: A Python Toolbox for Manifold Optimization using Automatic DifferentiationJames T. Townsend, Niklas Koep, Sebastian Weichwald — 2016
• A Novel Transfer Function for Continuous Interpolation between Summation and Multiplication in Neural NetworksWiebke Koepp — 2016
• Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic DifferentiationJames T. Townsend, Niklas Koep, Sebastian Weichwald — 2016
• Returnn: The RWTH extensible training framework for universal recurrent neural networksPatrick Doetsch, Albert Zeyer, Paul Voigtlaender — 2016
• Higher-Order Factorization MachinesMathieu Blondel, Akinori Fujino, Naonori Ueda — 2016
• WekaPyScript: Classification, Regression, and Filter Schemes for WEKA Implemented in PythonChristopher Beckham, Mark Hall, Eibe Frank — 2016
• Generic Inference in Latent Gaussian Process ModelsEdwin V. Bonilla, Karl Krauth, Amir Dezfouli — 2016
• Machine Learning Refined: Foundations, Algorithms, and ApplicationsJeremy Watt, Reza Borhani, Aggelos K. Katsaggelos — 2016
• Automatic Multivector Differentiation and OptimizationLars Tingelstad, Olav Egeland — 2016
• The Generalized Reparameterization GradientFrancisco J. R. Ruiz, Michalis K. Titsias, David M. Blei — 2016
• Operational calculus on programming spaces and generalized tensor networksŽiga Sajovic, Martin Vuk — 2016
• Getting Started with Neural Models for Semantic Matching in Web SearchKezban Dilek Onal, İsmail Sengör Altıngövde, Pınar Karagöz — 2016
• Around power law for PageRank components in Buckley-Osthus model of web graphAlexander Gasnikov, Zhukovskii, Maxim, Sergey Kim — 2017
• Operational Calculus for Differentiable ProgrammingŽiga Sajovic — 2016

Related papers

• A Brief Introduction to Automatic Differentiation for Machine LearningDavan Harrison — 2021
• An Introduction to Automatic Differentiation forMachine LearningDavan Harrison — 2021
• Machine Learning for SystemsHeiner Litz, Milad Hashemi — 2020
• Recent advance in machine learning for partial differential equationKa Chun Cheung, Simon See — 2021
• Overcoming the Static Learning Bottleneck - the need for adaptive neural learningCraig M. Vineyard, Stephen Verzi — 2016
• Generic Feature Learning in Computer VisionD Nithin, P. Bagavathi Sivakumar — 2015
• Beyond expert systems: Learning programs in large-physics control systemsScott H. Clearwater, E. G. Stern — 1990