Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming

Keywords

Stochastic programming, Stochastic optimization, Mathematical optimization, Convergence (economics), Class (philosophy)

References

• Problem Complexity and Method Efficiency in OptimizationJ. Darzentas — 1983
• Large Deviations of Vector-valued Martingales in 2-Smooth Normed SpacesAnatoli Juditsky, Arkadii S. Nemirovski — 2008
• Gradient Estimation Via Perturbation AnalysisP. Glasserman, Y. Ho — 1990
• Boosting Algorithms as Gradient Descent in Function SpaceLlew Mason, Jonathan Baxter, Peter Bartlett — 2007
• Confidence level solutions for stochastic programmingYu. Nesterov, Jean-Philippe Vial — 2000
• The Sample Average Approximation Method for Stochastic Discrete OptimizationAnton J. Kleywegt, Alexander Shapiro, Tito Homem‐de‐Mello — 2002
• Robust Stochastic Approximation Approach to Stochastic ProgrammingArkadi Nemirovski, Anatoli Juditsky, Guanghui Lan — 2008
• A Stochastic Approximation MethodHerbert Robbins, Sutton Monro — 1951
• Smoothing and worst-case complexity for direct-search methods in nonsmooth optimizationR. Garmanjani, L. N. Vicente — 2013
• Online dictionary learning for sparse codingJulien Mairal, Francis Bach, Jean Ponce — 2009
• Validation analysis of mirror descent stochastic approximation methodGuanghui Lan, Arkadi Nemirovski, Alexander Shapiro — 2012
• An optimal method for stochastic composite optimizationGuanghui Lan — 2011
• Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function MethodCharles Leake — 1994
• Worst case complexity of direct searchL. N. Vicente — 2013
• On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex MinimizationCoralia Cartis, Nicholas I. M. Gould, Philippe L. Toint — 2012
• On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization ProblemsCoralia Cartis, Nicholas I. M. Gould, Philippe L. Toint — 2010
• Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic FrameworkSaeed Ghadimi, Guanghui Lan — 2012
• Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function MethodJason H. Goodfriend — 1995
• Learning by mirror averagingA. Juditsky, P. Rigollet, A. B. Tsybakov — 2005
• Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C.; 2003) [book review]Wolfgang Nowak — 2007

Cited by

• Pseudomonotone Stochastic Variational Inequality Problems: Analysis and Stochastic Approximation SchemesAswin Kannan, Uday V. Shanbhag — 2014
• The Complexity of Large-scale Convex Programming under a Linear Optimization OracleGuanghui Lan — 2013
• Convergence rates for pretraining and dropout: Guiding learning parameters using network structureVamsi Krishna Ithapu, Sathya N. Ravi, Vikas Singh — 2015
• A nonmonotone learning rate strategy for SGD training of deep neural networksNitish Shirish Keskar, George Saon — 2015
• Simple Complexity Analysis of Simplified Direct SearchJakub Konečný, Peter Richtárik — 2014
• Convergence of gradient based pre-training in Denoising autoencodersVamsi Krishna Ithapu, Sathya N. Ravi, Vikas Singh — 2015
• Simple Complexity Analysis of Direct SearchJakub Konečný, Peter Richtárik — 2014
• STOCHASTIC GRADIENT METHODS FOR UNCONSTRAINED OPTIMIZATIONNataša Krejić, Nataša Krklec Jerinkić — 2014
• Stochastic Block Mirror Descent Methods for Nonsmooth and Stochastic OptimizationCong D. Dang, Guanghui Lan — 2013
• Accelerated gradient methods for nonconvex nonlinear and stochastic programmingSaeed Ghadimi, Guanghui Lan — 2013
• Convergence of Trust-Region Methods Based on Probabilistic ModelsAfonso S. Bandeira, Katya Scheinberg, L. N. Vicente — 2013
• Penalty methods with stochastic approximation for stochastic nonlinear programmingXiao Wang, Shiqian Ma, Ya-xiang Yuan — 2013
• Global convergence rate analysis of unconstrained optimization methods based on probabilistic modelsCoralia Cartis, Katya Scheinberg — 2015
• Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function EvaluationsJohn C. Duchi, Michael I. Jordan, Martin J. Wainwright — 2013
• Multiple Optimality Guarantees in Statistical LearningJohn C. Duchi — 2014
• Randomized Derivative-Free Optimization of Noisy Convex FunctionsRuobing Chen, Stefan M. Wild — 2015
• On Accelerated Gradient Approximation for Least Square Regression with L1-RegularizationYongquan Zhang, Jianyong Sun — 2015
• DiFacto: Distributed Factorization MachinesMu Li, Ziqi Liu, Alexander J. Smola — 2016
• Conditional gradient type methods for composite nonlinear and stochastic optimizationSaeed Ghadimi — 2016
• Iteration Bounds for Finding the ε-Stationary Points for Structured Nonconvex OptimizationBo Jiang, Shuzhong Zhang — 2014

Related papers

• Stability Analysis for Stochastic Optimization ProblemsJianwen Luo — 2007
• Multistage Stochastic Programs via Stochastic Parametric OptimizationVlasta Kaňková — 2008
• Stochastic Optimization; Proceedings of the International Conference, Kiev, USSR, September 1984В. И. Аркин, A. Shiraev, Roger J.‐B. Wets — 1986
• Stochastic Multi-Stage OptimizationPierre Carpentier, Jean‐Philippe Chancelier, Guy Cohen — 2015
• Scenario Optimization for Multi-Stage Stochastic Programming ProblemsRonald Hochreiter — 2005
• Risk objectives in two-stage stochastic programming modelsJitka Dupačová — 2008
• Approximation Algorithms for Stochastic Combinatorial Optimization ProblemsJian Li, Yu Liu — 2016