High-Dimensional Continuous Control Using Generalized Advantage Estimation

Keywords

Estimator, Artificial neural network, Reinforcement learning, Computer science, Variance (accounting)

References

• Steps toward Artificial IntelligenceMarvin Minsky — 1995
• An Analysis of Actor/Critic Algorithms Using Eligibility Traces: Reinforcement Learning with Imperfect Value FunctionHajime Kimura, Shigenobu Kobayashi — 1998
• Dynamic Programming and Optimal Control, Two Volume SetDimitri P. Bertsekas — 1995
• Approximate Gradient Methods in Policy-Space Optimization of Markov Reward ProcessesPeter Marbach, John N. Tsitsiklis — 2003
• Neuronlike adaptive elements that can solve difficult learning control problemsAndrew G. Barto, Richard S. Sutton, Charles W. Anderson — 1983
• Reinforcement Learning in POMDP's via Direct Gradient AscentJonathan Baxter, Peter L. Bartlett — 2000
• Policy Invariance Under Reward Transformations: Theory and Application to Reward ShapingAndrew Y. Ng, Daishi Harada, Stuart Russell — 1999
• Reinforcement learning in feedback controlRoland Hafner, Martin Riedmiller — 2011
• Real-time reinforcement learning by sequential Actor-Critics and experience replayPaweł Wawrzyński — 2009
• Principles of BehaviorClark L. Hull — 1945
• Optimal gait and form for animal locomotionKevin Wampler, Zoran Popović — 2009
• Stochastic policy gradient reinforcement learning on a simple 3D bipedRuss Tedrake, Tianbao Zhang, H. Sebastian Seung — 2004
• A Natural Policy GradientSham M. Kakade — 2001
• Bias in Natural Actor-Critic AlgorithmsPhilip S. Thomas — 2014
• Policy Gradient Methods for Reinforcement Learning with Function ApproximationRichard S. Sutton, David McAllester, Satinder Singh — 1999
• Fast biped walking with a reflexive controller and real-time policy searchingTao Geng, Bernd Porr, Florentin Wörgötter — 2005
• Actor-Critic AlgorithmsVijay R. Konda, John N. Tsitsiklis — 1999
• MuJoCo: A physics engine for model-based controlEmanuel Todorov, Tom Erez, Yuval Tassa — 2012
• Natural Actor-CriticJan Peters, Stefan Schaal — 2003
• Boekbespreking van D.P. Bertsekas (ed.), Dynamic programming and optimal control - volume 2J.W. Nieuwenhuis — 1999

Cited by

• Continuous-Time Fitted Value Iteration for Robust Policiesشیرین سخدری, ساندرا مهرعلی زاده, بهاره دادرسانفر — 2021
• Multiagent Trust Region Policy OptimizationСоколова Надежда Анатольевна — 2020
• Dueling Network Architectures for Deep Reinforcement LearningZiyu Wang, Tom Schaul, Matteo Hessel — 2015
• Deep Attention Recurrent Q-NetworkIvan Sorokin, А. А. Селезнев, Mikhail Pavlov — 2015
• Memory-based control with recurrent neural networksNicolas Heess, Jonathan J. Hunt, Timothy Lillicrap — 2015
• Easy Monotonic Policy IterationJoshua Achiam — 2016
• HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed RewardsSanjay Krishnan, Animesh Garg, Richard Liaw — 2016
• Review of state-of-the-arts in artificial intelligence with application to AI safety problemVladimir Shakirov — 2016
• Concrete Problems in AI SafetyRicky Aliae — 2016
• Safe, Multi-Agent, Reinforcement Learning for Autonomous DrivingShai Shalev‐Shwartz, Shaked Shammah, Amnon Shashua — 2016
• Hybrid computing using a neural network with dynamic external memoryAlex Graves, Greg Wayne, Malcolm Reynolds — 2016
• Learning and Transfer of Modulated Locomotor ControllersNicolas Heess, Gregory Wayne, Yuval Tassa — 2016
• Deep Learning Approximation for Stochastic Control ProblemsJiequn Han, E Weinan — 2016
• Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy CriticShixiang Gu, Timothy Lillicrap, Zoubin Ghahramani — 2016
• Sample Efficient Actor-Critic with Experience ReplayZiyu Wang, Victor Bapst, Nicolas Heess — 2016
• Improving Policy Gradient by Exploring Under-appreciated RewardsOfir Nachum, Mohammad Norouzi, Dale Schuurmans — 2016
• Self-Critical Sequence Training for Image CaptioningSteven J. Rennie, Etienne Marcheret, Youssef Mroueh — 2016
• Efficient iterative policy optimizationNicolas Le Roux — 2016
• RL^2: Fast Reinforcement Learning via Slow Reinforcement LearningYan Duan, John Schulman, Xi Chen — 2016
• Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement LearningFrancois Belletti, Daniel Haziza, Gabriel Gomes — 2017

Related papers

• End-to-end training of deep visuomotor policiesSergey Levine, Chelsea Finn, Trevor Darrell — 2016
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic ActorTuomas Haarnoja, Aurick Zhou, Pieter Abbeel — 2018
• Mastering the game of Go without human knowledgeDavid Silver, Julian Schrittwieser, Karen Simonyan — 2017
• Proximal Policy Optimization AlgorithmsNatan, Avraham, Stern, Roni, Kalech, Meir — 2017
• Emergence of Locomotion Behaviours in Rich EnvironmentsNicolas Heess, Dhruva Tb, Sriram Srinivasan — 2017
• Benchmarking Deep Reinforcement Learning for Continuous ControlYan Duan, Xi Chen, Rein Houthooft — 2016
• Mastering the game of Go with deep neural networks and tree searchDavid Silver, Aja Huang, Chris J. Maddison — 2016
• Continuous control with deep reinforcement learningTimothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel — 2015
• Deterministic Policy Gradient AlgorithmsDavid Silver, Guy Lever, Nicolas Heess — 2014