A Temporal-Difference Approach to Policy Gradient Estimation
Explore this paper's citation graph
Summary
This paper proposes a new approach of reconstructing the policy gradient from the start state without requiring a particular sampling strategy, and develops the first estimator that sidesteps the distribution shift issue in a model-free way.
- Type
- preprint
- Published
- 2022-02-04
- Cited by
- 4
- References
- 38
- Access
- Open access
- OpenAlex
- https://openalex.org/W4221141562
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:246634572
Keywords
Estimator, Realizability, Variance (accounting), Convergence (economics), Applied mathematics
References
- A convergent O ( n ) algorithm for off-policy temporal-difference learning with linear function approximation
- A Survey on Policy Search for Robotics
- Fast gradient-descent methods for temporal-difference learning with linear function approximation
- Learning to predict by the methods of temporal differences
- Least-Squares Policy Iteration
- Human-level control through deep reinforcement learning
- The Fixed Points of Off-Policy TD
- Bias in Natural Actor-Critic Algorithms
- Policy Gradient Methods for Reinforcement Learning with Function Approximation
- Linear Off-Policy Actor-Critic
- Deterministic Policy Gradient Algorithms
- Non-delusional Q-learning and value-iteration
- Off-Policy Policy Gradient with Stationary Distribution Correction
- Is the Policy Gradient a Gradient?
- Learning Continuous Control Policies by Stochastic Value Gradients
- AlgaeDICE: Policy Gradient from Arbitrary Experience
- A Nonparametric Offpolicy Policy Gradient
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Gradient Temporal-Difference Learning with Regularized Corrections
- Batch Reinforcement Learning With a Nonparametric Off-Policy Policy Gradient
Cited by
Related papers
- Time-Bridge Estimators of Integrated Variance
- Jackknifed variance estimators for simulation output analysis
- 3111 Structural Reliability Estimation Based on Repeated Importance Sampling Simulation : Construction of Optimal Importance Sampling Density Based on Conditional Expectation
- A WEIGHT-BOUNDED IMPORTANCE SAMPLING METHOD FOR VARIANCE REDUCTION
- Sampling Policy that Guarantees Reliability of Optimal Policy in Reinforcement Learning
- Directional importance sampling for ill-proportioned spaces
- State Relevance for Off-Policy Evaluation
- J044043 A Method of Structural Reliability Analysis Based on a Directional Importance Sampling Simulation : A Determination Method of Pseudo Directional Importance Sampling Probability Densities Weighted in the Direction of the Design Points