Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Explore this paper's citation graph
Summary
This work first investigates the performance of current offline RL algorithms under comprehensive data corruption, including states, actions, rewards, and dynamics, and proposes a more robust offline RL approach named Robust IQL (RIQL), which exhibits highly robust performance when subjected to diverse data corruption scenarios.
- Type
- preprint
- Published
- 2023-10-19
- Cited by
- 31
- References
- 104
- Access
- Open access
- OpenAlex
- https://openalex.org/W4387839219
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:264305992
Keywords
Computer science, Reinforcement learning, Robustness (evolution), Machine learning, Artificial intelligence
References
- Expected Normal Order Statistics (Exact and Approximate)
- Approximately Optimal Approximate Reinforcement Learning
- Bandits With Heavy Tail
- Measures of multivariate skewness and kurtosis with applications
- Robust Regression: Asymptotics, Conjectures and Monte Carlo
- Robust Dynamic Programming
- Robustness in Markov Decision Problems with Uncertain Transition Matrices
- On robust estimation of the location parameter
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Distributional Reinforcement Learning with Quantile Regression
- Statistical Estimates and Transformed Beta-Variables.
- Fast Bellman Updates for Robust MDPs
- Exponentially Weighted Imitation Learning for Batched Historical Data
- Reinforcement Learning with Perturbed Rewards
- Adaptive Huber Regression
- Action Robust Reinforcement Learning and Applications in Continuous Control
- Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
- When to Trust Your Model: Model-Based Policy Optimization
- A General Framework for Uncertainty Estimation in Deep Learning
- Almost Optimal Algorithms for Linear Stochastic Bandits with Heavy-Tailed Payoffs
Cited by
- Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption
- Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
- Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm
- Offline Reinforcement Learning from Datasets with Structured Non-Stationarity
- Inverse-RLignment: Inverse Reinforcement Learning from Demonstrations for LLM Alignment
- AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimization
- Augmenting Offline RL with Unlabeled Data
- Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs
- Robust Reinforcement Learning Under Dimension-Wise State Information Drop
- Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions
- Offline Deep Reinforcement Learning-Based Home Energy Management Systems With Heterogeneous EV Charging Load Models
- Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning
- Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks
- Robust Policy Expansion for Offline-to-Online RL under Diverse Data Corruption
- Adversarially Robust Decision Transformer
- ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning
- Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts
- Pretraining a Shared Q-Network for Data-Efficient Offline Reinforcement Learning
- Offline RL with Smooth OOD Generalization in Convex Hull and its Neighborhood
- Video-Enhanced Offline Reinforcement Learning: A Model-Based Approach
Related papers
- Cost-Efficient Reinforcement Learning for Optimal Trade Execution on Dynamic Market Environment
- Estimation of the general population parameter in single- and two-phase sampling
- Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes
- MOORe: Model-based Offline-to-Online Reinforcement Learning
- Exploiting Action Impact Regularity and Exogenous State Variables for Offline Reinforcement Learning
- Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
- Transformer-based On-Offline Hybrid Reinforcement Learning for Locomotion Tasks
- Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning