An MCMC Approach to Solving Hybrid Factored MDPs

Keywords

Mathematical optimization, Markov decision process, Computer science, Bottleneck, Markov chain Monte Carlo

References

• Planning under uncertainty in complex structured environmentsDaphne Koller, Carlos Guestrin — 2003
• Heuristic Refinements of Approximate Linear Programming for Factored Continuous-State Markov Decision ProcessesBranislav Kveton, Miloš Hauskrecht — 2004
• Exploiting Structure in Policy ConstructionCraig Boutilier, Richard Dearden, Moisés Goldszmidt — 1995
• The Linear Programming Approach to Approximate Dynamic ProgrammingDaniela Pucci de Farias, Benjamin Van Roy — 2003
• Introduction to linear optimizationDimitris Bertsimas, John N. Tsitsiklis — 1997
• Auxiliary Variable Methods for Markov Chain Monte Carlo with ApplicationsDavid Higdon — 1998
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesStuart Geman, Donald Geman — 1984
• Polynomial approximation—a new computational technique in dynamic programming: Allocation processesRichard Bellman, Robert E. Kalaba, Bella Kotkin — 1963
• A model for reasoning about persistence and causationThomas Dean, Keiji Kanazawa — 1989
• Optimization by Simulated AnnealingScott Kirkpatrick, C. D. Gelatt, M.P. Vecchi — 1983
• Generalized polynomial approximations in Markovian decision processesPaul J. Schweitzer, Abraham Seidmann — 1985
• Equation of State Calculations by Fast Computing MachinesN. Metropolis, Arianna W. Rosenbluth, M. N. Rosenbluth — 1953
• Hybrid Monte CarloA.D. Kennedy — 1988
• Rao-Blackwellisation of sampling schemesGeorge Casella — 1996
• Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingKasra Hazeghi, Martin L. Puterman — 1994
• Max-norm Projections for Factored MDPsCarlos Guestrin, Daphne Koller, Ronald Parr — 2001
• An Introduction to MCMC for Machine LearningChristophe Andrieu, Nando de Freitas, Arnaud Doucet — 2003
• Monte Carlo Sampling Methods Using Markov Chains and Their ApplicationsW. Keith Hastings — 1970
• Linear Program Approximations for Factored Continuous-State Markov Decision ProcessesMiloš Hauskrecht, Branislav Kveton — 2003
• Direct value-approximation for factored MDPsDale Schuurmans, Relu Patrascu — 2001

Cited by

• Approximate Linear Programming for Solving Hybrid Factored MDPsMiloš Hauskrecht — 2006
• Learning Basis Functions in Hybrid DomainsBranislav Kveton, Miloš Hauskrecht — 2006
• Probabilistic inference for solving (PO) MDPsMarc Toussaint, Stefan Harmeling, Amos Storkey — 2006
• Solving Factored MDPs with Exponential-Family Transition ModelsBranislav Kveton, Miloš Hauskrecht — 2006
• Efficient approximate linear programming for factored MDPsFeng Chen, Qiang Cheng, Jianwu Dong — 2015
• Planning in hybrid structured stochastic domainsMiloš Hauskrecht, Bransilav Kveton — 2006
• Integrated resource allocation and planning in stochastic multiagent environmentsEdmund H. Durfee, Dmitri Dolgov — 2006
• Partitioned Linear Programming Approximations for MDPsBranislav Kveton, Miloš Hauskrecht — 2008
• Clinical time series prediction: towards a hierarchical dynamical system frameworkZitao Liu, Miloš Hauskrecht — 2014
• Probabilistic inference for solving discrete and continuous state Markov Decision ProcessesMarc Toussaint, Amos Storkey — 2006
• Solving Factored MDPs with Hybrid State and Action VariablesBransilav Kveton, Miloš Hauskrecht, Carlos Guestrin — 2006
• Continuous Time Bayesian Networks for Reasoning and Decision Making in FinanceSara Villa — 2015
• Bayesian inference for motion control and planningMarc Toussaint — 2007
• Planning with Markov Decision Processes: An AI PerspectiveMausam Mausam, Andrey Kolobov — 2012
• Intelligent Planning for Autonomous Underwater Vehicles RSMG Report 3 - Thesis ProposalZ. Saigol — 2007
• On the Smoothness of Linear Value Function ApproximationsB. Kveton, M. Hauskrecht — 2006
• Overview of Linear Program Approximations for Factored Continuous and Hybrid-State Markov Decision ProcessesM. Hauskrecht — 2013
• Expectation-maximization Methods for Solving (po)mdps and Optimal Control ProblemsMarc Toussaint, A. Storkey, S. Harmeling — 2010
• Bayesian Time Series Models: Expectation maximisation methods for solving (PO)MDPs and optimal control problemsMarc Toussaint, A. Storkey, S. Harmeling — 2011

Related papers

• Efficient Solution Algorithms for Factored MDPsCarlos Guestrin, Ronald Parr, Shobha Venkataraman — 2003
• Exploiting Structure in Policy ConstructionCraig Boutilier, Richard Dearden, Moisés Goldszmidt — 1995
• Computing Factored Value Functions for Policies in Structured MDPsDaphne Koller, Ronald Parr — 1999
• Generalized polynomial approximations in Markovian decision processesPaul J. Schweitzer, Abraham Seidmann — 1985
• A model for reasoning about persistence and causationThomas Dean, Keiji Kanazawa — 1989
• Dynamic ProgrammingRichard Bellman — 1957
• Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingKasra Hazeghi, Martin L. Puterman — 1994
• Greedy linear value-approximation for factored Markov decision processesRelu Patrascu, Pascal Poupart, Dale Schuurmans — 2002
• The Linear Programming Approach to Approximate Dynamic ProgrammingDaniela Pucci de Farias, Benjamin Van Roy — 2003
• Solving Factored MDPs with Continuous and Discrete VariablesCarlos Guestrin, Miloš Hauskrecht, Branislav Kveton — 2004