A Q-Learning Algorithm with Continuous State Space - ENSTA Paris - École nationale supérieure de techniques avancées Paris Access content directly
Journal Articles Optimization Online Year : 2006

A Q-Learning Algorithm with Continuous State Space

Pierre Girardeau
  • Function : Author
  • PersonId : 764150
  • IdRef : 151159351
Jean-Sébastien Roy
  • Function : Author

Abstract

We study in this paper a Markov Decision Problem (MDP) with continuous state space and discrete decision variables. We propose an extension of the Q-learning algorithm introduced to solve this problem by Watkins in 1989 for completely discrete MDPs. Our algorithm relies on stochastic approximation and functional estimation, and uses kernels to locally update the Q-functions. We give a convergence proof for this algorithm under usual assumptions. Finally, we illustrate our algorithm by solving the classical moutain car task with continuous state space.
Not file

Dates and versions

hal-00977539 , version 1 (11-04-2014)

Identifiers

  • HAL Id : hal-00977539 , version 1

Cite

Kengy Barty, Pierre Girardeau, Jean-Sébastien Roy, Cyrille Strugarek. A Q-Learning Algorithm with Continuous State Space. Optimization Online, 2006. ⟨hal-00977539⟩

Collections

ENSTA UMA_ENSTA
56 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More