This Note presents an approximation scheme for second-order Hamilton-Jacobi-Bellman equations arising in stochastic optimal control. The scheme is based on a Markov chain approximation method. It is easy to implement in any dimension. The consistency of the scheme is proved, which guarantees its convergence. To cite this article: R. Munos, H. Zidani, C. R. Acad. Sci. Paris, Ser. I 340 (2005).