The aim of this article is to study the Hamilton Jacobi Bellman (HJB) approach for state-constrained control problems with maximum cost. In particular, we are interested in the characterization of the value functions of such problems and the analysis of the associated optimal trajectories, without assuming any controllability assumption. The rigorous theoretical results lead to several trajectory reconstruction procedures for which convergence results are also investigated. An application to a five-state aircraft abort landing problem is then considered, for which several numerical simulations are performed to analyse the relevance of the theoretical approach.