The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds - Département de mathématiques appliquées Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds

Résumé

A novel algorithm named Perturbed Prox-Preconditioned SPIDER (3P-SPIDER) is introduced. It is a stochastic variancereduced proximal-gradient type algorithm built on Stochastic Path Integral Differential EstimatoR (SPIDER), an algorithm known to achieve near-optimal first-order oracle inequality for nonconvex and nonsmooth optimization. Compared to the vanilla prox-SPIDER, 3P-SPIDER uses preconditioned gradient estimators. Preconditioning can either be applied "explicitly" to a gradient estimator or be introduced "implicitly" as in applications to the EM algorithm. 3P-SPIDER also assumes that the preconditioned gradients may (possibly) be not known in closed analytical form and therefore must be approximated which adds an additional degree of perturbation. Studying the convergence in expectation, we show that 3P-SPIDER achieves a near-optimal oracle inequality O(n^(1/2) /epsilon) where n is the number of observations and epsilon the target precision even when the gradient is estimated by Monte Carlo methods. We illustrate the algorithm on an application to the minimization of a penalized empirical loss.
Fichier principal
Vignette du fichier
FMtheory_HAL.pdf (164.83 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03183775 , version 1 (28-03-2021)
hal-03183775 , version 2 (24-05-2021)

Identifiants

  • HAL Id : hal-03183775 , version 2

Citer

Gersende Fort, E Moulines. The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds. SSP 2021 - IEEE Statistical Signal Processing Workshop, Jul 2021, Rio de Janeiro, Brazil. ⟨hal-03183775v2⟩
145 Consultations
110 Téléchargements

Partager

Gmail Facebook X LinkedIn More