Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning - Département de mathématiques appliquées Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Signal Processing Année : 2023

Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning

Résumé

Stochastic approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with uncertainties. An exemplar special case of SA pertains to the popular stochastic (sub)gradient algorithm which is the working horse behind many important applications. A lesser-known fact is that the SA scheme also extends to non-stochastic-gradient algorithms such as compressed stochastic gradient, stochastic expectation-maximization, and a number of reinforcement learning algorithms. The aim of this article is to overview and introduce the non-stochastic-gradient perspectives of SA to the signal processing and machine learning audiences through presenting a design guideline of SA algorithms backed by theories. Our central theme is to propose a general framework that unifies existing theories of SA, including its non-asymptotic and asymptotic convergence results, and demonstrate their applications on popular non-stochastic-gradient algorithms. We build our analysis framework based on classes of Lyapunov functions that satisfy a variety of mild conditions. We draw connections between non-stochastic-gradient algorithms and scenarios when the Lyapunov function is smooth, convex, or strongly convex. Using the said framework, we illustrate the convergence properties of the non-stochastic-gradient algorithms using concrete examples. Extensions to the emerging variance reduction techniques for improved sample complexity will also be discussed.
Fichier principal
Vignette du fichier
main-v3-commited_HAL.pdf (901.57 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03979922 , version 1 (13-02-2023)

Identifiants

Citer

Aymeric Dieuleveut, Gersende Fort, Eric Moulines, Hoi-To Wai. Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning. IEEE Transactions on Signal Processing, 2023, 71, pp.3117-3148. ⟨10.1109/TSP.2023.3301121⟩. ⟨hal-03979922⟩
132 Consultations
168 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More