Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy - INRIA - Institut National de Recherche en Informatique et en Automatique Accéder directement au contenu
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2023

Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy

Résumé

We present an algorithm for minimizing an objective with hard-to-compute gradients by using a related, easier-to-access function as a proxy. Our algorithm is based on approximate proximal point iterations on the proxy combined with relatively few stochastic gradients from the objective. When the difference between the objective and the proxy is δ-smooth, our algorithm guarantees convergence at a rate matching stochastic gradient descent on a δ-smooth objective, which can lead to substantially better sample efficiency. Our algorithm has many potential applications in machine learning, and provides a principled means of leveraging synthetic data, physics simulators, mixed public and private data, and more.
Fichier principal
Vignette du fichier
arxiv_june_7_2023.pdf (523.83 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03977083 , version 1 (07-02-2023)
hal-03977083 , version 2 (14-02-2023)
hal-03977083 , version 3 (31-03-2023)
hal-03977083 , version 4 (07-06-2023)

Licence

Paternité

Identifiants

Citer

Blake Woodworth, Konstantin Mishchenko, Francis Bach. Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy. 2023. ⟨hal-03977083v4⟩
52 Consultations
53 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More