QLSD: Quantised Langevin Stochastic Dynamics for Bayesian Federated Learning - Département de mathématiques appliquées Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

QLSD: Quantised Langevin Stochastic Dynamics for Bayesian Federated Learning

Résumé

The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients. FL raises many constraints which include privacy and data ownership, communication overhead, statistical heterogeneity, and partial client participation. In this paper, we address these problems in the framework of the Bayesian paradigm. To this end, we propose a novel federated Markov Chain Monte Carlo algorithm, referred to as Quantised Langevin Stochastic Dynamics which may be seen as an extension to the FL setting of Stochastic Gradient Langevin Dynamics, which handles the communication bottleneck using gradient compression. To improve performance, we then introduce variance reduction techniques, which lead to two improved versions coined QLSD and QLSD ++. We give both non-asymptotic and asymptotic convergence guarantees for the proposed algorithms. We illustrate their performances using various Bayesian Federated Learning benchmarks.
Fichier principal
Vignette du fichier
1106.pdf (1.33 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03589952 , version 1 (26-02-2022)

Identifiants

  • HAL Id : hal-03589952 , version 1

Citer

Maxime Vono, Vincent Plassier, Alain Durmus, Aymeric Dieuleveut, Eric Moulines. QLSD: Quantised Langevin Stochastic Dynamics for Bayesian Federated Learning. International Conference on Artificial Intelligence and Statistics, 2022, Online, France. ⟨hal-03589952⟩
151 Consultations
81 Téléchargements

Partager

Gmail Facebook X LinkedIn More