Naturally Constrained Online Expectation Maximization - ENSTA Paris - École nationale supérieure de techniques avancées Paris Access content directly
Conference Papers Year : 2021

Naturally Constrained Online Expectation Maximization


With the rise of big data sets, learning algorithms must be adapted to piece-wise mechanisms to tackle large-scale calculations' time and memory costs. Furthermore, for most learning embedded systems, the input data are fed sequentially and contingently: one by one, and possibly class by class. Thus, learning algorithms should not only run online but cope with time-varying, non-independent, and non-balanced training data for the system's entire life. Online Expectation-Maximization is a well-known algorithm for learning probabilistic models in real-time, due to its simplicity and convergence properties. However, these properties are only valid in the case of large, independent and identically distributed samples. In this paper, we propose to constrain the online Expectation-Maximization on the Fisher distance between the parameters. After presenting the algorithm, we make a thorough study of its use in Probabilistic Principal Components Analysis. First, we derive the update rules, and then we analyze the effect of the constraint on major problems of online and sequential learning: convergence, forgetting and interference. Furthermore, we use several algorithmic protocols: iid vs sequential data, and constraint parameters updated step-wise vs class-wise. Our results show that this constraint increases the convergence rate of online Expectation-Maximization, decreases forgetting and slightly introduces positive transfer learning.
Fichier principal
Vignette du fichier
ICPR2020_CR2_Grammarly.pdf (471.61 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03012909 , version 1 (18-11-2020)


  • HAL Id : hal-03012909 , version 1


Daniela Pamplona, Antoine Manzanera. Naturally Constrained Online Expectation Maximization. International Conference on Pattern Recognition (ICPR 2020), Jan 2021, Milan, Italy. ⟨hal-03012909⟩
71 View
218 Download


Gmail Facebook X LinkedIn More