PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning - Département de mathématiques appliquées Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

Résumé

Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning. In this work, we propose PODNet, a model inspired by representation learning. By carefully balancing the compromise between remembering the old classes and learning new ones, PODNet fights catastrophic forgetting, even over very long runs of small incremental tasks --a setting so far unexplored by current works. PODNet innovates on existing art with an efficient spatial-based distillation-loss applied throughout the model and a representation comprising multiple proxy vectors for each class. We validate those innovations thoroughly, comparing PODNet with three state-of-the-art models on three datasets: CIFAR100, ImageNet100, and ImageNet1000. Our results showcase a significant advantage of PODNet over existing art, with accuracy gains of 12.10, 6.51, and 2.85 percentage points, respectively. Code is available at https://github.com/arthurdouillard/incremental_learning.pytorch

Dates et versions

hal-03503824 , version 1 (28-12-2021)

Identifiants

Citer

Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, Eduardo Valle. PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning. ECCV 2020 - 16th European Conference on Computer Vision, Aug 2020, Glasgow, United Kingdom. pp.86-102, ⟨10.1007/978-3-030-58565-5_6⟩. ⟨hal-03503824⟩
59 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More