Improving Reward Estimation in Goal-Conditioned Imitation Learning with Counterfactual Data and Structural Causal Models - Equipe Robot interaction, Ambient system, Machine learning, Behaviour, Optimization Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Improving Reward Estimation in Goal-Conditioned Imitation Learning with Counterfactual Data and Structural Causal Models

Résumé

Imitation learning has emerged as a pragmatic alternative to reinforcement learning for teaching agents to execute specific tasks, mitigating the complexity associated with reward engineering. However, the deployment of imitation learning in real-world scenarios is hampered by numerous challenges. Often, the scarcity and expense of demonstration data hinder the effectiveness of imitation learning algorithms. In this paper, we present a novel approach to enhance the sample efficiency of goal-conditioned imitation learning. Leveraging the principles of causality, we harness structural causal models as a formalism to generate counterfactual data. These counterfactual instances are used as additional training data, effectively improving the learning process. By incorporating causal insights, our method demonstrates its ability to improve imitation learning efficiency by capitalizing on generated counterfactual data. Through experiments on simulated robotic manipulation tasks, such as pushing, moving, and sliding objects, we showcase how our approach allows for the learning of better reward functions resulting in improved performance with a limited number of demonstrations, paving the way for a more practical and effective implementation of imitation learning in real-world scenarios.
Fichier principal
Vignette du fichier
LICARSA_2023_5_CR.pdf (801.56 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04238529 , version 1 (12-10-2023)

Identifiants

  • HAL Id : hal-04238529 , version 1

Citer

Mohamed Khalil Jabri, Panagiotis Papadakis, Ehsan Abbasnejad, Gilles Coppin, Javen Shi. Improving Reward Estimation in Goal-Conditioned Imitation Learning with Counterfactual Data and Structural Causal Models. International Conference on Informatics in Control, Automation and Robotics (ICINCO), INSTICC, Nov 2023, Rome, Italy. ⟨hal-04238529⟩
80 Consultations
52 Téléchargements

Partager

Gmail Facebook X LinkedIn More