File(s) under permanent embargo

Multi-modal integration of dynamic audiovisual patterns for an interactive reinforcement learning scenario

conference contribution
posted on 01.01.2016, 00:00 authored by F Cruz Naranjo, G I Parisi, J Twiefel, S Wermter
Robots in domestic environments are receiving more attention, especially in scenarios where they should interact with parent-like trainers for dynamically acquiring and refining knowledge. A prominent paradigm for dynamically learning new tasks has been reinforcement learning. However, due to excessive time needed for the learning process, a promising extension has been made by incorporating an external parent-like trainer into the learning cycle in order to scaffold and speed up the apprenticeship using advice about what actions should be performed for achieving a goal. In interactive reinforcement learning, different uni-modal control interfaces have been proposed that are often quite limited and do not take into account multiple sensor modalities. In this paper, we propose the integration of audiovisual patterns to provide advice to the agent using multi-modal information. In our approach, advice can be given using either speech, gestures, or a combination of both. We introduce a neural network-based approach to integrate multi-modal information from uni-modal modules based on their confidence. Results show that multimodal integration leads to a better performance of interactive reinforcement learning with the robot being able to learn faster with greater rewards compared to uni-modal scenarios.

History

Event

IEEE Robotics and Automation Society. Conference (2016 : Daejeon, South Korea)

Series

IEEE Robotics and Automation Society Conference

Pagination

759 - 766

Publisher

Institute of Electrical and Electronics Engineers

Location

Daejeon, South Korea

Place of publication

Piscataway, N.J.

Start date

09/10/2016

End date

14/10/2016

ISSN

2153-0858

eISSN

2153-0866

ISBN-13

9781509037629

Language

eng

Publication classification

E1.1 Full written paper - refereed

Copyright notice

2016, IEEE

Editor/Contributor(s)

[Unknown]

Title of proceedings

IROS 2016 : Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems