File(s) under permanent embargo
Multi-modal integration of dynamic audiovisual patterns for an interactive reinforcement learning scenario
conference contribution
posted on 2016-01-01, 00:00 authored by F Cruz Naranjo, G I Parisi, J Twiefel, S WermterRobots in domestic environments are receiving more attention, especially in scenarios where they should interact with parent-like trainers for dynamically acquiring and refining knowledge. A prominent paradigm for dynamically learning new tasks has been reinforcement learning. However, due to excessive time needed for the learning process, a promising extension has been made by incorporating an external parent-like trainer into the learning cycle in order to scaffold and speed up the apprenticeship using advice about what actions should be performed for achieving a goal. In interactive reinforcement learning, different uni-modal control interfaces have been proposed that are often quite limited and do not take into account multiple sensor modalities. In this paper, we propose the integration of audiovisual patterns to provide advice to the agent using multi-modal information. In our approach, advice can be given using either speech, gestures, or a combination of both. We introduce a neural network-based approach to integrate multi-modal information from uni-modal modules based on their confidence. Results show that multimodal integration leads to a better performance of interactive reinforcement learning with the robot being able to learn faster with greater rewards compared to uni-modal scenarios.
History
Event
IEEE Robotics and Automation Society. Conference (2016 : Daejeon, South Korea)Series
IEEE Robotics and Automation Society ConferencePagination
759 - 766Publisher
Institute of Electrical and Electronics EngineersLocation
Daejeon, South KoreaPlace of publication
Piscataway, N.J.Publisher DOI
Start date
2016-10-09End date
2016-10-14ISSN
2153-0858eISSN
2153-0866ISBN-13
9781509037629Language
engPublication classification
E1.1 Full written paper - refereedCopyright notice
2016, IEEEEditor/Contributor(s)
[Unknown]Title of proceedings
IROS 2016 : Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and SystemsUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC