{"title":"An Enactive approach to autonomous agent and robot learning","authors":"Olivier L. Georgeon, Christian Wolf, S. L. Gay","doi":"10.1109/DEVLRN.2013.6652527","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652527","url":null,"abstract":"A novel way to model an agent interacting with an environment is introduced, called an Enactive Markov Decision Process (EMDP). An EMDP keeps perception and action embedded within sensorimotor schemes rather than dissociated. Instead of seeking a goal associated with a reward, as in reinforcement learning, an EMDP agent is driven by two forms of self-motivation: successfully enacting sequences of interactions (autotelic motivation), and preferably enacting interactions that have predefined positive values (interactional motivation). An EMDP learning algorithm is presented. Results show that the agent develops a rudimentary form of self-programming, along with active perception as it learns to master the sensorimotor contingencies afforded by its coupling with the environment.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"20 6 Suppl 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128055140","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dynamic shift in isolating referents: From social to self-generated input","authors":"Hanako Yoshida, Joseph M. Burling","doi":"10.1109/DEVLRN.2013.6652570","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652570","url":null,"abstract":"Infants as young as 6 months of age start comprehending some familiar words, yet there is little understanding of how young infants utilize information presented in their social environment in order to make sense of the world. As an initial step, we used recent technology that allowed us to narrow in on the point of view of the infant to explore how infants' visual input is dynamically synchronized with their own participation, as well as from social input in the context of parent-child word-learning play. The results reveal dynamic changes in the relation between infants' view and their level of participation.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127008991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Action understanding using an adaptive Liquid State Machine based on environmental ambiguity","authors":"Jimmy Baraglia, Y. Nagai, M. Asada","doi":"10.1109/DEVLRN.2013.6652528","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652528","url":null,"abstract":"Recently, humans-robots interaction steps aside the traditional master/slave relationship to evolve in a new paradigm of cognitive robotics. Their conception requires the comprehension of human cognitive functions and how they develop. In this paper, we present how an adaptive Liquid State Machine using environment ambiguity may lead to a better emergence of action prediction abilities in a simple robot. The simulation results indicate the efficiency of the proposed method by which a simple robot develop its own prediction capability. These results are promising towards building robots able to develop more complicated capability such as understanding others' intention and further, cooperation with other agents.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131551289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Continuous adaptive reinforcement learning with the evolution of Self Organizing Classifiers","authors":"Danilo Vasconcellos Vargas, H. Takano, J. Murata","doi":"10.1109/DEVLRN.2013.6652558","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652558","url":null,"abstract":"Learning classifier systems have been solving reinforcement learning problems for some time. However, they face difficulties under multi-step continuous problems. Adaptation may also become harder with time since the convergence of the population decreases its diversity. This article demonstrate that the novel Self Organizing Classifiers method can cope with dynamical multi-step continuous problems. Moreover, adaptation remains the same after convergence.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134191136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Grand, Ghilès Mostafaoui, Syed Khursheed Hasnain, P. Gaussier
{"title":"Combining synchrony and shape detection to sustain the robot focus of attention on a selected human partner","authors":"C. Grand, Ghilès Mostafaoui, Syed Khursheed Hasnain, P. Gaussier","doi":"10.1109/DEVLRN.2013.6652529","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652529","url":null,"abstract":"The present study deals with the problematic of attentional mechanism allowing to initiate and to maintain Human Robot Interactions (HRI) by orienting the robot's visual focus on interacting human partners. In our previous work, we took inspiration from human psychological and neurological data which suggest that synchrony is an important parameter for human-human interaction. We proposed synchrony as a way of interacting and presented a synchrony-based architecture capable of selecting the human partner and of locating the focus of attention. To deal with the problematic of initiating the HRI, we proposed, in our recent works, a neural model permitting to focus the robot visual attention on a selected partner on the basis of synchrony detection between its own dynamics and the human movements. This model maintain the interaction and the robot's focus of attention while the partner moves in synchrony. Consequently, the interaction is interrupted if the partner stops moving. For a more realist HRI, the agents have to be able to switch their roles (turn tacking), as a result, they could alternate moving and static interaction phases. In this case, we propose here to complete the previous neural model by adding a shape based attentional mechanism. After initiating the interaction on the basis of synchrony, the robot will automatically learn to recognize the selected partner and maintain its attention with the human during unsynchronized phases of interaction.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"199 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123269699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploration strategies in developmental robotics: A unified probabilistic framework","authors":"Clément Moulin-Frier, Pierre-Yves Oudeyer","doi":"10.1109/DEVLRN.2013.6652535","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652535","url":null,"abstract":"We present a probabilistic framework unifying two important families of exploration mechanisms recently shown to be efficient to learn complex non-linear redundant sensorimotor mappings. These two explorations mechanisms are: 1) goal babbling, 2) active learning driven by the maximization of empirically measured learning progress. We show how this generic framework allows to model several recent algorithmic architectures for exploration. Then, we propose a particular implementation using Gaussian Mixture Models, which at the same time provides an original empirical measure of the competence progress. Finally, we perform computer simulations on two simulated setups: the control of the end effector of a 7-DoF arm and the control of the formants produced by an articulatory synthesizer.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117306050","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Touch and emotion: Modeling of developmental differentiation of emotion lead by tactile dominance","authors":"Takato Horii, Y. Nagai, M. Asada","doi":"10.1109/DEVLRN.2013.6652538","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652538","url":null,"abstract":"Emotion is one of the important elements for humans to communicate with others. Humans are known to share basic emotions such as joy and anger although their developmental changes have been studied less. We propose a computational model for the emotional development in infancy. Our model reproduces the differentiation of emotion from pleasant/unpleasant states to six basic emotions as known in psychological studies. The key idea is twofold: the tactile dominance in infant-caregiver interaction and the inherent ability of tactile sense to discriminate pleasant/unpleasant states. Our model consists of probabilistic neural networks called Restricted Boltzmann Machines. The networks are hierarchically organized to first extract important features from tactile, auditory, and visual stimuli and then to integrate them to represent an emotional state. Pleasant/unpleasant information is directly provided to the highest level of the network to facilitate the emotional differentiation. Experimental results show that our model with the tactile dominance leads to better differentiation of emotion than others without such dominance.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125721863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Autonomous learning based on depth perception and behavior generation","authors":"Sungmoon Jeong, Yunjung Park, Minho Lee","doi":"10.1109/DEVLRN.2013.6652531","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652531","url":null,"abstract":"We propose a new neuro-robotic network that can simultaneously achieve a goal oriented behavior task and perception enhancement task for a visually-guided object manipulation based on learning by examples. The brain exploits action to develop perception qualities, and perceptual process helps to develop qualified-behavior. In order to import those action and perception inter-abilities of a brain into a humanoid robot, we consider two key inspirations: (1) Sensory Invariant Driven Action (SIDA) and (2) Object Size Invariance (OSI) characteristic. Considering robot manipulation of a target object with distance estimation as a perceptual process, we develop a new autonomous learning method based on the SIDA for behavior generation and OSI property for perceptual judgment. The proposed method is evaluated by using a humanoid robot (NAO) with stereo cameras, and the experimental results show that the proposed method is effective on autonomously improving the behavior generation performance as well as depth perception accuracy.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131140348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Adrien Jauffret, Marwen Belkaid, N. Cuperlier, P. Gaussier, P. Tarroux
{"title":"Frustration as a way toward autonomy and self-improvement in robotic navigation","authors":"Adrien Jauffret, Marwen Belkaid, N. Cuperlier, P. Gaussier, P. Tarroux","doi":"10.1109/DEVLRN.2013.6652540","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652540","url":null,"abstract":"Autonomy and self-improvement capabilities are still challenging in the field of robotics. Allowing a robot to autonomously navigate in wide and unknown environments not only requires a set of robust strategies to cope with miscellaneous situations, but also needs mechanisms of self-assessment for guiding learning and for monitoring strategies. Monitoring strategies requires feedbacks on the behavior's quality, from a given fitness system in order to take correct decisions. In this work, we focus on how an emotional controller can be used to modulate robot behaviors. Following an incremental and constructivist approach, we present a generic neural architecture, based on an online novelty detection algorithm that may be able to evaluate any sensory-motor strategies. This architecture learns contingencies between sensations and actions, giving the expected sensation from the past perception. Prediction error, coming from surprising events, provides a direct measure of the quality of the underlying sensory-motor contingencies involved. We show how a simple emotional controller based on the prediction progress allows the system to regulate its behavior to solve complex navigation tasks and to communicate its disability in deadlock situations. We propose that this model could be a key structure toward self-monitoring. We made several experiments that can account for such properties with different behaviors (road following and place cells based navigation).","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133906341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do humans need learning to read humanoid lifting actions?","authors":"A. Sciutti, Laura Patanè, F. Nori, G. Sandini","doi":"10.1109/DEVLRN.2013.6652557","DOIUrl":"https://doi.org/10.1109/DEVLRN.2013.6652557","url":null,"abstract":"Humans can infer, just from the observation of others' actions, several information on the actor's intents and on the properties of the manipulated objects. This intuitive understanding is very efficient and allows two collaborating partners to be prepared to handle the common tools, as they can estimate the weight of the object the other agent is passing to them even before the hand-over is concluded. Transferring this kind of mutual understanding to human - robot interactions would be particularly beneficial, as it would improve the fluidity of any collaborative task. The question that we address in this study is therefore under which conditions humans can estimate the weight lifted by a humanoid robot and whether the acquisition of this skill requires an extensive learning by the human subject. Moreover, we assess whether reading humanoid lifting actions implies the involvement of the observer's motor system, as it happens for weight judgment from the observation of human actions. Our results indicate that with a proper design of the humanoid lifting motions, human subjects are able to estimate the weight lifted by the humanoid robot with a similar accuracy as that exhibited during human observation. Furthermore, such ability is intuitive and does not require learning or training. Lastly, weight judgment seems to be dependent on the involvement of the observer's motor system both during human and humanoid observation. These findings suggest that the neural mechanisms at the basis of human interaction can be extended to human-humanoid interaction, allowing for intuitive and proficient collaboration between humanoid robots and untrained human partners.","PeriodicalId":106997,"journal":{"name":"2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114613063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}