{"title":"A cognitive basis for theories of intrinsic motivation","authors":"Nisheeth Srivastava, Komal Kapoor, P. Schrater","doi":"10.1109/DEVLRN.2011.6037327","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037327","url":null,"abstract":"Since intelligent agents make choices based on both external rewards and intrinsic motivations, the structure of a realistic decision theory should also present as an indirect model of intrinsic motivation. We have recently proposed a model of sequential choice-making that is grounded in well-articulated cognitive principles. In this paper, we show how our model of choice selection predicts behavior that matches the predictions of state-of-the-art intrinsic motivation models, providing both a clear causal mechanism for explaining its effects and testable predictions for situations where its predictions differ from those of existing models. Our results provide a unified cognitively grounded explanation for phenomena that are currently explained using different theories of motivation, creativity and attention.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126480090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
B. Grzyb, J. Boedecker, M. Asada, A. P. Pobil, Linda B. Smith
{"title":"Trying anyways: How ignoring the errors may help in learning new skills","authors":"B. Grzyb, J. Boedecker, M. Asada, A. P. Pobil, Linda B. Smith","doi":"10.1109/DEVLRN.2011.6037333","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037333","url":null,"abstract":"Traditional view stresses the role of errors in the learning process. The result obtained from our experiment with older infants suggested that omitting the errors during learning can also be beneficial. We propose that a temporal decrease in learning from negative feedback could be an efficient mechanism behind infant learning new skills. Herein, we claim that disregarding the errors is tightly connected to the sense of control, and results from extremely high level of self-efficacy (overconfidence). Our preliminary results with a robot simulator serve as a proof-of-concept for our approach, and suggest a possible new route for constraints balancing exploration and exploitation in intrinsically motivated reinforcement learning.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131962666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Modelling early infant walking: Testing a generic CPG architecture on the NAO humanoid","authors":"G. Lee, Robert J. Lowe, T. Ziemke","doi":"10.1109/DEVLRN.2011.6037318","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037318","url":null,"abstract":"In this article, a simple CPG network is shown to model early infant walking, in particular the onset of independent walking. The difference between early infant walking and early adult walking is addressed with respect to the underlying neurophysiology and evaluated according to gait attributes. Based on this, we successfully model the early infant walking gait on the NAO robot and compare its motion dynamics and performance to those of infants. Our model is able to capture the core properties of early infant walking. We identify differences in the morphologies between the robot and infant and the effect of this on their respective performance. In conclusion, early infant walking can be seen to develop as a function of the CPG network and morphological characteristics.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129916089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A neural-dynamic architecture for behavioral organization of an embodied agent","authors":"Yulia Sandamirskaya, Mathis Richter, G. Schöner","doi":"10.1109/DEVLRN.2011.6037353","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037353","url":null,"abstract":"How agents generate meaningful sequences of actions in natural environments is one of the most challenging problems in studies of natural cognition and in the design of artificial cognitive systems. Each action in a sequence must contribute to the behavioral objective, while at the same time satisfying constraints that arise from the environment, the agent's embodiment, and the agent's behavioral history. In this paper, we introduce a neural-dynamic architecture that enables selection of an appropriate action for a given task in a particular environment and is open to learning. We use the same framework of neural dynamics for all processes from perception, to representation and motor planning as well as behavioral organization. This facilitates integration and flexibility. The neural dynamic representations of particular behaviors emerge on the fly from the interplay between task and environment inputs as well as behavioral history. All behavioral states are attractors of the neural dynamics, whose instabilities lead to behavioral switches. As a result, behavioral organization is robust in the face of noisy and unreliable sensory information.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132754660","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christopher Larcombe, Anthony F. Morse, A. Cangelosi
{"title":"Learning to react to abstractions: Accumulating adaptations in a humanoid embodiment","authors":"Christopher Larcombe, Anthony F. Morse, A. Cangelosi","doi":"10.1109/DEVLRN.2011.6037350","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037350","url":null,"abstract":"Human beings and several other living organisms are capable of acquiring a diverse repertoire of adaptive behaviours or skills, through interaction with an appropriate environment. Based on observations of human embodiment and existing cybernetic theory, an operational description of this form of ‘scalable’ adaptive behaviour is derivied. An articulated mechanism using the principles identified is implemented and used to control the humanoid robot iCub. The experimental physical embodiment is tested with a number of environments. Preliminary results demonstrate a limited form of emergent behavioural growth and corresponding ‘task’-non-specificity: the iCub is able to cumulatively learn multiple ‘tasks’, by chaining together sequences of primative ‘reactions’, to ‘abstractions’.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124000160","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Emergence of mirror neuron system: Immature vision leads to self-other correspondence","authors":"Y. Nagai, Yuji Kawai, M. Asada","doi":"10.1109/DEVLRN.2011.6037335","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037335","url":null,"abstract":"The question of how the mirror neuron system (MNS) develops has attracted increased attention of researchers. Among various hypotheses, a widely accepted model is associative sequence learning, which acquires the MNS as a by-product of sensorimotor learning. The model, however, cannot discriminate self from others since it adopts too much simplified sensory representations. We propose a computational model for early development of the MNS, which is originated in immature vision. The model gradually increases the spatiotemporal resolution of a robot's vision while the robot learns sensorimotor mapping through primal interactions with others. In the early stage of development, the robot interprets all observed actions as equivalent due to a lower resolution, and thus associates the non-differentiated observation with motor commands. As vision develops, the robot starts discriminating actions generated by self from those by others. The initially acquired association is, however, maintained through development, which results in two types of associations: one is between motor commands and self-observation and the other between motor commands and other-observation (i.e., what the MNS does). Our experiments demonstrate that the model achieves early development of the MNS, which enables a robot to imitate others' actions.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130731334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning geometry from sensorimotor experience","authors":"J. Stober, R. Miikkulainen, B. Kuipers","doi":"10.1109/DEVLRN.2011.6037381","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037381","url":null,"abstract":"A baby experiencing the world for the first time faces a considerable challenging sorting through what William James called the “blooming, buzzing confusion” of the senses [1]. With the increasing capacity of modern sensors and the complexity of modern robot bodies, a robot in an unknown or unfamiliar body faces a similar and equally daunting challenge.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124766204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The two-dimensional organization of behavior","authors":"Mark B. Ring, T. Schaul, J. Schmidhuber","doi":"10.1109/DEVLRN.2011.6037326","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037326","url":null,"abstract":"This paper addresses the problem of continual learning [1] in a new way, combining multi-modular reinforcement learning with inspiration from the motor cortex to produce a unique perspective on hierarchical behavior. Most reinforcement-learning agents represent policies monolithically using a single table or function approximator. In those cases where the policies are split among a few different modules, these modules are related to each other only in that they work together to produce the agent's overall policy. In contrast, the brain appears to organize motor behavior in a two-dimensional map, where nearby locations represent similar behaviors. This representation allows the brain to build hierarchies of motor behavior that correspond not to hierarchies of subroutines but to regions of the map such that larger regions correspond to more general behaviors. Inspired by the benefits of the brain's representation, the system presented here is a first step and the first attempt toward the two-dimensional organization of learned policies according to behavioral similarity. We demonstrate a fully autonomous multi-modular system designed for the constant accumulation of ever more sophisticated skills (the continual-learning problem). The system can split up a complex task among a large number of simple modules such that nearby modules correspond to similar policies. The eventual goal is to develop and use the resulting organization hierarchically, accessing behaviors by their location and extent in the map.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124947643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Judith Gaspers, P. Cimiano, Sascha S. Griffiths, B. Wrede
{"title":"An unsupervised algorithm for the induction of constructions","authors":"Judith Gaspers, P. Cimiano, Sascha S. Griffiths, B. Wrede","doi":"10.1109/DEVLRN.2011.6037371","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037371","url":null,"abstract":"We present an approach to the unsupervised induction of constructions for a specific domain. The main features of our approach are that i) it does not require any supervision in the form of explicit tutoring, ii) it learns pairings between form and meaning and iii) it induces complex syntactic constructions (and their arguments) together with a mapping to a semantic representation beyond mere word-concept associations. A comparison to approaches from the area of learning grammars for semantic parsing shows that the results of our approach are indeed competitive.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123411612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Simultaneous acquisition of task and feedback models","authors":"M. Lopes, Thomas Cederbourg, Pierre-Yves Oudeyer","doi":"10.1109/DEVLRN.2011.6037359","DOIUrl":"https://doi.org/10.1109/DEVLRN.2011.6037359","url":null,"abstract":"We present a system to learn task representations from ambiguous feedback. We consider an inverse reinforcement learner that receives feedback from a teacher with an unknown and noisy protocol. The system needs to estimate simultaneously what the task is (i.e. how to find a compact representation to the task goal), and how the teacher is providing the feedback. We further explore the problem of ambiguous protocols by considering that the words used by the teacher have an unknown relation with the action and meaning expected by the robot. This allows the system to start with a set of known signs and learn the meaning of new ones. We present computational results that show that it is possible to learn the task under a noisy and ambiguous feedback. Using an active learning approach, the system is able to reduce the length of the training period.","PeriodicalId":256921,"journal":{"name":"2011 IEEE International Conference on Development and Learning (ICDL)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121548898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}