{"title":"Learning to deal with objects","authors":"M. Malfaz, M. Salichs","doi":"10.1109/DEVLRN.2009.5175508","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175508","url":null,"abstract":"In this paper, a modification of the standard learning algorithm Q-learning is presented: Object Q-learning (OQ-learning). An autonomous agent should be able to decide its own goals and behaviours in order to fulfil these goals. When the agent has no previous knowledge, it must learn what to do in every state (policy of behaviour). If the agent uses Q-learning, this implies that it learns the utility value Q of each action-state pair. Typically, an autonomous agent living in a complex environment has to interact with different objects present in that world. In this case, the number of states of the agent in relation to those objects may increase as the number of objects increases, making the learning process difficult to deal with. The proposed modification appears as a solution in order to cope with this problem. The experimental results prove the usefulness of the OQ-learning in this situation, in comparison with the standard Q-learning algorithm.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"101 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127122625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Detry, E. Baseski, M. Popovic, Y. Touati, N. Krüger, Oliver Kroemer, Jan Peters, J. Piater
{"title":"Learning object-specific grasp affordance densities","authors":"R. Detry, E. Baseski, M. Popovic, Y. Touati, N. Krüger, Oliver Kroemer, Jan Peters, J. Piater","doi":"10.1109/DEVLRN.2009.5175520","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175520","url":null,"abstract":"This paper addresses the issue of learning and representing object grasp affordances, i.e. object-gripper relative configurations that lead to successful grasps. The purpose of grasp affordances is to organize and store the whole knowledge that an agent has about the grasping of an object, in order to facilitate reasoning on grasping solutions and their achievability. The affordance representation consists in a continuous probability density function defined on the 6D gripper pose space - 3D position and orientation -, within an object-relative reference frame. Grasp affordances are initially learned from various sources, e.g. from imitation or from visual cues, leading to grasp hypothesis densities. Grasp densities are attached to a learned 3D visual object model, and pose estimation of the visual model allows a robotic agent to execute samples from a grasp hypothesis density under various object poses. Grasp outcomes are used to learn grasp empirical densities, i.e. grasps that have been confirmed through experience. We show the result of learning grasp hypothesis densities from both imitation and visual cues, and present grasp empirical densities learned from physical experience by a robot.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"103 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122808234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tingfan Wu, N. Butko, Paul Ruvulo, M. Bartlett, J. Movellan
{"title":"Learning to Make Facial Expressions","authors":"Tingfan Wu, N. Butko, Paul Ruvulo, M. Bartlett, J. Movellan","doi":"10.1109/DEVLRN.2009.5175536","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175536","url":null,"abstract":"This paper explores the process of self-guided learning of realistic facial expression production by a robotic head with 31 degrees of freedom. Facial motor parameters were learned using feedback from real-time facial expression recognition from video. The experiments show that the mapping of servos to expressions was learned in under one-hour of training time. We discuss how our work may help illuminate the computational study of how infants learn to make facial expressions.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129356930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anna-Lisa Vollmer, K. Lohan, K. Fischer, Y. Nagai, K. Pitsch, J. Fritsch, K. Rohlfing, Britta Wredek
{"title":"People modify their tutoring behavior in robot-directed interaction for action learning","authors":"Anna-Lisa Vollmer, K. Lohan, K. Fischer, Y. Nagai, K. Pitsch, J. Fritsch, K. Rohlfing, Britta Wredek","doi":"10.1109/DEVLRN.2009.5175516","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175516","url":null,"abstract":"In developmental research, tutoring behavior has been identified as scaffolding infants' learning processes. It has been defined in terms of child-directed speech (Motherese), child-directed motion (Motionese), and contingency. In the field of developmental robotics, research often assumes that in human-robot interaction (HRI), robots are treated similar to infants, because their immature cognitive capabilities benefit from this behavior. However, according to our knowledge, it has barely been studied whether this is true and how exactly humans alter their behavior towards a robotic interaction partner. In this paper, we present results concerning the acceptance of a robotic agent in a social learning scenario obtained via comparison to adults and 8-11 months old infants in equal conditions. These results constitute an important empirical basis for making use of tutoring behavior in social robotics. In our study, we performed a detailed multimodal analysis of HRI in a tutoring situation using the example of a robot simulation equipped with a bottom-up saliency-based attention model [1]. Our results reveal significant differences in hand movement velocity, motion pauses, range of motion, and eye gaze suggesting that for example adults decrease their hand movement velocity in an Adult-Child Interaction (ACI), opposed to an Adult-Adult Interaction (AAI) and this decrease is even higher in the Adult-Robot Interaction (ARI). We also found important differences between ACI and ARI in how the behavior is modified over time as the interaction unfolds. These findings indicate the necessity of integrating top-down feedback structures into a bottom-up system for robots to be fully accepted as interaction partners.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129676341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jie Chen, Cheri C. Y. Chan, Rachel Pulverman, T. Tardif, M. Casasola, Xiaobei Zheng, Xiangzhi Meng
{"title":"English- and Mandarin-speaking infants' discrimination of persons, actions, and objects in a dynamic event without audio inputs","authors":"Jie Chen, Cheri C. Y. Chan, Rachel Pulverman, T. Tardif, M. Casasola, Xiaobei Zheng, Xiangzhi Meng","doi":"10.1109/DEVLRN.2009.5175539","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175539","url":null,"abstract":"English learners typically have vocabularies that are dominated by nouns, whereas naturalistic observations, parental checklists and word mapping experiments reveal that verbs, primarily action words, are acquired early and in large quantities by learners of Mandarin Chinese. However, little research has examined whether English and Mandarin learners' early comprehension and production of nouns and verbs could be attributed to attentional patterns. In this study, we use a habituation paradigm to explore English- and Mandarin-learning infants' abilities to discriminate between Persons, Actions, and Objects presented without accompanying linguistic cues. The results revealed that English- and Mandarin-exposed 6–8 and 17–19 month-old infants showed similar patterns of attention. The younger infants showed significant increases to Person and Action changes only, whereas the older infants showed increased looking times to Person, Action, and Object changes, suggesting that the differential ease of acquiring verbs across languages might be attributed to cultural processes specific to word learning, rather than differences in early attentional preferences across cultures.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"168 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132171072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shane Griffith, J. Sinapov, Matthew Miller, A. Stoytchev
{"title":"Toward interactive learning of object categories by a robot: A case study with container and non-container objects","authors":"Shane Griffith, J. Sinapov, Matthew Miller, A. Stoytchev","doi":"10.1109/DEVLRN.2009.5175537","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175537","url":null,"abstract":"This paper proposes an interactive approach to object categorization that is consistent with the principle that a robot's object representations should be grounded in its sensorimotor experience. The proposed approach allows a robot to: 1) form object categories based on the movement patterns observed during its interaction with objects, and 2) learn a perceptual model to generalize object category knowledge to novel objects. The framework was tested on a container/non-container categorization task. The robot successfully separated the two object classes after performing a sequence of interactive trials. The robot used the separation to learn a perceptual model of containers, which, which, in turn, was used to categorize novel objects as containers or non-containers.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126738968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Pulse discrete cosine transform for saliency-based visual attention","authors":"Ying-jia Yu, Bin Wang, Liming Zhang","doi":"10.1109/DEVLRN.2009.5175512","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175512","url":null,"abstract":"This paper proposes a saliency-based attention model based on pulsed cosine transform that simulates the lateral surround inhibition of neurons with similar visual features. The model can be extended to Hebbian-based neural networks. The visual saliency can be represented in binary codes, which agrees with the firing pulse of neurons in human brain. In addition, motion saliency can be directly generated by these pulse codes. Due to its good performance in eye fixation prediction and low computational complexity, our model can be used in real-time system such as robot navigation, virtual human system, and intelligent auto-focus system embedded in digital camera.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125175820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Complex text processing by the temporal context machines","authors":"J. Weng, Qi Zhang, M. Chi, X. Xue","doi":"10.1109/DEVLRN.2009.5175540","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175540","url":null,"abstract":"It is largely unknown how the brain deals with time. Hidden Markov Model (HMM) has a probability based mechanism to deal with time warping, but no effective online method exists that can deal with general active temporal abstraction. By online, we mean that the agent must respond to spatial and temporal context immediately while a sensory stream flows in. By general active temporal context, we mean active (learned) attention selects desirable temporal subsets within a dynamic length of recent history (e.g., beyond bigrams and trigrams). By temporal abstraction, we mean using abstract meaning of context, supervised at the motor end, instead of iconic forms. This paper reports four experiments of complex text processing using the framework of a general-purpose developmental spatiotemporal agent called Temporal Context Machines (TCM), demonstrating its power of forming online, active, abstract, temporal contexts. We show that it perfectly (100%) solved a hypothetic problem called New Sentence Problem — after the TCM has learned synonyms under the corresponding contexts, it successfully recognized all possible new sentences (formed from the synonyms) that it has not learned. We show the TCM dealt with the Word Sense Disambiguation Problem where words are ambiguous without context. TCMs were also applied to the Part-of-Speech Problem, where the part of speech of the words in English language is identified according to contexts. In the fourth experiment, TCMs were employed to deal with the challenging Chunking Problem, in which subsequences of words are grouped and classified according to English linguistic units.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"66 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114279632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An intrinsic reward for affordance exploration","authors":"Stephen Hart","doi":"10.1109/DEVLRN.2009.5175542","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175542","url":null,"abstract":"In this paper, we present preliminary results demonstrating how a robot can learn environmental affordances in terms of the features that predict successful control and interaction. We extend previous work in which we proposed a learning framework that allows a robot to develop a series of hierarchical, closed-loop manipulation behaviors. Here, we examine a complementary process where the robot builds probabilistic models about the conditions under which these behaviors are likely to succeed. To accomplish this, we present an intrinsic reward function that directs the robot's exploratory behavior towards gaining confidence in these models. We demonstrate how this single intrinsic motivator can lead to artifacts of behavior such as “novelty,” “habituation,” and “surprise.” We present results using the bimanual robot Dexter, and explore these results further in simulation.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114458238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
F. Stulp, Ingo Kresse, A. Maldonado, Federico Ruiz, Andreas Fedrizzi, M. Beetz
{"title":"Compact models of human reaching motions for robotic control in everyday manipulation tasks","authors":"F. Stulp, Ingo Kresse, A. Maldonado, Federico Ruiz, Andreas Fedrizzi, M. Beetz","doi":"10.1109/DEVLRN.2009.5175511","DOIUrl":"https://doi.org/10.1109/DEVLRN.2009.5175511","url":null,"abstract":"Autonomous personal robots are currently being equipped with hands and arms that have kinematic redundancy similar to those of humans. Humans exploit the redundancy in their motor system by optimizing secondary criteria. Tasks which are executed repeatedly lead to movements that are highly optimized over time, which leads to stereotypical [25] and pre-planned [15] motion patterns. This stereotypical motion can be modeled well with compact models, as has been shown for locomotion [1]. In this paper, we determine compact models for human reaching and obstacle avoidance in everyday manipulation tasks, and port these models to an articulated robot. We acquire compact models by analyzing human reaching data acquired with a magnetic motion tracker with dimensionality reduction and clustering methods. The stereotypical reaching trajectories so acquired are used to train a Dynamic Movement Primitive [12], which is executed on the robot. This enables the robot not only to follow these trajectories accurately, but also uses the compact model to predict and execute further human trajectories.","PeriodicalId":192225,"journal":{"name":"2009 IEEE 8th International Conference on Development and Learning","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129811721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}