{"title":"Deep reinforcement learning for conversational robots playing games","authors":"H. Cuayáhuitl","doi":"10.1109/HUMANOIDS.2017.8246959","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246959","url":null,"abstract":"Deep reinforcement learning for interactive multimodal robots is attractive for endowing machines with trainable skill acquisition. But this form of learning still represents several challenges. The challenge that we focus in this paper is effective policy learning. To address that, in this paper we compare the Deep Q-Networks (DQN) method against a variant that aims for stronger decisions than the original method by avoiding decisions with the lowest negative rewards. We evaluated our baseline and proposed algorithms in agents playing the game of Noughts and Crosses with two grid sizes (3×3 and 5×5). Experimental results show evidence that our proposed method can lead to more effective policies than the baseline DQN method, which can be used for training interactive social robots.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117165532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Flayols, A. Prete, Patrick M. Wensing, Alexis Mifsud, M. Benallegue, O. Stasse
{"title":"Experimental evaluation of simple estimators for humanoid robots","authors":"T. Flayols, A. Prete, Patrick M. Wensing, Alexis Mifsud, M. Benallegue, O. Stasse","doi":"10.1109/HUMANOIDS.2017.8246977","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246977","url":null,"abstract":"This paper introduces and evaluates a family of new simple estimators to reconstruct the pose and velocity of the floating base. The estimation of the floating-base state is a critical challenge to whole-body control methods that rely on full-state information in high-rate feedback. Although the kinematics of grounded limbs may be used to estimate the pose and velocity of the body, modelling errors from ground irregularity, foot slip, and structural flexibilities limit the utility of estimation from kinematics alone. These difficulties have motivated the development of sensor fusion methods to augment body-mounted IMUs with kinematic measurements. Existing methods often rely on extended Kalman filtering, which lack convergence guarantees and may present difficulties in tuning. This paper proposes two new simplifications to the floating-base state estimation problem that make use of robust off-the-shelf orientation estimators to bootstrap development. Experiments for in-place balance and walking with the HRP-2 show that the simplifications yield results on par with the accuracy reported in the literature for other methods. As further benefits, the structure of the proposed estimators prevents divergence of the estimates, simplifies tuning, and admits efficient computation. These benefits are envisioned to help accelerate the development of baseline estimators in future humanoids.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116073412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Model-predictive control in multi-contact based on stability polyhedrons","authors":"H. Audren, A. Kheddar","doi":"10.1109/HUMANOIDS.2017.8246938","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246938","url":null,"abstract":"We propose a new linear model-predictive control scheme in multi-contact based on the center of mass reduced model. In order to linearize the dynamics of the CoM, we exploit the notion of stability polyhedrons associated to given contact stances, inside which the existence of contact forces is guaranteed. To compute stability polyhedrons, we have first to specify a convex polytope inside which the center of mass's acceleration lies. We then generate a minimum jerk trajectory inside these successive stability polyhedrons that also yields contact transitions timings, and integrate it with our quadratic programming whole-body controller as part of the multi-objective tasks.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131678328","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Adrien Pajon, Stéphane Caron, Giovanni De Magistris, S. Miossec, A. Kheddar
{"title":"Walking on gravel with soft soles using linear inverted pendulum tracking and reaction force distribution","authors":"Adrien Pajon, Stéphane Caron, Giovanni De Magistris, S. Miossec, A. Kheddar","doi":"10.1109/HUMANOIDS.2017.8246909","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246909","url":null,"abstract":"Soft soles absorb impacts and cast ground unevenness during locomotion on rough terrains. However, they introduce passive degrees of freedom (deformations under the feet) that complexify the tasks of state estimation and overall robot stabilization. We address this problem by developing a control loop that stabilizes humanoid robots when walking with soft soles on flat and uneven terrain. Our closed-loop controller minimizes the errors on the center of mass (COM) and the zero moment point (ZMP) with an admittance control of the feet based on a simple deformation estimator. We demonstrate its effectiveness in real experiments on the HRP-4 humanoid.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128005363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Real-time stabilization of a falling humanoid robot using hand contact: An optimal control approach","authors":"Shihao Wang, Kris K. Hauser","doi":"10.1109/HUMANOIDS.2017.8246912","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246912","url":null,"abstract":"This paper introduces a real-time strategy to stabilize a falling humanoid robot by making hand contact with rails or walls in the environment. It uses an optimal control strategy with a simplified three-link model and finds an optimal hand contact using a direct shooting method. The objective function is designed to maintain stability while minimizing the probability of the contact slipping and minimizing impact that may damage the robot's arm. To achieve real-time performance, the method uses a precomputed database of necessary sticking friction coefficients at the contact points for all possible postimpact states. Validation is performed on a number of simulated falls in several rails and walls.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117281622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Soenke Michalik, S. Michalik, J. Naghmouchi, Mladen Berekovic
{"title":"Real-time smart stereo camera based on FPGA-SoC","authors":"Soenke Michalik, S. Michalik, J. Naghmouchi, Mladen Berekovic","doi":"10.1109/HUMANOIDS.2017.8246891","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246891","url":null,"abstract":"Stereo image processing is one of the most demanding tasks in the field of 3D computer vision and robot vision requiring high-performance computing capabilities within embedded systems. Real-time constraints for autonomous vehicles such as humanoid robots, lead to hardware acceleration approaches for high resolution stereo imaging in human-like vision systems, where commonly FPGA device are employed to handle very high sensor data rates. This work presents a realtime smart stereo camera system implementation resembling the full stereo processing pipeline in a single FPGA device. We introduce the novel memory optimized stereo processing algorithm ”Sparse Retina Census Correlation” (SRCC) that embodies a combination of two well established window based stereo matching approaches. We have leveraged a Sum of Absolute Difference (SAD) of Sobel-filtered images and a Sum of Hamming Distance (SHD) using a modified Retina based Census Transform for increased robustness to lighting variations and for high accuracy. A color rectification module has been implemented to cope with the high frame rate of the stereo pipelining calculating image transformations and rectified pixel coordinates in real-time using parameters for camera intrinsic, image rotation, image distortion and image projection. In addition multiple post-processing algorithms like texture filtering, uniqueness filtering, speckle removal and disparity to depth conversion have been implemented to further enhance the output results. The presented smart camera solution has demonstrated real-time stereo processing of 1280×720 pixel depth images with 256 disparities on a Zynq XC7Z030 FPGA device at 60fps. Due to the universal USB3.0 UVC interface and the onboard depth calculation it is a replacement for RGBD 3D-Sensors with improved image quality and outdoor performance. The camera can easily be used in conjunction with ROS-enabled robots and in automotive or industrial applications.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126154048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Emmanouil Spyrakos-Papastavridis, P. Childs, N. Tsagarakis
{"title":"Variable impedance walking using Time-Varying Lyapunov Stability Margins","authors":"Emmanouil Spyrakos-Papastavridis, P. Childs, N. Tsagarakis","doi":"10.1109/HUMANOIDS.2017.8246892","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246892","url":null,"abstract":"This paper introduces novel methods for a humanoid robot's online balance monitoring, as well as for the tuning of its impedance parameters, based on Time-Varying Lyapunov Stability Margins (TVLSMs). It distinguishes itself from previous works by considering TVLSMs whose values evolve in real-time, in accordance with the references supplied to a trajectory tracking controller, as opposed to the previously-considered static LSMs that revolve around the use of set-point regulators. As a result, an analytical relationship between the system's energy and its Centre-of-Pressure (CoP) is derived, providing a means of predicting the robot's balancing behaviour based on the evolution of the closed-loop system's actual and critical, energy values. An appropriate manipulation of this expression facilitates a real-time tuning of the active stiffness gains, which may be viewed as a process of constraining the robot's energy to residing within a ‘safe region’ of operation. Walking experiments performed using the COmpliant huMANoid (COMAN), reveal the proposed technique's accuracy in terms of predicting stable states, in addition to its capability of enabling an automated real-time tuning of a robot's impedance levels. The proposed strategy has permitted the stable replication of a wide range of joint impedance values, which is imperative for ground interaction during walking.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116034930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A method for evaluating humanoid robots using anthropometric and biomechanical data","authors":"N. Thoma, James Holley, W. Verdeyen","doi":"10.1109/HUMANOIDS.2017.8239540","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8239540","url":null,"abstract":"Humanoids are generalists, often lacking specific tasks or goals that can be used to evaluate or compare robot performance. Evaluating humanoids through a standardized, generalizable approach can yield a measure of robot performance on yet to be defined tasks. Anthropometric and biomechanical data from human factors design guides can be used as a baseline for evaluating humanoid performance in human-engineered environments. Past humanoid designs have used anthropometric data to define their design parameters, but no standard exists for evaluating humanoid performance. By creating a standard for humanoids, a robots capabilities can be characterized without explicitly testing every conceivable task. NASAs Valkyrie robot was tested against the anthropometric, range of motion, mass property, and strength data from NASAs Constellation Program Human-Systems Integration Requirements. The robot model offered anthropometric and range of motion data, and hardware tests were used to collect mass proportion and strength data. According to the standard used, Valkyrie fell within acceptable bounds for 59% of anthropometry, 64% of range of motion, 13% of mass property and 35% of criticality 1 strength measurements. Evaluating humanoids against the standards upon which human systems are built yields a quantitative measure of robot performance in human-engineered environments. This proposed approach can be generalized to evaluate other humanoids against a common standard. Failures or weaknesses in the humanoid design can be identified and reevaluated over time. Metric performance can be improved through advancements in robot control, operator skill, and hardware redesign.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116162846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kazutoshi Tanaka, Satoshi Nishikawa, Ryuma Niiyama, Y. Kuniyoshi
{"title":"Humanoid robot performing jump-and-hit motions using structure-integrated pneumatic cable cylinders","authors":"Kazutoshi Tanaka, Satoshi Nishikawa, Ryuma Niiyama, Y. Kuniyoshi","doi":"10.1109/HUMANOIDS.2017.8246948","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8246948","url":null,"abstract":"In this study, a structure-integrated pneumatic cable cylinder has been developed to serve as an actuator for a humanoid robot. The performance of a robot in jumping and hitting a flying object (jump-and-hit motions) has been tested in order to predict its performance during immediate and dynamic whole-body motions. The authors tested the movement of the robofs arms using a cylinder, and jumping motions were simulated to determine the design parameters for the robot performing jump-and-hit operations. Test results for the robofs arms demonstrated that a two-kilogram arm, constructed using a C3, linder with a piston, 32 mm in diameter, moves 75 degrees in 0.44 s. Simulation results for a bipedal robot's forward jumping motion demonstrated that the top of its trunk, with a 50 mm joint force — torque ratio, moves forward by 3.0 m. Using the cylinder for the robofs arms and a joint with the above force — torque ratio in its legs, a prototype of a humanoid robot has been developed that performs a variety of jump-and-hit motions with a ball flying at it from different directions. Thus, the proposed design allows robots to conveniently perform immediate and dynamic whole-body motions.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114582887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards ergonomie control of human-robot co-manipulation and handover","authors":"L. Peternel, Wansoo Kim, J. Babič, A. Ajoudani","doi":"10.1109/HUMANOIDS.2017.8239537","DOIUrl":"https://doi.org/10.1109/HUMANOIDS.2017.8239537","url":null,"abstract":"In this paper, we propose a novel method for the control of human-robot eo-manipulation that takes into aeeount the ergonomie requirements for the human eo-worker. The robot uses a whole-body dynamic model of the human to optimise for the position of the eo-manipulation task in the workspace. In this configuration, the overloading joint torques, i.e. the effects of an external load in human body joints, are minimised. In addition, the optimisation process ineludes several constraints, sueh as human arm manipulability properties, to ensure that the human has a good manipulation eapaeity in the given eonfiguration. The main advantage of this approaeh is that the robot ean potentially help to reduee the work-related strain and increase the productivity of the human eo-worker. In addition, the on-line estimation of the overloading joint torques does not require an external foree plate or sensor insoles. We validated the proposed method with experiments in two eo-manipulation tasks: human using a device to polish an object that is delivered by the robot and a human-robot object handover. The results show that the proposed method is successful in achieving the ergonomie conditions for the human during the eollaboration.","PeriodicalId":143992,"journal":{"name":"2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128584984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}