{"title":"A Transformer-Based Thermal Surrogate Model for Cooling Control in Data Centers","authors":"Hanchen Zhou;Ni Mu;Qing-Shan Jia","doi":"10.1109/LRA.2024.3512192","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512192","url":null,"abstract":"With the rapid development of data centers in the Big Data era, the operation of their cooling systems has huge energy saving potential, so the optimization of their control is of great significance for research. The main challenge in the optimization problem above is the prediction of the complicated temperature field. The most recognized Computational Fluid Dynamics (CFD) simulation consumes too much time to be applied in the real-time optimization. To address this problem, a Transformer-based thermal surrogate model is proposed. Specifically, self-attention is used for capturing the temporal and spatial characteristics in the temperature field to replace CFD. Then, the optimization problem is formulated and a surrogate model-based Soft Actor-Critic (SAC) solution framework is proposed. Finally, the control performance is verified in the CFD-based platform 6SigmaRoom and the widely-used Artificial Neural Network (ANN) is selected as the baseline. Numerical experiments demonstrate that the proposed surrogate model makes predictions faster than CFD and more accurately than ANN while the control based on it achieves a 7.12% reduction in energy consumption, finally improving the energy efficiency.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"644-651"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Federico Pizarro Bejarano;Lukas Brunke;Angela P. Schoellig
{"title":"Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents","authors":"Federico Pizarro Bejarano;Lukas Brunke;Angela P. Schoellig","doi":"10.1109/LRA.2024.3512374","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512374","url":null,"abstract":"Reinforcement learning (RL) controllers are flexible and performant but rarely guarantee safety. Safety filters impart hard safety guarantees to RL controllers while maintaining flexibility. However, safety filters can cause undesired behaviours due to the separation between the controller and the safety filter, often degrading performance and robustness. In this letter, we analyze several modifications to incorporating the safety filter in training RL controllers rather than solely applying it during evaluation. The modifications allow the RL controller to learn to account for the safety filter. This letter presents a comprehensive analysis of training RL with safety filters, featuring simulated and real-world experiments with a Crazyflie 2.0 drone. We examine how various training modifications and hyperparameters impact performance, sample efficiency, safety, and chattering. Our findings serve as a guide for practitioners and researchers focused on safety filters and safe RL.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"788-795"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Se Hwan Jeon;Seungwoo Hong;Ho Jae Lee;Charles Khazoom;Sangbae Kim
{"title":"CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control","authors":"Se Hwan Jeon;Seungwoo Hong;Ho Jae Lee;Charles Khazoom;Sangbae Kim","doi":"10.1109/LRA.2024.3512254","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512254","url":null,"abstract":"The parallelism afforded by GPUs presents significant advantages in training controllers through reinforcement learning (RL). However, integrating model-based optimization into this process remains challenging due to the complexity of formulating and solving optimization problems across thousands of instances. In this work, we present \u0000<monospace>CusADi</monospace>\u0000, an extension of the \u0000<monospace>casadi</monospace>\u0000 symbolic framework to support the parallelization of arbitrary closed-form expressions on GPUs with \u0000<monospace>CUDA</monospace>\u0000. We also formulate a closed-form approximation for solving general optimal control problems, enabling large-scale parallelization and evaluation of MPC controllers. Our results show a ten-fold speedup relative to similar MPC implementation on the CPU, and we demonstrate the use of \u0000<monospace>CusADi</monospace>\u0000 for various applications, including parallel simulation, parameter sweeps, and policy training.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"899-906"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adaptive Neural Computed Torque Control for Robot Joints With Asymmetric Friction Model","authors":"Ruiqing Luo;Zhengtao Hu;Menghui Liu;Liang Du;Sheng Bao;Jianjun Yuan","doi":"10.1109/LRA.2024.3512372","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512372","url":null,"abstract":"The nonlinearity and uncertainty of dynamics pose significant challenges to ensuring the tracking performance of joint trajectories, especially time-varying effects on the load and temperature. In this letter, we present an adaptive neural computed torque control scheme to improve the tracking accuracy of the robot joint towards various tasks, which is a novel semiparametric model including a parametric friction model and a nonparametric compensator trained with multiple radial basis function neural networks \u0000<inline-formula><tex-math>$(text{MRBFNNs})$</tex-math></inline-formula>\u0000. Specifically, the asymmetric model considers velocity-, load-, and temperature-dependent friction phenomena. The computed torque controller integrates the sliding mode method and the proposed friction model to reduce the boundary layer of fluctuated disturbances and achieve globally asymptotic convergence. MRBFNNs are trained separately to further compensate for the unmodeled nonlinearity and parameter uncertainty in real time during the trajectory tracking process. The comparative experiments were carried out on a robot joint, validating that our asymmetric model significantly improves correspondence to reality in terms of friction; the proposed control strategy exhibits the superior tracking performance of joints with variable payloads.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"732-739"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multimodal Fusion SLAM With Fourier Attention","authors":"Youjie Zhou;Guofeng Mei;Yiming Wang;Yi Wan;Fabio Poiesi","doi":"10.1109/LRA.2024.3512252","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512252","url":null,"abstract":"Visual SLAM is particularly challenging in environments affected by noise, varying lighting conditions, and darkness. Learning-based optical flow algorithms can leverage multiple modalities to address these challenges, but traditional optical flow-based visual SLAM approaches often require significant computational resources. To overcome this limitation, we propose FMF-SLAM, an efficient multimodal fusion SLAM method that utilizes fast Fourier transform (FFT) to enhance the algorithm efficiency. Specifically, we introduce a novel Fourier-based self-attention and cross-attention mechanism to extract features from RGB and depth signals. We further enhance the interaction of multimodal features by incorporating multi-scale knowledge distillation across modalities. We also demonstrate the practical feasibility of FMF-SLAM in real-world scenarios with real time performance by integrating it with a security robot by fusing with a global positioning module GNSS-RTK and global Bundle Adjustment. Our approach is validated using video sequences from TUM, TartanAir, and our real-world datasets, showcasing state-of-the-art performance under noisy, varying lighting, and dark conditions.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1050-1057"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142875141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Camera-LiDAR Extrinsic Calibration Using Constrained Optimization With Circle Placement","authors":"Daeho Kim;Seunghui Shin;Hyoseok Hwang","doi":"10.1109/LRA.2024.3512253","DOIUrl":"https://doi.org/10.1109/LRA.2024.3512253","url":null,"abstract":"Monocular camera-LiDAR data fusion has demonstrated remarkable environmental perception capabilities in various fields. The success of data fusion relies on the accurate matching of correspondence features from images and point clouds. In this letter, we propose a target-based Camera-LiDAR extrinsic calibration by matching correspondences in both data. Specifically, to extract accurate features from the point cloud, we propose a novel method that estimates the circle centers by optimizing the probability distribution from the initial position. This optimization involves generating the probability distribution of circle centers from circle edge points and using the Lagrangian multiplier method to estimate the optimal positions of the circle centers. We conduct two types of experiments: simulations for quantitative results and real system evaluations for qualitative assessment. Our method demonstrates a \u0000<inline-formula><tex-math>$mathbf{21%}$</tex-math></inline-formula>\u0000 improvement in simulation calibration performance for 20 target poses with LiDAR noise of \u0000<inline-formula><tex-math>$mathbf{text{0.03},m}$</tex-math></inline-formula>\u0000 compared to existing methods, and also shows high visual quality in reprojecting point cloud onto images in real-world scenarios.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"883-890"},"PeriodicalIF":4.6,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep Reinforcement Learning-Based Mapless Navigation for Mobile Robot in Unknown Environment With Local Optima","authors":"Yiming Hu;Shuting Wang;Yuanlong Xie;Shiqi Zheng;Peng Shi;Imre Rudas;Xiang Cheng","doi":"10.1109/LRA.2024.3511437","DOIUrl":"https://doi.org/10.1109/LRA.2024.3511437","url":null,"abstract":"Local optima issues challenge mobile robots mapless navigation with the dilemma of avoiding collisions and approaching the target. Planning-based methods rely on environmental models and manual strategies to guide the robot. In contrast, learning-based methods can process original sensor data to navigate the robot in real-time but struggle with local optima. To address this, we designed reward rules that punish the robot for previously visited areas that may trap the robot, and reward it for exploring local areas in diverse ways and escaping from local optima areas. Then, we improved the Soft Actor-Critic (SAC) algorithm by making its temperature parameter adaptive to the current training status, and memorizing it in experiences for strategy updating, bringing additional exploratory behaviors and necessary stability into the training. Finally, with the assistance of auxiliary networks, the robot learns to handle various navigation tasks with local optima risks. Simulations demonstrate the advantages of our method in terms of both success rate and path efficiency compared to several existing methods. Experiments verified the proposed method in real-world scenarios.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"628-635"},"PeriodicalIF":4.6,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiawei Wang;Teng Wang;Wenzhe Cai;Lele Xu;Changyin Sun
{"title":"Boosting Efficient Reinforcement Learning for Vision-and-Language Navigation With Open-Sourced LLM","authors":"Jiawei Wang;Teng Wang;Wenzhe Cai;Lele Xu;Changyin Sun","doi":"10.1109/LRA.2024.3511402","DOIUrl":"https://doi.org/10.1109/LRA.2024.3511402","url":null,"abstract":"Vision-and-Language Navigation (VLN) requires an agent to navigate in photo-realistic environments based on language instructions. Existing methods typically employ imitation learning to train agents. However, approaches based on recurrent neural networks suffer from poor generalization, while transformer-based methods are too large in scale for practical deployment. In contrast, reinforcement learning (RL) agents can overcome dataset limitations and learn navigation policies that adapt to environment changes. However, without expert trajectories for supervision, agents struggle to learn effective long-term navigation policies from sparse environment rewards. Instruction decomposition enables agents to learn value estimation faster, making agents more efficient in learning VLN tasks. We propose the Decomposing Instructions with Large Language Models for Vision-and-Language Navigation (DILLM-VLN) method, which decomposes complex navigation instructions into simple, interpretable sub-instructions using a lightweight, open-sourced LLM and trains RL agents to complete these sub-instructions sequentially. Based on these interpretable sub-instructions, we introduce the cascaded multi-scale attention (CMA) and a novel multi-modal fusion discriminator (MFD). CMA integrates instruction features at different scales to provide precise textual guidance. MFD combines scene, object, and action information to comprehensively assess the completion of sub-instructions. Experiment results show that DILLM-VLN significantly improves baseline performance, demonstrating its potential for practical applications.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"612-619"},"PeriodicalIF":4.6,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Multi-Modal Fusion-Based 3D Multi-Object Tracking Framework With Joint Detection","authors":"Xiyang Wang;Chunyun Fu;Jiawei He;Mingguang Huang;Ting Meng;Siyu Zhang;Hangning Zhou;Ziyao Xu;Chi Zhang","doi":"10.1109/LRA.2024.3511438","DOIUrl":"https://doi.org/10.1109/LRA.2024.3511438","url":null,"abstract":"In the classical tracking-by-detection (TBD) paradigm, detection and tracking are separately and sequentially conducted, and data association must be properly performed to achieve satisfactory tracking performance. In this letter, a new multi-object tracking framework is proposed, which integrates object detection and multi-object tracking into a single model. The proposed tracking framework eliminates the complex data association process in the classical TBD paradigm, and requires no additional training. Secondly, the regression confidence of historical trajectories is investigated, and the possible states of a trajectory (weak object or strong object) in the current frame are predicted. Then, a confidence fusion module is designed to guide non-maximum suppression for trajectories and detections to achieve ordered and robust tracking. Thirdly, by integrating historical trajectory features, the regression performance of the detector is enhanced, which better reflects the occlusion and disappearance patterns of objects in real world. Lastly, extensive experiments are conducted on the commonly used KITTI and Waymo datasets. The results show that the proposed framework can achieve robust tracking by using only a 2D detector and a 3D detector, and it is proven more accurate than many of the state-of-the-art TBD-based multi-modal tracking methods.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"532-539"},"PeriodicalIF":4.6,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rawad E. H. Altagiuri;Omar H. A. Zaghloul;Brian H. Do;Fabio Stroppa
{"title":"A Motion Planner for Growing Reconfigurable Inflated Beam Manipulators in Static Environments","authors":"Rawad E. H. Altagiuri;Omar H. A. Zaghloul;Brian H. Do;Fabio Stroppa","doi":"10.1109/LRA.2024.3511405","DOIUrl":"https://doi.org/10.1109/LRA.2024.3511405","url":null,"abstract":"Soft growing robots have the potential to be useful for complex manipulation tasks and navigation for inspection or search and rescue. They are designed with plant-like properties, allowing them to evert and steer multiple links and explore cluttered environments. However, this variety of operations results in multiple paths, which is one of the biggest challenges faced by classic pathfinders. In this letter, we propose a motion planner based on A\u0000<inline-formula><tex-math>$^*$</tex-math></inline-formula>\u0000 search specifically designed for soft growing manipulators operating on predetermined static tasks. Furthermore, we implemented a stochastic data structure to reduce the algorithm's complexity as it explores alternative paths. This allows the planner to retrieve optimal solutions over different tasks. We ran demonstrations on a set of three tasks, observing that this stochastic process does not compromise path optimality.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"516-523"},"PeriodicalIF":4.6,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}