IEEE Transactions on Robotics最新文献

{"title":"Human-in-the-Loop Multirobot Information Gathering With Inverse Submodular Maximization","authors":"Guangyao Shi;Shipeng Liu;Ellen Novoseller;Jiazhen Liu;Feifei Qian;Gaurav Sukhatme","doi":"10.1109/TRO.2026.3738463","DOIUrl":"10.1109/TRO.2026.3738463","url":null,"abstract":"In this article, we consider a new type of inverse combinatorial optimization, inverse submodular maximization (ISM), and its application to human-in-the-loop multirobot information gathering. Forward combinatorial optimization, which solves a combinatorial problem given reward- or cost-related parameters, is widely used in multirobot coordination. These parameters are typically designed offline by domain experts and then used for online robot coordination. We consider settings in which human supervisors provide desired path primitives during task execution, and the robots adapt their internal decision-making parameters accordingly. From the perspective of inverse combinatorial optimization, we formulate ISM for a family of multirobot information gathering problems, where the goal is to find new parameters that minimally deviate from the current ones while causing a greedy algorithm to produce path primitives consistent with the human suggestions. For a single suggestion, the problem can be formulated as a mixed integer quadratic program, which becomes intractable for existing solvers as the problem size grows. We, therefore, propose a new branch-and-bound algorithm. For multiple suggestions from several human supervisors, the problem is formulated as a multiobjective optimization problem and addressed using a heuristic, with Pareto Monte Carlo tree search employed in the search stage. Numerical simulations in multirobot scientific data collection and event detection-driven coverage control demonstrate that the proposed formulations and algorithms enable robots to effectively adapt to human suggestions.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"3944-3963"},"PeriodicalIF":11.1,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149078132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Nonholonomic Dynamic Movement Primitives","authors":"Giuseppe Infantone;Giorgio Grioli;Antonio Bicchi","doi":"10.1109/TRO.2026.3706555","DOIUrl":"10.1109/TRO.2026.3706555","url":null,"abstract":"We address imitation-based trajectory generalization for robots subject to nonholonomic constraints. Standard dynamic movement primitives (DMPs) are effective for unconstrained systems, but they can generate infeasible trajectories when applied to wheeled robots, such as unicycles or car-like vehicles. We introduce nonholonomic DMPs (NHDMPs), a framework that reformulates the imitation objective in quasi-velocity space and replaces the standard linear attractor with a Lyapunov-stable controller that enforces nonholonomic feasibility, together with two forcing terms learned from demonstrations. We derive a numerical optimization-based reference benchmark and evaluate two NHDMP variants: an optimized version with per-instance parameter optimization and a trained version with fixed parameters learned offline. On 1000 random trajectories, the optimized variant achieves imitation discrepancy close to the numerical optimum while being an order of magnitude faster; the trained variant achieves comparable trajectory quality at lower computational cost, outperforming prior nonholonomic DMP methods on the considered benchmark. Physical experiments on the AlterEgo wheeled platform, together with simulations of sequential primitive chaining for car-like road navigation, demonstrate the approach across systems of increasing complexity.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"2745-2764"},"PeriodicalIF":11.1,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generative Adversarial Self-Imitation Learning With Large Language Model Feedback for Robot Control and Navigation","authors":"Ke Zhang;Zheng Fang;Enqi Zhao;Zicheng Sun;Jianwu Fang;Jie Huang;Eric Nichols;Randy Gomez;Bo He;Jianru Xue;Guangliang Li","doi":"10.1109/TRO.2026.3710412","DOIUrl":"10.1109/TRO.2026.3710412","url":null,"abstract":"Deep reinforcement learning (DRL) has achieved great success in many simulated and real-world robotic tasks. However, the difficulty of designing efficient and dense reward functions makes applying DRL to tackle complex long-horizon and open-world tasks a great challenge. Generative adversarial imitation learning (GAIL) can directly learn policies from the expert trajectories and generalize well in large and complex environments, but relies on high-quality demonstrations and can seldom surpass the performance of the demonstration. Recent work used additional human evaluative feedback to facilitate GAIL to learn faster and surpass the demonstrations, but still requires suboptimal demonstrations. Moreover, it is costly and difficult for human expert to provide relatively high-quality demonstrations and evaluative feedback for various tasks. To address the above issues, in this article, we propose generative adversarial self-imitation learning from demonstration and large language model (LLM) feedback (GASL<inline-formula><tex-math>$^{3}$</tex-math></inline-formula>MF), since LLMs encode rich commonsense knowledge and can perform a variety of reasoning tasks. GASL<inline-formula><tex-math>$^{3}$</tex-math></inline-formula>MF allows a robot to learn from poor demonstrations and gradually replace them with its own good trajectories evaluated by LLM feedback. Our results in four physics-based control tasks and a mobile robot navigation task show that, even with demonstrations of poor performance or not completing the task, GASL<inline-formula><tex-math>$^{3}$</tex-math></inline-formula>MF can learn faster with close to optimal performance, and generalize well to different environments and the real world with sim-to-real adaptation. Further analysis shows that the overall distribution of LLM feedback closely resembles that of human feedback and remains closer to that of ground-truth rewards than human feedback. Finally, our GASL<inline-formula><tex-math>$^{3}$</tex-math></inline-formula>MF method works regardless of the LLM employed, and the LLM feedback from different LLMs remain robust across tasks and even better consistency than human feedback for robot learning in some tasks. These results shed light on the potential of robot imitation learning from even poor or failed demonstrations and broaden its application to a wide range of real-world tasks.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"2877-2897"},"PeriodicalIF":11.1,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148401108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"AnyUser: Translating Sketched User Intent Into Domestic Robots","authors":"Songyuan Yang;Huibin Tan;Kailun Yang;Wenjing Yang;Shaowu Yang","doi":"10.1109/TRO.2026.3686173","DOIUrl":"10.1109/TRO.2026.3686173","url":null,"abstract":"We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interprets multimodal inputs (sketch, vision, and language) as spatial-semantic primitives to generate executable robot actions requiring no prior maps or models. Novel components include multimodal fusion for understanding and a hierarchical policy for robust action generation. Efficacy is shown via extensive evaluations. Quantitative benchmarks on the large-scale dataset showing high accuracy in interpreting diverse sketch-based commands across various simulated domestic scenes. Real-world validation on two distinct robotic platforms, a statically mounted 7-DoF assistive arm (KUKA LBR iiwa) and a dual-arm mobile manipulator (Realman RMC-AIDAL), performing representative tasks such as targeted wiping and area cleaning, confirming the system’s ability to ground instructions and execute them reliably in physical environments. A comprehensive user study involving diverse demographics (elderly, simulated nonverbal, and low technical literacy) demonstrating significant improvements in usability and task specification efficiency, achieving high task completion rates (85.7%–96.4%) and user satisfaction. AnyUser bridges the gap between advanced robotic capabilities and the need for accessible nonexpert interaction, laying the foundation for practical assistive robots adaptable to real-world human environments.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"1921-1941"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147731764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Extreme High-Gain Friction Observer of Flexible Joint Robots With $mathcal {L}_{1}$ Adaptive Framework","authors":"Young Bin Lee;Tae Ho Yun;Min Jun Kim","doi":"10.1109/TRO.2026.3686177","DOIUrl":"10.1109/TRO.2026.3686177","url":null,"abstract":"Compliance control enables flexible joint robots (FJRs) to interact with unknown environments, but joint friction may significantly degrade control performance and backdrivability. While several model-free friction observers for FJRs have been studied in recent decades, current approaches still face challenges when the robot interacts with stiff environments. To tackle this, this article proposes a new friction observer based on an <inline-formula><tex-math>$mathcal {L}_{1}$</tex-math></inline-formula> adaptive framework. The main advantage of the proposed method is that it overcomes a fundamental tradeoff in the state-of-the-art method between accurate friction compensation and natural environmental interactions. Moreover, the proposed approach enables the use of extremely high gains, which yield several additional benefits. First, unlike the conventional methods, which require feedback of so-called nominal signals obtained through simulation, measured motor signals can be fed back into the controllers, leading to a simpler implementation. Second, we provide performance analysis showing that increasing the gain improves performance and results in near-zero steady-state error. Third, the observer’s performance can be adjusted using only a single parameter. Lastly, the numerical issues arising from extremely high gains are alleviated by employing a stable numerical method. The above theoretical findings are validated through simulations, and the effectiveness of the proposed approach is further evaluated with real-world experiments using both single- and seven-joint FJR systems. The results demonstrate that the proposed approach enables robots to interact with stiff environments more naturally, while achieving enhanced friction compensation performance.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"2087-2106"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147731754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"From Hitch to Lift: Autonomous Cable Interlacing by Multi-UAV Teams for Aerial Grasping and Transportation","authors":"Diego S. D'Antonio;Tongshu Wu;Subhrajit Bhattacharya;David Saldaña","doi":"10.1109/TRO.2026.3677011","DOIUrl":"10.1109/TRO.2026.3677011","url":null,"abstract":"The use of cables in aerial manipulation offers a lightweight and flexible alternative to rigid grasping mechanisms. However, achieving autonomous tying and secure transportation of objects with cables remains a significant challenge. In this work, we present a novel method for autonomously securing and transporting objects using multilayer hitches formed in midair by a team of aerial robots. Building upon prior work on polygonal hitch formation, we extend the framework to include layered cable interlacing that increases frictional grip and enables secure object grasping. We introduce two new manipulation actions: multilayer tying and autonomous object release, completing the pipeline for aerial grasping and transportation. We develop a capstan-based analytical model that establishes an exponential scaling law for the effect of cable layers and provides a conservative guideline. The formation algorithm operates in parallel, ensuring scalability to large teams with constant execution time. We validate our system through simulation and hardware experiments, demonstrating fully autonomous object tying, lifting, and releasing using cables alone without human intervention.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"1769-1783"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147506853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"RoEL: Robust Event-Based 3-D Line Reconstruction","authors":"Gwangtak Bae;Jaeho Shin;Seunggu Kang;Junho Kim;Ayoung Kim;Young Min Kim","doi":"10.1109/TRO.2026.3672540","DOIUrl":"10.1109/TRO.2026.3672540","url":null,"abstract":"Event cameras in motion tend to detect object boundaries or texture edges, which produce lines of brightness changes, especially in man-made environments. While lines can constitute a robust intermediate representation that is consistently observed, the sparse nature of lines may lead to drastic deterioration with minor estimation errors. Only a few previous works, often accompanied by additional sensors, utilize lines to compensate for the severe domain discrepancies of event sensors along with unpredictable noise characteristics. In this article, we propose a method that can stably extract tracks of varying appearances of lines using a clever algorithmic process that observes multiple representations from various time slices of events, compensating for potential adversaries within the event data. We then propose geometric cost functions that can refine the 3-D line maps and camera poses, eliminating projective distortions and depth ambiguities. The 3-D line maps are highly compact and can be equipped with our proposed cost function, which can be adapted for any observations that can detect and extract line structures or projections of them, including 3-D point cloud maps or image observations. We demonstrate that our formulation is powerful enough to exhibit a significant performance boost in event-based mapping and pose refinement across diverse datasets, and can be flexibly applied to multimodal scenarios. Our results confirm that the proposed line-based formulation is a robust and effective approach for the practical deployment of event-based perceptual modules.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"1607-1626"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147439772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"IA-TIGRIS: An Incremental and Adaptive Sampling-Based Planner for Online Informative Path Planning","authors":"Brady Moon;Nayana Suvarna;Andrew Jong;Satrajit Chatterjee;Junbin Yuan;Muqing Cao;Sebastian Scherer","doi":"10.1109/TRO.2026.3672542","DOIUrl":"10.1109/TRO.2026.3672542","url":null,"abstract":"Planning paths that maximize information gain for robotic platforms has wide-ranging applications and significant potential impact. To effectively adapt to real-time data collection, informative path planning must be computed online and be responsive to new observations. In this work, we present Incremental and Adaptive Tree-based Information Gathering Using Informed Sampling (IA-TIGRIS), which is an incremental and adaptive sampling-based informative path planner designed for real-time onboard execution. Our approach leverages past planning efforts through incremental refinement while continuously adapting to updated belief maps. We additionally present detailed implementation and optimization insights to facilitate real-world deployment, along with an array of reward functions tailored to specific missions and behaviors. Extensive simulation results demonstrate IA-TIGRIS generates higher-quality paths compared to baseline methods. We validate our planner on two distinct hardware platforms: a hexarotor uncrewed aerial vehicle (UAV) and a fixed-wing UAV, each having different motion models and configuration spaces. Our results show up to a 38% improvement in information gain compared to baseline methods, highlighting the planner's potential for deployment in real-world applications.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"1695-1713"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147439771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Large-Scale Multirobot Task Planning Using Efficient Hierarchical Reinforcement Learning","authors":"Xuan Zhou;Xiang Shi;Lele Zhang;Chen Chen;Hongbo Li;Lin Ma;Fang Deng;Jie Chen","doi":"10.1109/TRO.2026.3686265","DOIUrl":"10.1109/TRO.2026.3686265","url":null,"abstract":"Multirobot task planning (MRTP) at scale in robotic mobile fulfillment systems (RMFS) remains a challenge due to the curse of dimensionality and complex dynamic properties. Aiming to solve these challenges, we construct an end-to-end scalable multirobot task planner capable of scaling to large-scale systems by learning hierarchical planning policies. In this planner, we design a centralized hierarchical temporal task planning framework to mitigate the curse of dimensionality while ensuring timely dynamic response. Following this framework, we propose a novel cycle-constrained asynchronous temporal graph to provide foundation for modeling the system dynamics. Based on the graph representation, we formulate the MRTP problem as a semi-Markov decision process (SMDP) that focuses solely on critical interaction points to improve computational and sampling efficiency. The policies in SMDP are parameterized via a hierarchical temporal attention network with temporal embedding layers to enhance spatio-temporal feature extraction. In addition, the decoder masks in this network naturally ensure that the generated actions strictly satisfy the required dynamic hard constraints. The above hierarchical policies are jointly optimized using an efficient hierarchical REINFORCE with rollout counterfactual baseline method. To further enhance generalization performance on unlearned instances while preventing catastrophic forgetting, we extend it with region expansion curricula. Experiments demonstrate that our planner outperforms state-of-the-art methods on different MRTP instances across simulated and real-world RMFS. It successfully scales to instances with up to 200 robots, 1000 retrieval racks on unlearned maps while maintaining performance advantages.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"2146-2165"},"PeriodicalIF":10.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147731761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Third-Order Gaussian Process Trajectory Representation Framework With Closed-Form Kinematics for Continuous-Time Motion Estimation","authors":"Thien-Minh Nguyen;Ziyu Cao;Kailai Li;William Talbot;Tongxing Jin;Shenghai Yuan;Timothy D. Barfoot;Lihua Xie","doi":"10.1109/TRO.2026.3701957","DOIUrl":"10.1109/TRO.2026.3701957","url":null,"abstract":"In this article, we propose a third-order, i.e., white-noise-on-jerk, Gaussian process trajectory representation (TR) framework for continuous-time motion estimation (ME) tasks. Our framework features a unified TR that encapsulates the kinematic models of both <inline-formula><tex-math>$mathrm{SO(3)}times mathbb {R}^{3}$</tex-math></inline-formula> and <inline-formula><tex-math>$mathrm{SE(3)}$</tex-math></inline-formula> pose representations. This encapsulation strategy allows users to use the same implementation of measurement-based factors for either choice of pose representation, which facilitates experimentation and comparison to make a better choice for the ME task. In addition, unique to our framework, we derive the kinematic models with the <italic>closed-form temporal derivatives of the local variables of <inline-formula><tex-math>$mathrm{SO(3)}$</tex-math></inline-formula> and <inline-formula><tex-math>$mathrm{SE(3)}$</tex-math></inline-formula></i>, which so far has only been approximated based on Taylor expansion in the literature. Our experiments show that these kinematic models can improve the estimation accuracy in high-speed scenarios. All analytical Jacobians of the interpolated states with respect to the support states of the TR, as well as the motion prior factors, are also provided for accelerated Gauss–Newton optimization. Our experiments demonstrate the efficacy and efficiency of the framework in various ME tasks, such as localization, calibration, and odometry, facilitating fast prototyping for ME researchers. We release the source code for the benefit of the community.","PeriodicalId":50388,"journal":{"name":"IEEE Transactions on Robotics","volume":"42 ","pages":"2648-2667"},"PeriodicalIF":11.1,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}