Effects of specifying robotic missions in behavior trees and state machines

The task of defining the robot’s mission is moving from professional developers and roboticists to the end-users. Robot missions, traditionally implemented in source code with text-based programming languages, present challenges for non-programmers. To this end, many domain-specific languages (DSLs) have been established in robotics. They are typically built upon an established paradigm, where behavior trees and state machines have become the most popular ones in robotics. These paradigms offer different levels of abstraction and control structures, which promise to improve the comprehension, correctness, and usability of missions. However, so far, there are no evaluation and validation studies to determine the effects of using either paradigm for mission specification by end-users. We present a controlled experiment on the effectiveness and efficiency of these paradigms for specifying robot missions by end-users. It measures mission comprehension, correctness and usability by examining language constructs, documentation, and usage. Our findings indicate that participants rated both paradigms above the neutral midpoint that is, greater than three on a 5-point scale in comprehension, with negligible variance in preference. However, state machine received marginally higher ratings in overall usability. The results further indicate that in the concrete syntax of the DSLs used in the experiments, user interfaces need improvement, more tutorials (including videos/audios) are required. End-users also need basic training in behavior trees, state machines, programming, and robotics. While the DSLs provide reasonable abstraction compared to text-based languages, further refinement is needed to enhance usability and correctness. We discuss actionable insights for improving the usability of behavior trees and state machines in robotics and automation.