Facilitating Physical-Computer System Design through Data-Driven Natural-Language Interaction

Designing and creating physical computing system can be challenging for novice user.In this paper, we present FritzBot, an intelligent conversational agent offering assistance for novice users on constructing physical-computing systems through natural-language interaction. We create a lexical circuit-event database based on 152 student reports from the undergraduate physical-computing course in a local art school. The LSTM-CRF network of FrzitBot is trained on that database, and is able to extract the input and the output events from the user's description, and generate the circuit and the code along with the construction guidelines. A user study shows that FritzBot can significantly reduce the construction effort and time spent for novice users on physical-computing task.