Deep reinforcement learning with artificial microswimmers

Abstract

Artificial microswimmers are active particles designed to mimic the behavior of living microorganisms. The adaptive behavior of the latter is based on the experience they gain through interactions with the environment. They are also subjected to Brownian motion at these length scales which randomizes their position and propulsion direction making it a key feature in the adaptation process. However, artificial systems are limited in their ability to adapt to such noise and environmental stimuli. In this work, we combine artificial microswimmers with a reinforcement learning algorithm to explore their adaptive behavior in a noisy environment. These self-thermophoretic active particles are propelled and steered by generating thermal gradients on their surface with a tightly focused laser beam. They are also imaged under a microscope in real-time to monitor their dynamics. With such a versatile platform capable of real-time control and monitoring, we demonstrated the solution to a standard navigation problem under the inevitable influence of Brownian motion by introducing deep reinforcement learning, specifically deep-Q-learning. We also identified different noises in the system and how they affected the learning speed and navigation strategies picked up by the microswimmer.