Assured Autonomy for Multi-Agent Systems: Methods from Game Theory and Reinforcement Learning

Multi-agent systems appear in many applications of robotics and cyber-physical systems such as manufacturing, logistics, precision agriculture, environmental monitoring, and smart infrastructure/city. Such dynamical systems typically involve numerous agents who have limited capabilities (e.g., sensing, communication, computation) and operate in the face of uncertainty and partial information about the overall system. Achieving the autonomy of these large-scale complex systems under performance guarantees (e.g., safety, efficiency, robustness) requires scalable and provably correct control, machine learning, and optimization algorithms/architectures.
In this talk, I will present methods based on game theory and reinforcement learning with applications to planning
and control of mobile robots under performance guarantees. In the first part of the talk, I will discuss how
game theoretic learning can be used to develop distributed algorithms to achieve cooperative multi-robot missions such as optimally covering an area or serving complex tasks that are distributed over space and time.
More specifically, I will present how such coordination problems can be solved in a scalable manner by formulating the underlying coordination problem as a game with a special structure and using a suitable learning algorithm to drive the robots to optimal configurations. In the second part, I will talk about a constrained reinforcement learning problem where the goal is to learn an optimal control policy in a Markov Decision Process (MDP) while ensuring a probabilistic satisfaction of a complex spatio-temporal constraint on the resulting trajectories. I will present a novel graph-theoretic approach, which can be integrated into standard RL algorithms such as Q-learning, to ensure the satisfaction of the constraint with a desired probability throughout learning. I will conclude the talk with some ongoing work and future directions.