فاطمه مهدوي گلميشه

This thesis aims to design safe an‎d robust formation control for heterogeneous cooperative robots in an unknown environment as a multi-agent system. Since agents need to see the results of their actions to achieve better performance, multi-agent reinforcement learning is suggested. In this research, the dynamics of agents were assumed to be nonlinear an‎d affine to the input. Additionally, we used model-free reinforcement learning, so detailed information about the system dynamics was not needed to design the optimal control. Therefore, the optimal state feedback design could only be determined based on the collected data. In the first step, by assuming that the environment is known for the agents an‎d by integrating the local barrier functions an‎d the multi-agent reinforcement learning algorithm, the constrained optimization problem became an unconstrained optimization problem. In this way, the control inputs of the follower agents were obtained simultaneously using the proposed safe multi-agent reinforcement learning algorithm. Stability analysis an‎d safety assurance have been done in this thesis section. In the second step, after completing the abovementioned steps for the known environment, we proposed a multilayer approach to tackle this problem in an unknown two-dimensional environment with convex obstacles. This approach comprised three main layers, each carrying out its responsibility by receiving information from the preceding layers. Therefore, we used the concept of barrier function to design the leader’s safe path in the first layer. The second layer was responsible for designing the optimal parameters for a safe formation. Finally, using the multi-agent reinforcement learning algorithm in the third layer led to designing the optimal control for the follower agents. Finally, the proposed layered approach in the second step was extended to the disturbed system to perform operations in the three-dimensional environment. In addition, the inverse reinforcement learning algorithm was used to improve the tracking performance in the third layer. As a result, the problem of optimal, safe, an‎d robust formation control for nonlinear an‎d heterogeneous cooperative robots was solved without the need for information about all agents’ dynamic parameters an‎d without the need for information about the environment an‎d the cost function for followers in the three-dimensional environment. Multi-agent systems consisting of UAVs an‎d USVs have been considered simulation examples to eva‎luate the proposed approaches an‎d observe their numerical results.

يادگيري تقويتي ايمن , تابع مانع , سيستم چند عاملي غير خطي و غير همگن , كنترل بهينه , يادگيري تقويتي معكوس , كنترل 𝐻∞ , محيط ناشناخته

Barrier function, convex obstacles, formation, 𝐻∞ control, inverse optimal control, inverse reinforcement learning, model-free reinforcement learning, nonlinear an‎d heterogeneous multi-agent system, optimal control, robotics, robust reinforcement learning, safe reinforcement learning, unknown environment

Fateme Mahdavi

Dr. Shamaghdari