Learning PID controller

[jpark0315]

Hey great work! I am trying to extend your work and get a learning agent to auto-tune the PID parameters, and I was just wondering how you would personally go about doing this??

Thanks!

[NaleRaphael]

Hi @jpark0315
Really sorry for the late response, I've been busy with other stuffs recently.

And since I haven't gotten into fields related to reinforcement learning and control theory for a long period of time, I'm afraid that I cannot give you a constructive answer about this. Therefore, the following comment might not be completely correct.

In my opinon, the difficulty of this problem is how to formulate the problem of updating those controller coefficients. With a quick search for "reinforcement learning + PID controller tuning", I found the following paper may be helpful to you:

• paper: An adaptive deep reinforcement learning approach for MIMO PID control of mobile robots
• code: https://github.com/IgnacioCarlucho/deepPID

Although it's a MIMO system, but it seems that we can simplify it into the one like cart-pole system. And if you have checked the file pid.py in deepPID, you can see that those coefficients for PID controllers are actually determined by the input parameter action. And that parameter action will be given by model through this line.

As for the cart-pole system, note that the input space of it is discrete (0 or 1). You might need to replace it with a continuous input space, see also this part I've written. Otherwise, you might need to try Sliding Mode Control instead. (e.g. Deep Reinforcement Learning Based Self-Configuring Integral Sliding Mode Control Scheme for Robot Manipulators)

And these are what I can provide to you currently. I'll keep this thread open and let you know if there is any progress on this.