Intelligent Thermal Control Algorithm Based on Deep Deterministic Policy Gradient for Spacecraft

Abstract

In practical applications, the control parameters of a proportional integral derivative (PID) thermal controller are difficult to self-tuning online. As the control object or environment changes, the control parameters are required to change accordingly. An intelligent thermal controller based on the deep deterministic policy gradient, called DRLTC, is proposed. Two types of reinforcement learning agents were designed in DRLTC, which can automatically adjust the control parameters of the thermal controllers and self-optimize online after training. Both theoretical and experimental results revealed that, when the control object was the main mirror support, the DRLTC achieved a control precision of 0.01 °C. Additionally, the steady-state error was reduced by 40.2%, 62.5%, and 33.3% in the simulation, and by 5.6%, 80.6%, and 85.7% in the experiment, compared with the reinforcement learning PID, neural network PID, and Fuzzy PID, respectively. When the control object was changed to the main mirror installation, the DRLTC achieved a control precision of 0.02 °C, and the steady-state error was reduced by 87.5%, 91.7%, and 90.9% in the simulation, and by 80.2%, 90.6%, and 85.7% in the experiment, compared with the above-mentioned thermal control strategies, respectively. Therefore, the DRLTC has better universality, stronger robustness, and achieve more energy saving.

Yan XIONG(熊琰)

Ph.D. student from the University of Chinese Academy of Sciences

My research interests include intelligent control of gyroscopes using model-based reinforcement learning, intelligent thermal control based on deep reinforcement learning for space load, and engineering applications of AI, such as robotic systems, spacecraft strategic planning, etc.