Multi-agent maintenance scheduling of generation unit in electricity market using safe deep reinforcement learning algorithm

Improving maintenance scheduling of generation units in an electricity market would increase the safety and reliability of the system. This problem can be modeled as a multi-agent bi-level decision-making problem associated with some safety constraints. In the first level, the units, modeled as agents, decide about their maintenance scheduling and are responsible to satisfy the safety constraints. In the second level, the independent system operator (ISO) clears the market and calculates the electricity price while ensuring that the demand of the system is satisfied. Incomplete information of other units' decisions and the requirement to satisfy safety and demand constraints make this problem particularly challenging. This paper proposes a safe reinforcement learning algorithm for generation unit maintenance scheduling in a competitive electricity market environment. In this problem, each unit aims to find a preventive maintenance scheduling which retains its reliability and satisfies safety constraints. Bi-level optimization and reinforcement learning are potential candidates for solving this problem. However, bi-level optimization and reinforcement learning cannot handle the challenges of incomplete information and safety constraints, respectively. To handle these challenges, we propose a safe deep reinforcement learning algorithm which combines reinforcement learning and a predicted safety filter. In the proposed method, the reinforcement learning algorithm can tackle the challenges of incomplete information by getting feedback from the environment and learning strategies of the other agents. In addition, the predicted safety filter guarantees that the safety constraints are satisfied and handles the challenges of critical safety constraints. We evaluate the performance of the proposed algorithm on the IEEE 30-bus system. We compare the results of the proposed algorithm with other state of the art Q-learning algorithm. The results demonstrate that the proposed approach can satisfy the system safety constraints and increase the profit of units.