Advanced Training Methods for Reinforcement-learning-based AC Optimal Power Flow

Abstract:
Solving AC optimal power flow (OPF) problems is becoming increasingly important for distribution system operators to coordinate the use of flexibilities from distributed energy resources. State-of-the-art mathematical AC-OPF approaches suffer from a high computational burden and convergence issues. An emerging computationally efficient alternative is the reinforcement-training-based AC-OPF (RT-OPF), where an artificial neural network (ANN) is trained based on the evaluation of an augmented loss function. Using time series training data to train the ANN of the RT-OPF often fails to achieve satisfactory training results. To improve the training results and at the same time reduce the training time, we therefore propose to use cube-scenario-based training. Cube-scenario-based training assumes a strong correlation between all PV generators, all wind generators and all loads, respectively. Each cube scenario thus consists of only three scaling factors, one for each of these three types. The three scaling factors are scaled independently from 0% to 100% spanning the whole range of possible grid states while keeping the number of scenarios small. After a topology change, time consuming retraining of the ANN is required. We therefore propose to use an approach from transfer learning where only the last two layers of the ANN are retrained achieving a reduction in retraining time while maintaining the optimization quality after a topology change. In our simulations, the performance of the RT-OPF is compared to that of a mathematical AC-OPF in terms of optimization quality showing comparable results. Our simulations demonstrate significant performance improvements over the state-of-the-art approach.