Scalable AI for the continuous improvement of energy forecasts

Abstract:
The weather-dependent volatility in the electrical energy supply system requires reliable forecasting models that take dynamic changes in power grids into account. With the increasing share of renewable energies, a fast and scalable process for energy forecasting becomes crucial. Therefore, in this article, we present a Machine Learning Operations (MLOps)- based concept for the continuous adaptation of energy forecasts and deploy and test it as a forecasting system in a Kubernetes environment. The forecasting system allows to regularly train and roll out individual models for each forecasting object. Experiments on scalable forecasting show that the system meets the time-critical requirements for renewable energy forecasts. We compare a python-based implementation with a java-based one with varying scaling levels. By scaling forecasting applications on demand, forecasts can be generated for 10,000 plants in less than 5 minutes. The shortest runtime was achieved by running a Java application at medium scale with up to 30 services running in parallel. The fact that higher scaling with up to 60 parallel running services had a longer runtime shows that a higher horizontal scaling does not necessarily lead to higher throughput in forecast generation. We therefore conclude that the runtime can be reduced by using a suitable implementation language and the optimal level of horizontal scaling.