Development and Validation of a Testbed for AI/ML QoS Prediction Algorithm Evaluation

Abstract:
New use cases for connected vehicles have emerged with the introduction of 5G New Radio, such as tele-operated driving or information sharing from infrastructure-mounted external sensors. Predictive Quality of Service has been recently introduced as a way to improve such use cases. For example, Predictive Quality of Service can inform the autonomous vehicle about upcoming network performance degradation. Machine Learning can be used for Predictive Quality of Service, but a large amount of data with appropriate characteristics is needed to train corresponding models. Simulation-based Machine Learning can help to generate relevant data including a sufficient amount of outlier data. However, uncertainties in the input generation process can affect model evaluation. We designed and implemented a test environment, including a 5G network, in a lab to evaluate the sensitivity of an Machine Learning model to variation in the input generation process of offered data traffic. It was found that the Machine Learning model exhibits different sensitivities against changes in the input data process. We suggest methods from the area of sensitivity analysis for evaluating input-output relations of Machine Learning models with the purpose to identify the impact of parameter uncertainties on the variability of prediction performance. More specifically, these methods can be used to improve confidence in the adoption of Machine Learning model proposals.