Data Clustering and Categorization for Processing Results from Penetration Testing

Abstract:
As cyber threats and their potential impacts increase, the need for testing cyber resilience has become more important. Web applications are one of the frequent targets of cyberattacks, and therefore, the need for their penetration testing is desirable. However, these applications can contain up to tens of thousands of web resources, making the testing process very difficult. Our paper focuses on categorizing similar web resources from the reconnaissance phase to increase the overall effectiveness of penetration testing. For this purpose, we designed and developed a system for clustering and categorizing web resources using cluster analysis. We experimentally tested our solution in two iterations on 10,000 and 50,000 resources. The results show that e-commerce and newspaper websites contain a large amount of similar content, which our system was able to detect and provide penetration testers with the filtered sources for the next phase of penetration testing.