XLNet: A Multimodal Fusion Network for Breast Cancer BI-RADS Classification Based on Ultrasound Images and RF Signals

Inhalt:
Early and accurate diagnosis of breast cancer can significantly increase patient survival rates. It is valuable for predicting breast malignancies to assess the risk stratification of a breast lesion. In this study, we propose a multimodal deep learning network XLNet for breast cancer BI-RADS classification, which takes ultrasound images and radio-frequency signals in the form of channel fusion as input. Furthermore, we introduce a novel feature extraction module ICMA which combines the InceptionNeXt architecture and the Efficient Multi-Scale Attention (EMA) mechanism. To enhance the robustness of the network, data augmentation techniques were employed. The proposed method has been evaluated on the publicly available OASBUD dataset. The results show that the XLNet provides a superior performance across all evaluation metrics, with an accuracy of 81.72%, precision of 83.53%, and F1-score of 81.70%, confirming its effectiveness for the BIRADS classification. In conclusion, this study introduces a novel application of multimodal deep learning to breast cancer BI-RADS classification. The proposed multimodal fusion network XLNet achieves the highest diagnostic accuracy, offering significant value in supporting more reliable clinical decision-making.