Research on Chinese Text Sumarization Generation Method Based on ERNIE-GEN

Abstract:
Chinese text summarization methods based on transfer learning suffer from insufficient deep information focus and exposure bias issue. This paper propose the ERNIE-GEN-CL method based on ERNIE-GEN, which introduces the continual learning mechanism and uses large chinese corpora for secondary training of the pre-trained model to achieve multi-layer focus of the model on lexical, syntactic and semantic information. the noise-aware generation method and the infilling generation mechanism are also used to optimise the exposure bias issue. Finally, for the text summarisation task, the pre-trained model was fine-tuned using the NLPCC2017 dataset, and and the experimental results verify that the method in this paper is effective.