Research on multi-feature fusion algorithm for subject words extraction and summary generation of text

• Published in: Cluster computing Vol. 22; no. Suppl 5; pp. 10883 - 10895
• Main Authors: Xu, Gui-Xian, Yao, Hai-Shen, Wang, Changzhi
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2019; Springer Nature B.V
• Subjects: Algorithms; Artificial intelligence; Asian literature; Chinese literature; Computer Communication Networks; Computer Science; Computers; Data processing; Information retrieval; Information technology; Internet; Keywords; Mathematical analysis; Methods; Natural language processing; Operating Systems; Processor Architectures; Sentences; Statistical analysis; Websites; Words (language); Text sentence weighting algorithm; Multi-feature fusion algorithm; Subject words extraction; Tibetan text information processing; Text summary generation
• ISSN: 1386-7857; 1573-7543
• DOI: 10.1007/s10586-017-1219-3

Subject words represent the brief information of the text. Text automatic summary reflects its theme and core content. In this paper, the research is conducted on multi-feature fusion algorithm on subject words extraction and summary generation of Tibetan network text. Firstly, Tibetan web pages are collected and preprocessing is conducted to extract the useful information from web pages. Secondly, BCCF algorithm of word segmentation is utilized to cut the text’s words. Then multi-feature fusion algorithm is proposed to extract the subject words of the text. The algorithm takes into account the multi-factors such as the word’s frequency, length, type to calculate the words’ weight and effectively select the text’s subject words. For text summary generation, the algorithm of the sentence weight calculation is designed in terms of the word frequency, position and so on. The algorithm of text summary generation is to compute the sentences’ weight, remove the redundant sentences and form the text summary. The experiments show that multi-feature fusion algorithm of the subject words extraction and the summary generation have reached the better achievement. The research is useful and helpful to the study of Tibetan information processing.