【摘要】 近年来自动摘要方面的研究大多是关于多文档和Web网页的,而对网站自动摘要的研究较少。为此,基于主题模型隐含狄利克雷分布(LDA)和网站层次结构提出一个可以自动生成网站摘要的算法。该算法可获取整个网站内的网页信息并进行整合,根据提出的句子权重公式计算句子权重,选取权重最高的句子作为网站摘要。以20个商业和学术网站作为实验对象,使用ROUGE评测标准,结果表明,与仅使用主题模型LDA获取的网站摘要相比,不带停用词的ROUGE-1和ROUGE-L提高0.32,带停用词的ROUGE-1提高0.39,ROUGE-L提高0.38。与网站首页摘要相比,不带停用词的ROUGE-1提高0.03,ROUGE-L提高0.06,带停用词的ROUGE-1提高0.08,ROUGE-L提高0.07。更多还原

【Abstract】 In recent years,the research of automatic summarization is mostly about multi-documents and Web pages,but less about website summarization. A method that summarizes a website automatically based on the hierarchical structure of the website and Latent Dirichlet Allocation is proposed. This method gets the information from web pages in the given website and fuses it,and calculates the weight of sentences according to the proposed sentence weighting formula,and selects the highest weight sentences as the website summarization. An experiment is done based on 20 commercial websites and academic websites,and using ROUGE evaluation. Results showthat compared with the summaries only using LDA,ROUGE-1 and ROUGE-L are increased by 0. 32 with no stop words; ROUGE-1 is increased by 0. 39 and ROUGE-L is increased by 0. 38 with stop words. Compared with the summaries only from homepage,ROUGE-1 is increased by 0. 03 and ROUGE-L is increased by 0. 06 with no stop words; ROUGE-1 is increased by 0. 08 and ROUGEL is increased by 0. 07 with stop words.更多还原