节点文献

基于改进贝叶斯算法的主题爬虫方法与实现

METHOD AND IMPLEMENTATION OF TOPICAL CRAWLER BASED ON IMPROVED BAYESIAN ALGORITHM

  • 推荐 CAJ下载
  • PDF下载
  • 不支持迅雷等下载工具,请取消加速工具后下载。

【作者】 李光荣薛翔曹远国

【Author】 Li Guangrong;Xue Xiang;Cao Yuanguo;Nanjing Institute of Technology;Institute of Industrial Economy and Innovation Management, Nanjing Institute of Technology;

【机构】 南京工程学院南京工程学院产业经济与创新管理研究院

【摘要】 为了解决主题爬虫中存在主题度判别不足的问题,在PageRank算法和Bayes算法结合的爬行策略方法的基础上,提出一种改进贝叶斯分类算法并融合TextRank算法的主题度判别方法PTB。引用熵值法对朴素贝叶斯分类算法进行加权处理,融合TextRank算法实现关键词提取,再结合链接分析的PageRank算法完成主题度判别模型。通过4种主题爬虫方法进行实验对比,发现PTB方法拥有最优的准确率、召回率、F值,证明该方法提高了主题相关度判别的精度。

【Abstract】 The purpose is to solve the problem of the accuracy of topic crawler in identifying topic relevance. Based on the crawler strategy of PageRank and Bayesian algorithm, an improved Bayesian classification of topic degree discrimination PTB(PageRank combines TextRank and Bayes) method is proposed. Entropy method was used to weight the naive Bayes classification algorithm, and TextRank algorithm was used to achieve keyword extraction. The algorithm and PageRank algorithm were combined into the topic degree discrimination model. The experiment was compared with four themed crawling methods. Research has found that the PTB method has the best accuracy, recall, and F-value. It is proved that this method can improve the accuracy of topic relevancy.

【基金】 教育部人文社科青年基金项目(17YJCZH083);江苏省自然科学基金青年基金项目(BK20220687);南京工程学院产业经济与创新管理研究院开放基金项目(JGKB202203);南京工程学院研究生教育教学改革课题(2022YJYJG13)
  • 【文献出处】 计算机应用与软件 ,Computer Applications and Software , 编辑部邮箱 ,2026年03期
  • 【分类号】TP18;TP393.09
  • 【下载频次】53
节点文献中: 

本文链接的文献网络图示:

本文的引文网络