【摘要】 随着新闻网信息的急剧增加,研究面向新闻网的智能抓取技术变得十分必要。在基于主题网络爬虫的基础上,改进了网页与主题的相关度算法,设计出面向新闻网的智能抓取技术。按照预先确定的主题,分析超链接和刚刚抓取的网页内容,获取下一个要爬行的URL。有选择的访问新闻网上的网页和相关的链接,尽可能保证多爬行与主题相关的网页。实验结果表明,改进后的抓取技术比以往的新闻网抓取技术在性能上有较大提升,抓取准确率显著提高。更多还原

【Abstract】 With the rapid increase of network information,it becomes necessary to research intelligent fetching technology for news.In this paper,on the basis of topic web crawler,it improved web page with the theme of correlation algorithm,designed news oriented intelligent fetching.According to the predetermined theme,it should be analyzed hyperlinks and grab web content to obtain the next crawling URL.Web pages online and related links are selective to access,the related subject web pages are crawled as much as possible.The experiment results show that the improved technology was highly promoted in the performance and the grab significantly accuracy compared to the previous technology.更多还原