【作者】 赖清楠； 金建栋； 周昌令；

【Author】 LAI Qing-nan;JIN Jian-dong;ZHOU Chang-ling;Computer Center,PekingUniversity;

【机构】 北京大学计算中心；

【摘要】 近年来网站安全越来越受到重视,高校主页类网站大多采用网站群纯静态页面的方式,安全防护做得更加严格。但是对网站的内容安全重视程度不够,主页类网站上存在大量指向外部网站的链接,这些链接便成了不可控因素,一旦指向恶意网站,会对网站主体造成恶劣影响。文中提出了一种基于主题网络爬虫的方法用于分析网站内容安全,首先采用通用网络爬虫爬取网站上所有链接,然后对外部链接采用主题网络爬虫进行内容爬取,通过关键词匹配的方法判断是否为恶意网站,在接收到反馈后,爬虫对关键词权重进行调整,以便提高主题识别的准确率。对某高校主页网站进行测试,对比了爬取源码和Selenium浏览器模拟两种方式,并利用主题网络爬虫进行检测,发现存在53个无法访问的链接以及2个恶意链接。更多还原

【Abstract】 In recent years,more and more attention has been paid to website security.Most university homepage websites use purely static pages and security protection has become stricter.But insufficient attention has been paid to website content security.There are a lot of links to external websites on homepage websites,which become uncontrollable factors.Once pointing to malicious websites,they will have a bad impact on the websites.This paper proposes a method based on topical crawler to analyze website content security.First,we use a general purposeweb crawler to crawl all links on the website,and then we use topical crawler to crawl external links,judge whether it is a malicious website by keyword matching,and adjust the keyword weight after receiving feedback,so as to improve the accuracy of topic identification.A test is conducted on the homepage website of a university,compared to two ways of crawling source code and Selenium browser simulation,and use topical crawler to detect,the results show that there are 53 unreachable links and 2 malicious links on the homepage.更多还原