【摘要】 提出了一种基于网页框架和规则的网页去除噪音的新方法,该方法根据网页中HTML标签<table>将网页分成若干部分,对各个table的长宽比属性进行比较,去掉长宽比很大的部分,并对其余table中的内容进行分析,根据内部是否存在和段落文字有关的标签<p>或<br>等来区分主题内容和噪音内容,在此基础上去除噪音内容。对来自CWT200G语料的132 559个网页进行测试后的结果表明,该方法可以有效地去除网页噪音,使索引文件减少约75%,大大地提高了检索速度,准确度也得到一定提高。更多还原

【Abstract】 This paper presents an approach to eliminate noise based on framework of Web pages and rules.This approach divides a page into several parts according to HTML tag <table> in a Web page,then compares the ratio of width and height attributes of every table and deletes the part of bigger ratio.To the rest tables,topic and noise content are differentiated according to tag <p> or <br> related to paragraph,the noise content is eliminated based on this way.Experiments performed on a set of 132 559 Web pages from CWT200G show that this approach can eliminate noise content of Web pages effectively and decrease the size of index files to about 75%.The information retrieval speed can be faster,and the accuracy of retrieval can be improved.更多还原