【作者】 张力；

【导师】 叶鹰；

【作者基本信息】 浙江大学 ， 信息资源管理， 2014， 博士

【摘要】 互联网络的发展及信息化工程的推进,促使Web信息逐步累积成为一个能够提供信息交互、信息共享,并影响人类生活各个层面的巨大资源空间。为了从具有海量性、无序性、异构性、实时更新性、多样性等特征的Web信息中快速、准确地获取所需资源,人们开始逐渐认识到Web信息组织管理的重要性,并开始探索各种Web信息处理方法,自动标引即为其中之一。本研究以自动提取Web信息标引词为切入点,以Web坐标系、Web页面组织结构和Web页面浏览者的阅读习惯等特点为研究对象,探索Web信息自动标引过程中的具体影响因素。在总结前人研究工作的基础上,提出设想：根据网页坐标系,按照不同站点类型,用不同分割比例把网页划分若干区域；判析Web信息块归属区域并针对网站类型,探索各区域信息块在自动标引过程中的权重,最后编写程序验证以上设想,完成自动标引各个环节。具体步骤如下：(1)研究实现Web页面采集。根据研究需要,分别实现Web页面批量采集和手动采集,解决Web页面采集过程中的页面编码转换、html转换xml等问题。(2)利用Web页面坐标系,结合页面浏览者阅读习惯,将Web页面划分成9个区域。每个区域占据页面一定比例,且区域中信息块被视为一个信息块集群,在后期运算中具有同样的标引权重并被统一处理。(3)寻找发现不同类型网站的适宜页面分割比例。不同类型网站有着自己独特的页面信息发布方式。如新闻类站点,往往图片较少,文字报道占主要部分；大部分新闻类站点都向页面浏览者提供对某新闻进行评价的功能,从而造成网页高度变动幅度较大。本文分别选择新闻类、体育类、科学类站点页面,用不同页面分割比例进行测试,找出各类型站点的适宜页面分割比例值。(4)摸索不同区域信息块在自动标引过程中的权重。浏览者在访问Web页面时,总会有视觉焦点、阅读习惯等特性,从而Web页面设计者在制作网页时,也会有所重点地安排Web页面信息。因此能否发现不同Web页面区域的信息重要程度,对后期自动标引结果的准确性有着直接影响。本文通过样本实验,对新闻类、科学类站点网页的不同区域信息块重要性进行了摸索,并分别得出不同类型站点的Web页面区域信息块在自动标引中的权重。(5)实现对Web页面进行自动标引。在考虑Web页面信息噪音和区域特性的基础上,结合文本方法特色,给出一种Web信息自动标引的方法,编写程序予以实现和验证。此外,本文还分别对网页宽度、网页高度与不同页面分割比例下的信息抽取查全率、准确率等的相关性等问题进行了探讨,以期对以后该领域研究有所帮助。综上所述,本文对Web信息自动标引过程中各环节的关键技术进行了探索,探讨了不同类型站点网页的适宜分割比例,研究了网页坐标系与Web信息自动标引过程的相互关系,对相关研究有着借鉴和参考意义。更多还原

【Abstract】 With the development of internet and the promotion of information project, Web is becoming a great resource space, which provides a way to exchange or share information for us and has a profound influence on all areas of our life. In order to search for necessary resources from the Web information, which is massive, isomers, disordered and updating with time, people begin to realize the importance of Web information management:such as auto-indexing for Web information.The search sets auto-indexing for web information as the entry point and web page coordinate system, organization of web page, reading habit of web page browsing, as the object to explore the factors that have influence with the result of indexing web page.Based on literature review and summarizing previous work, the author sets out the research hypothesis:according to the web page coordinate system and the type of website, the suitable dissection ratios of web page will be used to divide the web page into several regions;With judging which region the information block of web page belongs to, we find the importance of different web page regions in auto-indexing, and write a program to realize the hypothesis.The detail of above works will be described as following:First, study and realization on web page gathering. we archive the function of batch collection and manual collection, solve the problem of web page code conversion and html to xml conversion in the process of collecting web page.are described as following:Second, based on the web page coordinate system and reading habit of web page browsing, we divide the web page into nine regions with suitable dissection ratios of web page.The information block in the same region are regarded as a information cluster and processed with the same auto-indexing weight.Third, searching for suitable dissection ratios of web page for types of websites.Different website has different methods of information publication, such as news website, most news are reported by text including few photos, some website allows readers to publish their comments on news at web page, so the height of web page changes relatively greatly.we test news website, science website, sports website with different dissection ratio to get the most suitable ratio.Fourth, searching for the auto-indexing weight of regions.When people visits a web page, there are some characteristics, such as visual focus, reading habit etc, have an effect on people’s reading action, so web page design should be the key to organize information for web page maker. If we find the importance of different web page regions, the accurateness of web page auto-indexing will be improved, so the sample tests for news website, science website, sport website are made to get the auto-indexing weight of different regions in web page.Last, write a program to test and verify the author’s assumption.Experiments of web page auto-indexing were done for types of websites, with considering of information noises and regional characteristics of web page etc, and the results is good.At the completion of the work, we argue about the association of page height, page width with recall rate, precision rate and hope the result is useful for later research plan.Summarizing the above work, we feel about the key technology in every link of web page auto-indexing, such as suitable dissection ratios of web page for types of websites, the association of web page coordinate system and auto-indexing, complete the whole work flow of web page auto-indexing. Those researches will be useful and instructive to the web information management. Certainly, the research has many deficiencies that need to be optimized in future.更多还原