【作者】 王晶；

【导师】 蒲戈光；

【作者基本信息】 华东师范大学 ， 软件工程， 2009， 硕士

【摘要】 随着互联网的飞速发展,针对于日益膨胀的Web信息,搜索引擎成为人们应用Web的一个信息导航工具。其中,有着行业色彩并且兼有“专、精、深”特点的垂直搜索引擎越来越受到用户的青睐。现在,垂直搜索引擎已经覆盖了很多行业:房产、求职、商务、医疗健康和公交出行等。但是经调查研究发现,关于外汇新闻的垂直搜索技术还没有一个成熟的平台,但是市场的需求是存在的。现在关注外汇市场交易的人越来越多,基于这样的需求,本文即结合垂直搜索引擎技术设计了以外汇新闻为主题的新闻分析引擎系统,用以帮助汇市的用户更加方便的掌握外汇市场方面的讯息。本文应用开源软件Heritrix作为爬虫工具,对国内外著名外汇新闻网站进行信息的抓取,通过对Heritrix的一些组件进行定制,得以对抓取的新闻信息进行搜集和初步过滤。然后对网页内容进行分析,去除文本格式,通过过滤算法,转换为文本文字。进而将这些信息进行进一步的分词,过滤高频词汇和虚词,得到适合于索引的语素单元组合。利用Lucene工具并且结合倒排索引结构进行关键词匹配,采用文档关键字作为索引,实现用户查询接口,为查询请求提供检索服务。当在Server端收到DataType为用户搜索内容的信息后,通过新闻搜索类,进行搜索,将搜索到的数据的关键部分,通过已经建立好的TCP链接,发送给相应用户。经过试验证明,本文设计和实现的外汇新闻分析引擎能够满足用户结构化查询的需求,该系统具有较好的实用价值和应用前景。更多还原

【Abstract】 As internet’s rapidly developing, search engines of increasing web information have been an important tool which guide people use the web. Among these engines, the vertical search engines, which are more vocational and including profession, proficiency and deepness, are more and more popular. Nowadays, the vertical search engines have involved many industries such as real estate, job application, commerce, medical treatment, public traffic etc. The vertical search engines on foreign exchange news doesn’t have a mature technique, however, there will be huge demands on it. The people concerning foreign exchange business are increasing. On the base of this, this paper designed a news analysis engine system on foreign exchange news, which makes use of the vertical search engines’ technique. This system will assist users master the foreign exchange information more expediently.This system got information from domestic and international foreign exchange news website, using the "Heritrix" as crawler tool, collected and filtrate these information across customizing Heritrix’s tools. Analyzing the web page, the system can transform the text formatting to text character using filtration arithmetic. And then, the system sorts this information by filtrating high-frequency vocabulary and empty word, and keeps morpheme compounding applicable to be indexical. Using the Lucene tools and combined of inverted index, the system takes the document key words as index, achieves a interface for users to search and provides service for search requests. If the server receives searching information from Data Type, the system will perform the search among news search class and send the key part to users across the founded TCP link.Tested by experimentations, the foreign exchange news analysis engine designed in this paper can satisfy users’ structural search. Above all, this system has great application value and foreground.更多还原