【摘要】 研究并实现了一个基于主题的智能信息采集系统IFWC,该系统以全信息理论为支撑,吸收传统向量空间模型的思想,采用基于概念的向量空间模型,从词的语义层次对文本进行主题相关性分析;使用扩展元数据的语义相关性判定算法,对页面内的URL进行主题相关性预测。实验证明,该系统采集速度快,采集下来的页面精度高。更多还原

【Abstract】 An Intelligent Focused Web Crawler(IFWC) is investigated detailedly.Supported by comprehensive information theory,Concept-based VSM is applied to filter the crawled web pages,while extended metadata semantic relevance algorithm for predicting the relativity between URL and topic.The result of experiments has shown that IFWC has more accurate and wider coverage for web pages relevant to a predefined set of topics.更多还原