【摘要】 从商品评论抽取算法出发,使用开放应用编程接口调用和链接跟踪的方法,分别设计了基于通用爬虫原理和开放应用编程接口(OpenAPI)的商品评论聚焦爬虫算法。在此基础上实现了淘宝网和京东网商品评论收集程序。最后将两者与通用爬虫算法进行比较,证实了二者的程序设计具有针对性强、数据采集实时性好、易嵌入开发等优点,为实时评论数据采集技术的研究提供了新思路。更多还原

【Abstract】 From the product reviews extracting algorithm,focused crawler algorithms of product reviews were designed based on the general crawler principle and the OpenAPI respectively,using OpenAPI call and link tracking method.Then the comments′ collection procedure of taobao.com and 360buy.com were realized.Finally both algorithms are compared with the general crawler algorithm,and some merits who are targeted program,timeliness data collection and easily embedded development were demonstrated.The results provide a good idea for the research on real-time reviews data collecting technology.更多还原