【作者】 徐晓峰；

【导师】 过敏意；

【作者基本信息】 上海交通大学 ， 计算机科学与技术， 2013， 硕士

【摘要】 最近十年来全世界网民的数量呈现快速增长的趋势，截止2011年1月全世界网民的数量已经超过了20亿。在线用户规模的急剧扩大使得用户在线行为分析在很多研究领域的意义显得越来越重大，如在经济、学术以及社会事务等领域。其中，最具有商业价值要数用户在线商业意图（Online CommercialIntention）的挖掘。通过分析用户的在线行为来挖掘出用户个人兴趣及倾向，为有针对性的广告投放提供有力的依据，从而达到更好的广告推荐效果。文中提出了一种检测用户商业意图的新方法架构，通过分析用户浏览过的网页信息，从中挖掘出用户个性化的购买倾向。借助于大型购物平台上搜索引擎的历史记录以及搜索结果页面上商品的点击分布情况，通过统计分析建立起搜索词与商品类目之间的关联——关键词词典。利用这一词典我们可以从用户浏览过的网页内容中抽取出一系列能够体现用户商业意图的关键词，关键词选取的标准包括关键词的TF、ICF、QF以及关键词本身的长度。每一个关键词根据关键词词典将被映射到一系列的商品类目，其中分值最高的几个商品类目将视为页面的商业意图检测结果。为了消除一些不相关类目对于用户商业意图结果的影响并使得OCI结果更加集中，类目的相似度模型被引入来修正OCI结果中的分值。用户每天的商业意图通过聚合其当天浏览过页面的OCI结果得到。为了体现出商业意图短暂、灵活多变的特点，我们提出了用户时间维度上的商业意图模型来更真实地体现出用户的实时商业意图变化。为了评估所提出OCI检测方法的效果，在实验部分，通过制定统一的评测标准，评测人员对于随机抽取的1036个测试页面的商业意图结果进行了人工标注。从标注结果与系统检测结果的对比中可知，在检测商业意图存在的角度，系统判断的准确率达到了70%，召回率达到了将近90%；而从系统匹商业类目的角度来看，系统匹配商业类目的正确率达到了86%。通过后续的实验我们还发现，通过限制页面关键词的数量以及设定匹配类目最小阙值等方法可以有效地提升系统检测商业意图的整体效果。作为最终的分析结果，系统以一系列商品类目的形式给出了用户在一段时间内的商业意图。根据商品类目在一段时间内的分值变化情况，可以推测出用户在这一段时间内购买某一类商品的倾向及对应程度。通过构建合适的用户购买行为模型，可以对目标用户进行有针对性的商品推荐。借助于大规模分布式计算架构，文中所提到的OCI检测系统流程可以应用于大规模用户行为数据的分析。正在运营的系统每天可以处理十亿级别的用户浏览记录，用来分析得到几千万用户的具体商业意图，每个小时的数据处理量大约在150GB。更多还原

【Abstract】 Within the recent ten years, the scale of internet users shows a rapidly expandingtendency. UntilJan.2011thewholenumberofinternetusersintheworldhadexceeded2billion. Since more and more people are surfng on the internet, it makes big sense toanalyze user online behaviour in many research felds, such as economic, academy andsocial afairs, etc. Among them, the most proftable application is the detection of useronline commercial intention (OCI). Through efective user online behaviour analysis,personal interest and tendency can be detected to provide strong evidence for targetedadvertising, thus receiving better recommendation efect.In this thesis we propose a new framework for detecting user online commercialintention based on user surfng log to dig out user’s personalized shopping tenden-cy. Based on the analysis of click distribution on query result pages on a large onlineshopping platform, we set up a”keyword dictionary”, which reveals the relationshipbetween queries and product categories. With this dictionary, frstly we extract key-words from the pages, which are estimated using parameters like TF, ICF, QF andlength of the keyword. Each extracted keyword is mapped to a commercial categoryhistogram and the top scored categories are used to represent page OCI. In order toeliminate the noise of some unrelated categories and keep the OCI result intensive, acategory similarity model is provided to revise the category scores obtained from eachpage. Since user’s commercial intention is transient and varying, we attempt to calcu-late user’s current OCI by merging the history OCI result and today’s analysis resultwith three diferent strategies.To evaluate the detection efect of proposed framework, in chapter3human la-bellers are required to evaluate the OCI result of1036sampled pages according toestablished labelling regulation. From the labelling result, we can see that the detec-tion precision of page OCI existence reaches70%while the recall rate reaches90%. From the view of matched category, about86%from the pages that are correctly clas-sifed as commercial can be matched with correct categories. In later experiment wealso fnd that through setting a threshold of keyword number in the page and categoryscore, the general detection efect can be well improved.As the fnal analysis result, a list of product categories will be outputted to repre-sent one user’s commercial intention. Through analyzing user’s OCI fuctuation withina period we can predict his shopping tendency for some products. Based on proper us-er shopping behaviour model, suitable products can be recommended to the targetedusers.Depending on the computing capability of large-scale distributed systems, theproposed framework for user OCI detection can be realized in parallel, thus enablinglarge-scale user online behaviour analysis. In the current running system, billion-leveluser browsing log will be processed per day to serve OCI detection of tens of millionusers. About150GB page data will be processed per hour.更多还原