【作者】 郭锐；

【导师】 吕英杰； 牛宏斌；

【作者基本信息】 北京化工大学 ， 工程管理（专业学位）， 2017， 硕士

【摘要】 随着电商平台的蓬勃发展,用户数量逐渐增多,使得商品评价数目飙升。如何能高效、精确获得基于这些语料的相关信息,在此基础上进行分析研究逐渐成为了当前信息科学和技术领域中面临到的重大挑战。电商网站因流量巨大,可为各种类别的产品提供交易平台,而这其中的电子类产品是在电商平台上非常热销的典型产品之一。尤其是随着物联网的普及带来了穿戴智能、智慧家装、健康医疗等一系列智能设备,智能手环是其中典型代表。本文即采用京东商城智能手环类产品用户评论作为数据集,充分结合机器学习及自然语言处理相关理论和方法,对文本语料进行了文本情感倾向性的建模、文本主题分析研究,从而提出高效、精确的文本挖掘技术和方法。本文所研究的技术成果可广泛用于各类电商门户、点评网站及线上交易平台。本文从文本挖掘的具体流程,包括语料数据采集、文本预处理、语义分析、情感分类、主题分析等为文本挖掘分析提供相关的技术指导,并着重介绍了数据获取、自然语言处理、算法选择与情感分类比较等多种技巧。首先通过python采集京东商城手环类产品评论文本语料数据,得到近20万条用户评论文本。然后对文本预料进行数据预处理,主要是文本语句间去重:删除行与行之间完全重复的内容,只保留唯一的一条评论;文本语句内去重:用多重遍历的方式对有单字、多字重复,从句间重复等进行筛查;后续处理和低含义词语删除:通过设置排查筛查的长度,剔除掉低于预设值的评论内容。特征选取部分,主要是采用TF-IDF值作为特征选择的方式,计算每个词的TF-IDF值,将其和设定的阈值进行比较,低于阈值的词过滤掉,最后剩余的词作为特征项,并且得到的特征项对应的权重值即为其所对应的TF-IDF值。语义分析,主要是将用户的评论中,出现频率最高的词语找出来,然后分析这些词语的含义或者关系,从得出用户对产品的重要评价或印象。情感分析,本文使用python的自然语言处理包NLTK及包含分类算法的工具包sklearn进行分类模型的训练。根据卡方统计量进行特征值选择,并把语料文本使用特征表示。用不同的分类算法构建分类器,并测试其准确度。LDA主题模型分析,主要是使用了Python中的Gensim包,把评论分词为词包,然后分配生成词典,建立语料库,把文本转换为稀疏向量,指定主题数量,进行LDA模型的学习和训练,最终形成多个正负主题,可以直观的了解到商品得到的好评究竟好在哪些特点,得到的差评究竟问题在哪些部分。在对模型设计环节完成之后,进行了数据实验验证,比较多个模型的准确率、召回率,结论表明,模型预测精度满足实际使用要求。更多还原

【Abstract】 With the vigorous development of the electronic business platform,the users are becoming more and more.This makes the number of the commodity assessment increase very quickly.How to get the relative information based on these meterials efficiently and precisely and study and analysis on this is facing a huge challege in the mordern information science and technology area.Electronic business platform can be a trade place including all kinds of commodities and the electronic.Especially with the universal Internet of things which brings a set of intelligent equipment such as wearing intelligent,wisdom home improvement and health care and so on,smart bracelet is one of the typical representatives.This paper uses the number of the assessment of users in the Jingdong mall products like the smart bracelet product and combines the machine learning and the relative theory and methods of the natural language processing.It also analyses and studies on language materials with the modeling emotional tendency of text and the theme of the text so as to propose more efficient and accurate technology and methods of the text digging.The techniques described in this paper can be largely used in electronic.This article provides technical guidance for text mining from the specific process of text mining,including data acquisition,text preprocessing,semantic analysis,sentiment classification,topic analysis and emphatically introduces the data acquisition,Natural Language Processing,algorithm selection and emotion classification etc.This paper first acquires comment corpus data of bracelet product in Jingdong mall through python,and obtains nearly 200 thousand user views text.Then it preprocesses data,eliminates the same content between lines,retaining only a comment statement to the text.It screens single word repetition,repetition of multiple words and repetition between clauses with the multiple traversal of words.The subsequent processing and deletion of low words:It sets screening length to delete reviews that are less than a preset value.Feature selection is mainly with the TF-IDF value as the feature selection,calculates TF-IDF value of the word,and compares the threshold with the set threshold to filter out the word below the threshold with the last remaining words as features.Corresponding weight value of features is the corresponding TF-IDF value.Semantic analysis is mainly about finding out the highest frequency words and then analyzing the meaning or relationship of these words to obtain the user evaluation or impression on the product from the user’s important comments.As for sentiment analysis,this paper uses Python’s NLTK,the Natural Language Processing and sklearn toolkit that includes classification algorithm to implement the training of classification model.It selects the feature value according to Chi-square statistics,and represents the corpus text in features.It constructs classifiers using different classification algorithms,and tests the accuracy.LDA model analysis mainly uses the Gensim package of Python.It divides the comment into word package,then generates dictionary,establishes corpus to turn text into sparse vector and specifies the number of topics to carry on LDA model of learning and training,and ultimately forms multiple positive and negative themes.From that we can intuitively know what aspects of goods to praise and what aspects of goods to criticize.After the completion of the model design,it continues experimental verification to compare accuracy,recall rate of different model.Results show that the prediction accuracy of the model meets the actual requirements.更多还原