【作者】 王帆；

【导师】 张学；

【作者基本信息】 电子科技大学 ， 软件工程， 2017， 硕士

【摘要】 随着城市化的高速发展,越来越多的人选择在发达的城市工作和生活。这将对城市的公共管理带来巨大挑战。在规划城市建设时,有效收集市民反馈的城市服务请求信息尤为重要。推特平台作为社交媒体的典型代表,拥有广大的用户群体。市民常常通过推特分享周边的日常事件,这些社交数据将为挖掘城市服务请求提供了可能。然而,推特属于点对面的即时通信平台。该平台在降低信息发布门槛的同时,也提高了虚假信息的传播速度。这导致社交数据价值的急剧降低。虽然平台提供人工过滤虚假信息的途径,但标识这些信息是非常耗时的。因而本文提出了一个自动评估框架完成推文可信度的评估。本文对个人推文进行可信度评估,其主要工作如下:第一,提出了基于KLD(Kullback-Leibler divergence)的推文话题检测方法。推文数据集涉及的话题数不确定导致聚类方法在推文话题检测的应用受到限制。因此,采用基于KLD来计算文本相似度从而完成推文的话题归类。第二,提出了基于用户类、内容类以及单词类的多层特征可信度建模方案。在现有的可信度建模方案基础上,引入单词类特征。具体思路是对文本进行分词,并转换成由多元分词构成的特征向量。通过TF-IDF(term frequency-inverse document frequency)算法计算特征向量的权值,将带有权值的特征向量作为单词类特征。采用随机森林算法建立可信度评估模型,实验表明建立的模型在性能方面有了比较明显的提升。第三,基于建立的模型实现了一个可信度实时评估系统。本系统是智慧城市项目CityFeed的子项目,该系统结合众包思想来提高城市服务水平并且弥补了CityFeed在可信度评估方面的缺陷。为了解决系统在高负载时由于处理不及时导致数据丢失的问题,该系统结合分布式消息队列Kafka和流式计算框架Storm实现了低延时的可信度评估。该系统还提供地图服务,为城市管理人员提供可视化的热点地区分析,从而为城市资源的分配提供了依据。更多还原

【Abstract】 With the rapid development of urbanization,an increasing number of people tend to choose the developed city for their work and life,which presents big challenges to the city’s public management.So,an effective method of collecting feedbacks in terms of urban services given by citizens is greatly essential in the urban planning and urban construction.Twitter regarded as a popular social media has a large group of users.It enables users to publish tweets to share daily events surrounding them.These tweets provide possibilities for distilling urban services request.However,Twitter belongs to the node-to-surface instant messaging platform.Even though twitter provides a channel to upload and propagate information in an easier way,it promotes the rapid propagation of fake information,thus reducing the value of social data.Though the platform provides a way to manually filter fake information,it is extremely time-consuming.So,we proposed an automated framework to assess the credibility of tweet.Our main contributions in tackling the problem of assessing the credibility of individual tweets are as follows.First,the method of topic detection based on KLD(Kullback-Leibler divergence)is proposed.The number of topic related to a tweet dataset is uncertain,which sets the obstacles to apply clustering method in topic detection.A method of tweet classification by calculating text similarity based on KLD is discussed.Second,our approach addresses multi-level features,namely user-level features,content-level features and word-level features,to construct the credibility model.The word-level features are added to mature the existing credibility modeling.Here is the approach: the content of tweet is tokenized and then converted to a feature vector composed of N-gram tokens.The weight of attribute inside the feature vector can be computed by TF-IDF(term frequency-inverse document frequency).And,the weighted vector is regarded as the word-level features.The Random Forest algorithm was adopted to build credibility model.The result of experiments shows a clear improvement in the performance of the built model.Third,a real-time assessment of credibility system is implemented based on the built model.As a sub-project of the smart city project called CityFeed,our system improves the level of urban services by combining crowdsourcing as well as making up the defects of CityFeed in credibility assessment.To solve the problem that the system suffers from a critical problem of data loss when it cannot process the data in time during peak loads,here the system combines Kafka with Apache Storm to ensure acceptable computation timings in the assessment of tweet credibility.In addition,the system provides map services and visual analysis of hot regions for municipality officers,which provides a basis for allocation of urban resources.更多还原