【作者】 张蕾；

【导师】 魏芳；

【作者基本信息】 北京邮电大学 ， 电子与通信工程， 2017， 硕士

【摘要】 随着以智能手机和平板电脑为代表的移动终端设备的不断普及和更新,以及移动网络相关软件、硬件的飞速发展,互联网已经从传统的桌面互联网时代进入到了移动互联网时代。伴随着移动互联网时代的来临,人们许多相关行为或者使用的工具与桌面互联网时代相比都发生了巨大的变化。近几年,异常火爆的移动应用程序APP就是移动互联网时代的产物。在信息化爆炸的今天,每天都有大量的APP上线,同时有大量的APP被淘汰。对于APP投资方,一次错误的投资或决策都将会造成巨大的损失,所以就需要提前掌握APP的使用模式,其可以作为该APP的欢迎度等情况的一个重要维度。因此,在这个信息化时代需要善于从大数据中提取到想要的信息来方便做出正确的决策。但是,要想从海量的网络访问数据中提取出APP的使用模式,首先需要获得这些APP在下载或使用等情况下所具有的特征,即客户端与服务器端进行通信时产生的流量规则。本论文旨在研究如何自动化抓取APP下载的网络流量并自动化提取其流量规则。首先对移动APP的相关课题背景进行研究,提出了论文主要内容和研究目标。其次介绍一些Wireshark抓包的相关理论知识,并在理论知识的基础上介绍手动抓取APP下载和使用情况下的网络流量,以及流量规则文件的生成和后续使用。当APP数量不多时,可以人工抓取APP的网络流量。但是,当APP数量非常多时,就需要考虑自动化的方法,节约人力和时间成本。再次,介绍如何实现移动APP下载操作的自动化,即编写自动化测试框架Robotium程序模拟人的手势操作实现APP在安卓应用市场中的自动化下载,包括应用商店安装包重签名,Android控件定位方式等。接着利用Tcpdump命令行工具实现APP下载流量的抓包,然后编写Python多线程程序将两者结合实现移动APP下载流量抓取自动化。最后,通过编写Python程序自动化提取流量规则,缩小人为找寻流量规则的范围,减小工作量。通过提取APP在下载或使用等情况下产生的流量规则,进而生成规则文件,随后在海量的网络访问数据中利用这些规则文件,编写MapReduce程序挖掘出哪些用户曾经下载或者使用过该APP,从而分析得到该APP的欢迎度,最终实现APP的使用模式研究。更多还原

【Abstract】 With the smart phones and tablet PCs as the representative of the growing popularity of mobile devices, and the development of mobile network related software and hardware, the Internet has been changed from the traditional desktop Internet era into the mobile Internet era. With the advent of the mobile Internet era, many related behaviors or tools have undergone tremendous changes. In recent years, the unusually hot mobile apps are one of the results of the mobile Internet era.In the information age, a large number of APPs were released, while a large number of APPs were eliminated. For APP investors, a wrong investment or decision-making will result in huge losses. So we need to grasp the usage pattern of the APP, which can be used as an important dimension of the APP’s popularity. Therefore, in this information age, we should be good at extracting the desired information from the big data, and make the right decisions based on the information. But, in order to extract the desired information from the massive network access data and obtain the APP mode of using, first of all, we need to get these characteristics of the APPs when they are downloaded or used, by generating traffic rules when the client and the server communicate.The purpose of this paper is to study how to automatically fetch the network traffic of APP’s downloading operation and extract the traffic rules automatically. Firstly, the background of mobile APP is studied, and the main contents and research objectives are proposed. Secondly, this paper introduces some theoretical knowledge about Wireshark capturing packet and introduce the network traffic under the situation of APP’s downloading and using, as well as the generation and follow-up of the traffic rules file based on the theoretical knowledge. Since there is only a small number of APPs, we can manually capture the APP network traffic. However, when the number of APPs is increasing, we need to consider the method of automation for saving manpower and time costs.Once again, the paper introduces how to achieve the automation of mobile APPs downloading operation, that is, the preparation of automated testing framework Robotium to simulate human gestures to achieve automation-download of APPs in the Android application market,including application store installation package re-signature, Android control positioning methods. And then use the Tcpdump command-line tool to achieve the traffic of APP downloading, and then write Python multi-threaded program with combining the above two to achieve the traffic capture automation of mobile APP downloading. Finally, we automate the extraction of traffic rules by writing Python programs to reduce the scope of human search for traffic rules and reduce the workload.At the end, the rules file is generated by downloading and using the APP. And then, we can get the welcome of the APP, and ultimately the usage pattern of APP by using the rules file in the massive network access data.更多还原