【作者】 王敏；

【导师】 韩国强； 谢德才；

【作者基本信息】 华南理工大学 ， 工程硕士（专业学位）， 2016， 硕士

【摘要】 中国电信企业使命是让客户尽情享受信息新生活,战略目标是要想成为世界级的综合信息服务提供商。这样的使命和目标的背后是由一个个的系统来支撑的,而这样的系统的背后,隐藏着的是——数据。大量的客户,长年的积累,使各个支撑系统均出现海量的数据,比如每个客户的话单,基础资料信息,程序日志,各类的数据库表等等,他们具有庞大的数据量,具有巨大的价值,是实现企业使命和目标的最基本元素。故,数据存储和使用,成为一个重要的战略支撑点。综合计费结算系统作为一个最基础最核心的系统,其数据的存储和管理直接影响用户感知,也就直接影响广东电信的发展。而随着电信业务这几年间的快速发展,4G时代的到来,大量的语音,短信,上网数据,大量的日志和数据库表的记录数出现了爆炸式增长。并且随着业务复杂度的增加,数据结构也越来越复杂,数据量越来越大,导致数据存储和处理的难度加大,而目前的IOE模式已经无法支撑巨大的数据存储量,紧急情况下只能减少数据在线保存时间,将数据备份到磁带。但是这个方式又存在恢复周期长且无法进行数据分析等缺点,不满足现有需求,影响数据分析和故障的核查,进而影响客户感知和公司战略的实现。因此,亟需一种新的数据存储和管理方式,来缓解数据日益增长的需求。本文从实际情况出发,对电信系统的数据特点和数据清理需求进行分析,从总体上考虑数据的清理方案。主要做了三部分工作:一:分析数据特点,并进行归类。二:针对分类结果,对大部分常规数据设计一个数据清理系统。三:针对具有特殊特点的数据,设计一个特殊的清理备份方案。其中核心为设计一个“IT数据清理备份系统”,主要研究的是IT备份查询系统的需求分析、设计和实现,从体系结构、技术应用、数据库设计、与其他系统交互和系统分析设计等方面进行考虑,建立一个能够存储海量数据,并可实时提供查询和清理功能的工具。在功能体系方面,“IT数据清理备份系统”采用MVC模式进行设计,技术上采用BS结构编程技术[7];浏览器端编程语言采用包括HTML、CSS、JavaScript语言;在服务端编程技术主要采用JSP/Servlet;数据库设计采用JDBC链接MySQL数据库;与云平台的服务调用采用基于rest协议和XML文档传输数据的Web Services技术,依托于Hadoop架构的云平台HDFS文件系统的HBase数据库做数据存储,开发相应的接口供“IT数据清理备份系统”调用;在业务体系方面,主要是描述了根据实际应用需求;将功能进行模块分类,完成了包括六大模块的设计,主要有:场景一,一键健康检查;场景二,清理垃圾;场景三,健康检查;场景四,查找大数据;场景五,申诉查询;场景六,云查询。更多还原

【Abstract】 China Telecom enterprise aims to allow the customers to enjoy a wonderful life with new information technology.And its strategic goal is to become a world-class integrated information service provider.Either this mission or this goal is insisted by many systems.And behind the systems,there is something hidden--data.On account of years of accumulation,these systems have to support a large quantity of data which is produced by massive customers.The huge amount of various data includes each customer’s bills,basic information,the application logs and so on,all of which are generated ceaselessly and at the same time of great value.Data is the most basic element of achieving the corporate missions and goals.Therefore,data storage has become an important strategic fulcrum.Integrated billing system,as one of the most important systems of MBOSS,of which the data storage and management directly affect the users’ perception,and will directly influence the development of Guangdong Telecom.With the rapid development of telecommunication business in the past few years,the advent of the era of 4G,a lot of voice,SMS,Internet data,a wide range of logs and database tables appeared and grew explosively.Besides,with the increase in the complexity of business,there are more and more complex data structures,and the data volume is becoming lager and lager,leading to great difficulties in data storing and processing.Consequently,the Oracle Database is unable to support the storage of huge data.In case of an emergency,we can only reduce online data which is stored in Oracle Database.What’s worse,there will be a long recovery period if we use tapes to backup the data that can’t be analyzed.Without doubt,this approach,on the one hand,can not meet the present requirements;on the other hand,it has a bad effect on the data analysis and fault verification.And worse still,it will affect the customer awareness and implementation of company strategy.Therefore,we are in urgent need of a new way of data storage and management,to relieve the growing of data.Based on the actual situation,this paper analyzes the data characteristics and data cleaning requirements of the telecommunication system and then designs the data cleaning backup scheme which is taken into consideration generally.There are three parts of the work: Part one: analyze data characteristics and classify data.Part two: depending on the classification results,a data cleaning system is designed for the majority of conventional data.Part three: in view of the special characteristics of the data,design a special clean-up backup program.The core for the paper is the designation of the IT Data Cleaning Backup System which mainly studies analysis,design,and implementation of the system,trying to establish a system in which huge amounts of data considered from aspects of architecture,technology application,database design,interaction with other systems,system analysis and design can be stored.And the most important is that it can provide real-time query and cleaning function of the system.The MVC pattern is adopted to improve the design of IT Data Cleaning Backup System.Technologies,such as the BS structure programming techniques,the browser programming language including HTML,CSS,JavaScript,are used in the design.And we adopt JSP/Servlet as the server-side programming technology.We use JDBC to link the MySQL Database and Web Services technology to invocate cloud services using rest protocol and XML for data transmission.At the same time,we store data in HBase.In the aspects of systems analysis,the paper mainly described actual application requirements and the function module classification,including: Scene 1,a key to health examination;Scene 2,rubbish cleaning;Scene 3,health assessment;Scene 4,in search of big data;Scene 5,enquiry about complaint;Scene 6,query.更多还原