节点文献
基于Elasticsearch的商业变现平台的设计与实现
Design and Implementation of Commercial Cash Platform Based on Elasticsearch
【作者】 刘涛;
【导师】 冀振燕;
【作者基本信息】 北京交通大学 , 软件工程(专业学位), 2021, 硕士
【摘要】 互联网的发展以及5G的普及,使得短视频成为了DAU(Daily active user)增速最快的细分领域之一,各大短视频平台的竞争关系日益加剧。随着短视频人口红利的触顶,以及流量瓜分的完成,剩下的竞争则围绕着“留量”和“商业变现效率”两个方面。广告主愈发增长的推广需求,以及达人迫切的变现需求,使得商业流量变现系统应运而生,此系统致力于打造达人生态营销平台,通过直播或商业短视频的形式,帮助用户实现直播带货、App下载、品牌营销、表单收集等目标,从而撮合客户营销需求与达人变现诉求。在商业变现平台的初步构建阶段,首先进行了技术选型,使用Elastic Search引擎对达人数据及热榜完成近乎实时的存储、检索;考虑到直播场景中的高并发,故选用Redis来降低短时间内的并发量;而为了便于消息之间的异步通信及日志的处理,使用Kafka实现复杂的日志处理和服务间的异步调用通知。基于平台自身的优势,得出系统完备的功能点,根据功能点的分类,通过用例图给出模块的功能划分。接着又从易用性、安全性、可靠性、健壮性等性能方面对非功能性需求做出了要求。为了应对不同用户对系统功能的需求,将系统分为任务管理、达人广场、数据展示、达人智能推荐、用户信息管理、基础服务六大模块,并通过流程图、功能模块图从不同的角度对系统功能做了详细的介绍。通过比较与传统推荐算法的优劣,智能推荐模块采用了基于达人的协同过滤模型做召回,feature-based模型做精排的方案,但由于内积函数限制了用户-物品交互关系的表现力,故又采用了Conv MF(convolution matching function)模型,它通过卷积神经网络(CNN)充分挖掘出用户和物品之间的交互特征,之后排序阶段用更加精细的特征和复杂的wide&deep类模型来进行精排,通过如此设计帮助客户更高程度地精确找到相关领域内的达人,实现了广告主推广、达人盈利、平台发展的目标;达人流量助推使用粉丝头条算法,通过多标签定向,帮助用户获取高价值曝光和粉丝,满足有急切推广需求的达人;在详细设计阶段,通过类图描述了类的属性,类之间的关系,用时序图明确了不同服务之间的交互工作,并通过流程图形象化描述了数据流的执行过程。测试阶段给出了测试使用的环境、工具以及测试内容,并通过截图展示了测试的成果。通过平台功能的优化升级,撮合客户营销需求与达人变现诉求,加之平台活跃的社区氛围,激发出了平台强大的商业转化力。目前本文设计的项目已经成功上线,成为公司连接广告主推广需求和达人变现需求的桥梁,上线后增加将近50%的DAU,为企业在商业化的发展道路中做出了突出贡献。
【Abstract】 With the development of Internet and the popularity of 5G,short video has become one of the fastest growing segments of DAU,and the competition among major short video platforms is becoming increasingly fierce.With the peak of short video demographic dividend,and the division of traffic has finished,there is a competition around "reserve" and "commercial cash efficiency".Based on the influx of traffic and the urgent demand for cash flow,commercial cash flow system emerges as the times require.It is committed to building a talent ecological marketing platform to match customers’ marketing needs and talent cash flow demands.Through live or commercial short video,it helps users achieve the goals of live delivery,App download,brand marketing,form collection,etc.In the initial construction stage of the commercial realization platform,the technology selection is carried out firstly,the elasticsearch engine is used to store and retrieve the talent data and hot list in near real time;Considering the high concurrency in the live scene,redis is selected to reduce the concurrency in a short time;In order to facilitate asynchronous communication between messages and log processing,Kafka is used to realize complex log processing and asynchronous call notification between services.Based on the advantages of the platform,the complete function points of the system are obtained.According to the classification of the function points,the function division of the module is given through the use case diagram.Then the non functional requirements are required from the aspects of ease of use,security,reliability and robustness.In order to meet the needs of different users for system functions,the system is divided into six modules: task management,talent square,data display,talent intelligent recommendation,user information management and basic services.The system functions are introduced in detail from different perspectives through flow chart and function module diagram.Compared with the traditional recommendation algorithm,the intelligent recommendation module adopts the collaborative filtering model based on talents for recall and the feature-based model for fine arrangement.However,because the inner product function limits the expressiveness of the user article interaction,it also the conv MF(convolution matching function)model are adopted,it uses convolution neural network(CNN)to fully excavate the interaction characteristics between users and items,and then uses more refined features and complex wide&deep class model for fine arrangement in the sorting stage.Through this design,it helps customers find the talents in related fields more accurately,and achieves the goals of advertisers’ promotion,talents’ profit and platform development;Through multi label orientation,talent traffic boosts the use of fan headline algorithm to help users get high-value exposure and fans,to meet the urgent promotion needs of talent;In the detailed design stage,the attributes of the class and the relationship between the classes are described by the class diagram,the interaction between different services is defined by the sequence diagram,and the execution process of the data flow is described by the flow graph.In the test phase,the environment,tools and test content are given,and the test results are shown through the screenshot.Through the optimization and upgrading of the platform functions,matching the marketing needs of customers and the realization demands of talents,coupled with the active community atmosphere of the platform,the strong commercial transformation power of the platform has been stimulated.At present,the project designed in this paper has been put online,and has become a bridge between the company’s promotion needs and the realization needs of advertisers.After going online,the DAU has increased by nearly 50%,which made outstanding contributions to the development of commercialization.
【Key words】 Commercial Cash Platform; Elasticsearch; Redis; Intelligent Recommendation; Traffic Boost;
- 【网络出版投稿人】 北京交通大学 【网络出版年期】2022年 03期
- 【分类号】TP311.52
- 【被引频次】2
- 【下载频次】183