【作者】 杨琳；

【导师】 嵇少林；

【作者基本信息】 山东大学 ， 统计学， 2022， 硕士

【摘要】 近年来,在线广告的蓬勃发展催生了计算广告的兴起,根据点击次数结算已经成为计算广告中使用最频繁的结算方式,而广告点击率预估是其中最重要的操作。因此,准确预估广告点击率进而有针对性地投放广告,已经成为各大企业的重要决策。广告点击率预测问题的数据通常包含大部分高基数的离散特征和少量连续特征,以往的机器学习模型需要人工组合交互特征以挖掘更多信息,深度学习模型使用Embedding技术能够将高维稀疏特征映射成低维稠密特征,通过神经网络自动进行特征交叉实现模型端到端的学习。本文主要研究广告点击率预测问题,基于目前主流深度学习模型DeepFM进行改进,提出了一种融合注意力机制与DeepFM模型的A_DeepFM模型,并进一步提出融合序列模型、注意力机制与DeepFM模型的SA_DeepFM模型,并在两个公开数据集上实验和验证。本文的主要研究工作如下:(1)探究广告点击率预测问题常用的机器学习模型和深度学习模型,研究发现DeepFM模型具有能够同时捕获低阶交叉特征和高阶交叉特征的优势,而且它不需要进行复杂的人工特征工程,可以进行端到端的训练,同时它的FM部分和Deep部分能够共享Embedding向量,模型的结构完整而且预测效果较好。(2)针对DeepFM模型没有考虑不同特征重要性的缺陷,本文提出一种融合注意力机制与DeepFM模型的A_DeepFM模型,使用AFM替换DeepFM中的FM,使用DIN替换DeepFM中的DNN。考虑到用户历史行为是一个随时间动态变化的序列,本文进一步提出融合序列模型、注意力机制与DeepFM模型的SA_DeepFM模型,使用DIEN替换上述A_DeepFM模型中的DIN。(3)选取两个公开数据集Amazon和MovieLens进行实验和验证,对比多个模型在其上的LogLoss和AUC评价指标,比较模型的结构差异和预测性能好坏。实验表明,本文提出的A_DeepFM模型和SA_DeepFM模型的预测效果优于其他模型,而且SA_DeepFM模型的预测效果更好,验证了引入序列模型和注意力机制对于模型预测效果的提升。最后对最优模型SA_DeepFM中的多个超参数进行优化分析,得到了最优参数组合,进一步提升了模型的预测效果。更多还原

【Abstract】 In recent years,the vigorous development of online advertising has given birth to the rise of computational advertising.Cost Per Click has become the most frequently used settlement method in computational advertising,among which the advertising click-through rate prediction is the most important operation.Therefore,it has become an important decision for large enterprises to accurately predict the click-through rate of advertisements and then put advertisements in a targeted way.The data of the advertisement click-through rate prediction usually contains most of the discrete features with high cardinality and a small number of continuous features.The previous machine learning models need manual feature engineering to combine interactive features to mine more information.Deep learning models can map high dimensional sparse features into low dimensional dense features by Embedding technology.The neural network is used to automatically cross features to realize end-to-end model learning.This thesis mainly studies the prediction of advertising click-through rate.Based on the current mainstream DeepFM model,it proposes a A_DeepFM model that integrates attention mechanism and DeepFM model,and further proposes a SA_DeepFM model that integrates sequence model,attention mechanism and DeepFM model.Experiments and validation were performed on two public datasets.The main research work of this thesis are as follows:(1)Explore the machine learning models and deep learning models commonly used for advertising click-through rate prediction.This thesis finds that DeepFM model has the advantage of capturing both low-order cross features and high-order cross features,and it does not need to carry out complex artificial feature engineering,but can carry out end-to-end training.At the same time,FM part and Deep part can share Embedding vector,the structure of the model is complete and the prediction effect is good.(2)Aiming at the defect that DeepFM model does not consider the importance of different features,this thesis proposes an A_DeepFM model that integrates the attention mechanism and the DeepFM model,replace FM in DeepFM with AFM,and replace DNN in DeepFM with DIN.Considering that the user’s historical behavior is a sequence that changes dynamically over time,we propose the SA_DeepFM model that integrates the sequence model,the attention mechanism and the DeepFM model,and uses DIEN to replace the DIN in the above A_DeepFM model.(3)Select two public datasets,Amazon and MovieLens,for experiment and verification,and compare the LogLoss and AUC evaluation indicators of multiple models,and compare the structural differences and prediction performance of the models.Experiments show that the prediction effect of A_DeepFM model and SA_DeepFM model proposed in this thesis is better than other models,and the effect of SA_DeepFM model is better,which verifies the improvement of model prediction effect by introducing sequence model and attention mechanism.Finally,the hyper-parameters in the optimal model SA_DeepFM are optimized and analyzed to obtain the optimal parameter combination and further improve the prediction effect of the model.更多还原