【作者】 王玉玲；

【导师】 于艳华；

【作者基本信息】 北京邮电大学 ， 计算机科学与技术， 2024， 博士

【摘要】 随着数据科学和人工智能领域的蓬勃发展,图结构作为一种极其灵活的数据表示形式,已成为研究和应用的重要焦点之一。图神经网络能够将图结构特征映射到低维向量空间,其出色的复杂关系捕获能力备受青睐。尽管当前的图神经网络已经在下游任务中取得了瞩目的效果,但是这些模型大多数是针对单一的同质图结构提出的,并不适配于具有多种关系类型的多关系图结构。在真实世界复杂的应用场景中,多关系图结构是更加普遍存在的,这些多关系在不同的应用场景下被广泛地定义和区分,蕴含了各领域内独特的语义信息,例如,分子图中各种类型的化学键,以及社交网络中人们之间的多样关系。多关系图扩展了传统单关系图数据结构表达信息的能力,能够更加全面地理解和分析真实世界的复杂系统中的交互关系。因此,研究多关系图建模与分析对于数据挖掘和实际应用具有重要的意义。多关系图的建模重点在于学习不同关系的独特语义,同时有效地处理多种关系之间的复杂交互。当前多关系图神经网络在模型理论和下游应用方面仍然面临诸多挑战:(1)随着关系种类的增多、神经网络的层数堆叠,当前的模型面临过度参数化、过平滑等问题。(2)当前的模型多数基于黑盒式的启发式设计,面临缺乏理论保障、鲁棒性差等问题。(3)在下游任务的隐式多关系挖掘中,由于人工数据标注的困难性和关系含义的模糊性,推荐系统中隐藏着许多未定义的隐式多关系类型,如何基于稀疏的交互数据挖掘这些潜在的多关系信息是重要的应用挑战。针对以上挑战,本研究针对多关系图神经网络的模型与应用展开研究。具体来说,本研究的主要研究成果包括以下几条:一,研究从优化目标的角度设计多关系图神经网络。具体来说,提出了一个集成优化框架,包括:一个特征拟合项使得节点能够保留自身信息,和一个集成多关系图正则项为每种关系仅配备一个可学习的权重系数。基于该优化目标能够推导出基于迭代更新规则的集成消息传递机制,随后该机制被自然地整合到深度神经网络中,得到集成多关系图神经网络。二,推理证明了集成消息传递机制的良好性质,并提出了鲁棒集成多关系图神经网络框架。具体而言,证明了集成消息传递机制算法的收敛性,同时融入了非线性激活函数增加模型的完整性。此外,证明了集成消息传递机制与一些流行的传播机制之间的良好的联系,包括同质和异质场景。最后,提出了鲁棒的语义图正则项,在不同扰动比例下的实验证明了提出模型优异的鲁棒性能。三,提出基于解耦图对比学习的意图感知推荐系统。具体来说,将用户行为数据建模为用户-物品-概念图,然后设计基于图神经网络的行为解耦模块学习不同的潜在意图(隐式多关系)。接着,提出了基于意图的对比学习方法,以促进有意义的意图解耦,同时推断行为在不同意图上的分布。最后,引入了编码率降低正则化,使不同意图的行为变得正交。进一步的分析显示,学习到的意图表示和行为分布具有可解释性。四,提出基于蒸馏大语言模型的意图感知序列推荐系统。具体而言,首先提出一种逐步知识蒸馏策略,将教师模型的推理能力转移到“小型”学生模型上,以用于序列推荐。随后,直接部署“小”语言模型作为序列推荐的知识生成器,它可以推导出与用户多意图相关的高质量知识。最后,这些知识可以灵活地与任何序列推荐的骨干模型集成,同时可以作为额外的知识/监督信息促进图模型以及其它模型解耦出多种用户意图(隐式多关系)。在不同推荐骨干模型上验证了模型的性能,同时表明了模型良好的解释性和缓解长尾偏差的能力。更多还原

【Abstract】 With the thriving development in the fields of data science and artificial intelligence,graph structures have become one of the crucial focal points for research and application,owing to their highly adaptable nature as a data representation form.Graph Neural Networks(GNNs)are capable of mapping graph structure features into low-dimensional vector spaces,and their remarkable ability to capture complex relationships has garnered considerable attention.Despite the remarkable performance achieved by current GNNs in downstream tasks,most of these models are proposed for homogeneous graph structures and do not adapt well to multi-relational graph structures with various types of relations.In complex real-world application scenarios,multi-relational graph structures are more prevalent,with these multiple relations being widely defined and distinguished across different application contexts,encapsulating unique semantic information within various domains,such as different types of chemical bonds in molecular graphs and diverse relationships among individuals in social networks.Multi-relational graphs extend the capability of traditional single-relational graph data structures in representing information,enabling a more comprehensive understanding and analysis of interaction relationships within complex systems in the real world.Therefore,research on multi-relational graph modeling and analysis holds significant importance for data mining and practical applications.The modeling focus of multi-relational graphs lies in learning the unique semantics of different relations while effectively handling the complex interactions among multiple relationships.Currently,multirelational GNNs still face several challenges in both model theory and downstream applications:(1)With an increase in the number of relation types and the stacking of neural network layers,current models encounter issues such as over-parameterization and over-smoothing.(2)Most current models are based on heuristic designs lacking theoretical support,resulting in poor robustness.(3)In implicit multi-relation mining for downstream tasks,due to the difficulty of manually annotating data and the ambiguity of relations,there are many undefined implicit multi-relations hidden in recommendations.How to mine these latent multi-relation information based on sparse interaction data is an important application challenge.In response to these challenges,this study investigates the model and application of multi-relation GNNs.Specifically,the main research achievements of this study include the following:First,the research focuses on designing multi-relational GNNs from the perspective of optimization objectives.Specifically,it proposes an ensemble optimization framework,including a feature fitting term to preserve node information and an ensemble multi-relational graph regularization term with a learnable weight coefficient for each relation.Based on this optimization objective,an ensemble message passing mechanism is derived from iterative update rules.Subsequently,this mechanism is naturally incorporated into deep neural networks to obtain integrated multi-relational GNNs.Second,the inference demonstrates the favorable properties of the ensemble message passing mechanism and proposes a robust ensemble multi-relational GNN framework.Specifically,it proves the convergence of the ensemble message passing algorithm while incorporating nonlinear activation functions to enhance model integrity.Additionally,it demonstrates the good relationship between the ensemble message passing mechanism and some popular propagation mechanisms,including homogeneous and heterogeneous scenarios.Finally,a robust semantic graph regularization term is proposed,and experimental results under different perturbation ratios demonstrate the excellent robust performance of the proposed model.Third,it introduces an intent-aware recommender system based on disentangle graph contrastive learning.Specifically,it models user behavior data as a user-item-concept graph,then designs a behavior disentangling module based on GNNs to learn different latent intents(implicit multi-relations).Next,it proposes an intent-based contrastive learning method to facilitate meaningful intent disentangling while inferring behavior distributions.Finally,it introduces regularization to reduce encoding redundancy,making behaviors of different intents orthogonal.Further analysis shows that learned intent representations and behavior distributions are interpretable.Fourth,it proposes an intent-aware sequential recommender system based on distilling large language models.It introduces a step-by-step knowledge distillation strategy to transfer the reasoning ability of teacher models to smaller student models for sequential recommendation.Subsequently,it directly deploys "small" language models as knowledge generators for sequential recommendation,which can derive high-quality knowledge related to multi-intents of users.Finally,these knowledge can be flexibly integrated with any sequential recommendation backbone and serve as additional knowledge/supervision information to facilitate the disentangling of user intents(implicit multi-relations)by graph models and other models.The model’s performance is verified on different recommendation backbones,demonstrating its good interpretability and ability to alleviate long-tail biases.更多还原