【作者】 杨飞；

【导师】 张沪寅；

【作者基本信息】 武汉大学 ， 计算机系统结构， 2022， 博士

【摘要】 图结构数据在现实世界中无处不在,如引文网络、社交网络、网页网络和生物网络等,其包含丰富的有待挖掘的信息。相应地,图数据挖掘旨在挖掘图拓扑结构,节点特征以及边特征之间相关性,并且这已被证明在节点分类(Node Classification)、节点聚类(Node Clustering)和链接预测(Link Prediction)等任务中非常成功。其中,面向节点分类的图神经网络模型研究是一个值得深入研究的方向。节点分类通常是给定图中某些节点对应的类别,然后通过传统的图表示学习方法或图神经网络来预测图中没有标签的节点所属的类别。近些年,图表示学习和图神经网络已成为图数据挖掘领域的热门研究方向。随着深度学习的发展和图神经网络表示能力的展现,在面对上述图表示学习的任务时,大量工作提出了基于深度学习的图神经网络。依据卷积是应用于谱域还是空间域,图神经网络的研究进展通常被归类为谱方法和空间方法。然而,考虑到图神经网络的邻域聚合,模型深度和运算效率,在设计图神经网络模型时,仍面临如下挑战:(1)针对谱方法,如何简化邻域聚合过程,同时提升模型深度并缓解过平滑问题从而提升模型分类性能?(2)针对空间方法,在提取节点特征时,如何在不改变图结构的情况下引入所有节点的影响从而提升模型的特征聚合能力?(3)针对谱方法和空间方法之间的隔阂,如何设计一个混合模型来融合谱方法和空间方法的优势?针对以上问题,本文主要研究内容及创新点包括如下几个方面:(1)提出了简单跳跃知识网络(SJK-Net)模型,一种处理图结构数据的新型神经网络模型。本文首先利用简单的无学习方法实现邻域聚合,该方法不仅可以快速有效地聚合每个节点的特征表示,而且占用更少的存储空间。然后,利用跳跃知识网络(JK-Net)在最后一层选择性地组合不同层的特征,从而使所有节点特征表示跳转到最后一层。在这种情况下,SJK-Net提升了模型深度的同时还能快速地学习节点不同邻域范围对应的子图特征表示,而且缓解了过平滑问题。在引文网络和社交网络上的大量实验证明,所提出的模型在节点分类任务上匹配或优于最新的方法。(2)提出了全图注意力神经网络(FGANN)模型。FGANN在计算每个节点的隐藏特征表示时,使用注意力机制引入所有节点的影响。首先将两个节点特征之间的相似性定义为注意力系数,然后通过自注意力机制计算任意两个节点之间的注意力系数。其次,将一种简单有效的技术softmax函数应用于注意力矩阵,将非邻域节点的影响注入到邻域节点中。最后,再次关注邻域中的节点,利用掩码注意力引入图结构(邻接矩阵)并(隐式)为邻域中的不同节点分配不同的权重,从而构建重整化邻接矩阵。在这种情况下,FGANN考虑了所有节点之间的相互影响,并利用全图注意力构建新的邻接矩阵,从而能更有效地处理节点分类任务。引文网络上的大量实验证明,所提出的FGANN模型在节点分类任务上能匹配或优于最新的方法。(3)提出了混合深度图卷积网络(HDGCN)模型,一种结合谱方法和空间方法的新型深度GCN模型。首先,本文定义了一种简单有效的方法来结合谱方法和空间方法。为了充分释放这种组合的优势,再将其引入到深度GCN模型中。HDGCN分别利用谱方法和空间方法计算邻接矩阵,然后将这两个邻接矩阵组合生成新的邻接矩阵。该新的邻接矩阵包含更丰富的图结构信息,有利于提取更准确的节点特征表示。此外,HDGCN利用初始残差和恒等映射来缓解深度GCN模型中的过平滑问题。在这种情况下,HDGCN从结合邻接矩阵的角度消除了谱方法和空间方法之间的隔阂,并展现了深度GCN模型提取更准确节点特征表示的优势。在引文网络和网页网络上的大量实验提供了证据,表明HDGCN在节点分类任务上匹配或优于最先进的方法。综上所述,本文针对节点分类任务,分别从谱方法、空间方法和混合方法三个方面,对图神经网络模型进行了深入、细致的研究。实验结果表明,本文提出的图神经网络模型能够有效提高运行效率,提升模型深度,缓解过平滑问题,从而高效处理节点分类任务。更多还原

【Abstract】 Graph-structured data are ubiquitous in the real world,such as citation networks,social networks,web networks,and biological networks,and they contain rich information to be mined.Correspondingly,graph mining aims to mine the correlations among graph topology,node features,and edge features,and this has been proven to be very successful in downstream tasks such as node classification,node clustering,and link prediction.Among them,the research of graph neural network model for node classification is a direction worthy of further study.Node classification is usually given the class of some nodes in the graph,and then predicts the class of unlabeled nodes in the graph through traditional graph representation learning methods or graph neural networks.In recent years,graph representation learning and graph neural networks have become popular research directions in the field of graph mining.With the development of deep learning and the explosion of graph neural network representation capabilities,a large number of works have proposed graph neural networks based on deep learning when dealing with the above tasks of graph representation learning.According to whether the convolution is applied to the spectral domain or the spatial domain,the research progress of graph neural networks is usually divided into spectral approaches and spatial approaches.However,considering the neighborhood aggregation,model depth and computational efficiency of the graph neural network,when designing the graph neural networks,there are the following challenges:(1)For spectral approaches,how to simplify the neighborhood aggregation process,while improving the model depth and alleviating the oversmoothing to improve the model classification performance?(2)For spatial approaches,when extracting node features,how to introduce the influence of all nodes without changing the graph structure to improve the feature aggregation ability of the model?(3)In view of the gap between spectral approaches and spatial approaches,how to design a hybrid model to integrate the advantages of spectral approaches and spatial approaches?Given the above problems,the main research contents and innovations of this dissertation include the following aspects:(1)Simple Jumping Knowledge Network(SJK-Net)is proposed.SJK-Net is a novel neural network model for processing graph-structured data.In this dissertation,we first implement neighborhood aggregation with a simple no-learning approach,which not only aggregates the feature representations of each node quickly and efficiently,but also takes up less storage space.Then,in the last layer,we utilize Jumping Knowledge Networks(JK-Net)to combine the features of different layers,so that all node feature representations jump to the last layer.In this case,SJK-Net improves the depth of the model and can quickly learn the subgraph feature representations corresponding to different neighborhood ranges of nodes,thus alleviating the problem of oversmoothing.Extensive experiments on citation networks and social networks demonstrate that the proposed model matches or outperforms state-of-the-art methods on the node classification tasks.(2)Full Graph Attention Neural Network(FGANN)is proposed.FGANN leverages an attention mechanism to introduce the influence of all nodes when computing the hidden feature representation of each node.First,we define the similarity between two node features as the attention coefficient,and then calculate the attention coefficient between any two nodes through the self-attention mechanism.Second,a simple and effective technique,the softmax function,is applied to the attention matrix to inject the influence of non-neighborhood nodes into in-neighborhood nodes.Finally,we again focus on the nodes in the neighborhood,and use masked attention to introduce the graph structure(adjacency matrix)and(implicitly)assign different weights to different nodes in the neighborhood,thus constructing the renormalization adjacency matrix.Under these circumstances,FGANN not only addresses several important challenges of spatialbased graph neural networks,but also handles node classification tasks efficiently.In this case,FGANN considers the mutual influence among all nodes and utilizes full-graph attention to construct a new adjacency matrix,which can handle the node classification task more efficiently.Extensive experiments on the citation networks demonstrate that the proposed FGANN model can match or outperform state-of-the-art methods on the node classification tasks.(3)Hybrid Deep Graph Convolutional Network(HDGCN)is proposed.HDGCN is a novel deep GCN model that combines spectral and spatial approaches.First,this dissertation defines a simple and effective method to combine spectral and spatial approaches.To take advantage of this combination,we introduce it into a deep GCN model.HDGCN calculates the adjacency matrix using spectral method and spatial method respectively,and then combines these two adjacency matrices to generate a new adjacency matrix.The new adjacency matrix contains richer graph structure information,which is beneficial to extract more accurate node feature representation.In addition,HDGCN utilizes initial residual and identity mapping to alleviate the oversmoothing problem in deep GCN models.In this case,HDGCN bridges the gap between spectral and spatial methods from the perspective of combining adjacency matrices,and exhibits the advantage of deep GCN models to extract more accurate node feature representation.Extensive experiments on citation networks and web networks provide evidence that the proposed model matches or outperforms state-of-the-art methods on the node classification tasks.In summary,for the node classification tasks,this dissertation conducts in-depth and detailed research on the graph neural network models from three aspects: spectral approaches,spatial approaches and hybrid approaches.The experimental results show that the proposed graph neural networks can effectively improve the operating efficiency,improve the depth of the model,and alleviate the oversmoothing problem,so that it can efficiently handle the node classification tasks.更多还原