【作者】 张杰；

【导师】 陈晓辉； 陈力；

【作者基本信息】 中国科学技术大学 ， 信息与通信工程， 2024， 硕士

【摘要】 近年来,得益于人工智能(AI)技术和移动通信技术的蓬勃发展,大量无线终端设备被接入网络,利用这些设备上丰富的用户数据能够训练出更强大的机器学习模型。然而,传统的模型训练方法是由云服务器统一收集数据后集中训练,面临着用户隐私泄漏等挑战。为了应对这些挑战,于2016年提出的联邦学习(FL)逐渐获得研究者的青睐。该技术允许设备节点能够在用户数据不离开本地的情况下训练模型并与服务器交互模型参数,从而避免了原始数据的泄漏。而无中心联邦学习(DFL)进一步将隐私保护下的人工智能向网络边缘下沉,以满足用户对于低延迟、高可靠性等方面的要求。然而边缘网络和终端设备具有拓扑一致性低、网络通信受限、终端资源受限和系统异构等特性,这些因素严重影响了模型训练的效率和收敛性能。为了解决上述挑战,本文主要研究在无线边缘网络场景下,如何提高无中心联邦学习的模型收敛性能和训练速度,其主要研究工作如下:针对无中心联邦学习网络侧拓扑一致性低和通信受限导致模型收敛缓慢的问题,本文提出了一种基于分层网络的无中心联邦学习架构,该架构结合小区内设备间的D2D通信和不同小区基站间的回程通信,提高了通信拓扑的一致性和设备间模型参数的共识能力,改善了模型的收敛性能。同时,考虑到该系统有限的通信资源,本文根据各节点通信、计算阶段的时延和能耗构建了资源分配问题,通过解决该问题获得了各节点带宽分配和算力选择的最优解,在限制系统的时延和能耗的情况下降低了通信资源开销。仿真实验验证了所提方案的有效性。针对无中心联邦学习终端侧资源受限和系统异构导致模型训练效率低下的问题,本文提出了一种面向资源受限和异构终端的无中心联邦学习方法。具体来说,我们首先根据模型参数的依赖关系对参数进行分组,确保被移除的参数是一致冗余的。然后用权重值和批次归一化层缩放因子评估参数重要性,并在模型训练过程中平滑地剪枝模型,尽可能减小因剪枝造成的模型性能损失,同时避免了传统剪枝方法的微调和重训练。为了缓解系统异构问题,我们为终端动态配置剪枝率,即让计算能力强的终端采用较小的修剪比例以保障模型性能,而让计算能力弱的采用较大的修剪比例缓解了系统异构问题。除了理论研究,我们还利用Linux开发板作为终端节点搭建了系统平台并在平台上开展了实验验证。实验结果验证了所提方案的有效性。更多还原

【Abstract】 In recent years,thanks to the booming development of Artificial Intelligence(AI)technology and mobile communication technology,a large number of wireless terminal devices have been connected to the network,and the rich user data on these devices can be utilized to train more powerful machine learning models.However,traditional model training methods,in which data is collected uniformly by cloud servers and then centrally trained,face challenges such as user privacy leakage.To address these challenges,federated learning(FL),which was proposed in 2016,is gradually gaining favor among researchers.This technique allows device nodes to be able to train models and interact with servers on model parameters without user data leaving the local area,thus avoiding leakage of raw data.And decentralized federated learning(DFL)further sinks privacypreserving AI towards the edge of the network to satisfy users’ requirements for low latency,high reliability,and so on.However,edge networks and terminal devices are characterized by low topological consistency,restricted network communication,limited terminal resources,and system heterogeneity,which seriously affect the efficiency of model training and convergence performance.In order to solve the above challenges,this paper focuses on how to improve the model convergence performance and training speed of decentralized federated learning in wireless edge network scenarios,and its main research work is as follows:Aiming at the problem of slow model convergence due to low topology consistency and communication constraints on the network side of decentralized federated learning,this paper proposes a hierarchically decentralized federated learning architecture,which combines the D2D communication between devices in a cell and the backhaul communication between base stations in different cells to increase the consistency of the communication topology and the consensus ability of the model parameters among the devices,and improves the convergence performance of the model.Meanwhile,considering the limited communication resources of the system,this paper constructs a resource allocation problem based on the delay and energy consumption of the communication and computation phases of each node,and solves the problem to obtain the optimal solution of bandwidth allocation and arithmetic power selection of each node,which reduces the communication resource overhead while restricting the delay and energy consumption of the system.Simulation experiments verify the effectiveness of the proposed scheme.Aiming at the problem of inefficient model training due to resource constraints and system heterogeneity on the terminal side of decentralized federated learning,this paper proposes a decentralized federated learning method for resource-constrained and heterogeneous terminals.Specifically,we first group the parameters based on the dependencies of the model parameters to ensure that the removed parameters are consistently redundant.Then we evaluate parameter importance using weight values and batch normalized layer scaling factors,and smoothly prune the model during model training to minimize model performance loss due to pruning,while avoiding fine-tuning and re-training in traditional pruning methods.In order to alleviate the system heterogeneity problem,we dynamically configure the pruning ratio for the terminals,i.e.,letting terminals with strong computational power adopt smaller pruning ratios to safeguard the model performance,and letting those with weak computational power adopt larger pruning ratios to alleviate the system heterogeneity problem.In addition to the theoretical study,we also built a system platform using Linux development boards as terminal nodes and carried out experimental validation on the platform.The experimental results verify the effectiveness of the proposed scheme.更多还原