【摘要】 参与联邦学习的终端设备只需在各自的本地数据集上训练本地模型,并在服务器的协同下共同训练一个全局预测模型.因此,联邦学习可以在不共享终端设备的隐私和敏感数据的情况下实现机器学习的目的.然而,大量终端设备对服务器的高并发访问会增加模型更新的传输延迟,并且本地模型可能是与全局模型收敛方向相反的恶意模型,因此联邦学习过程中会产生大量额外的通信成本.现有工作主要集中在减少通信轮数或清除本地脏数据,本文研究了一种基于边缘的模型清洗和设备聚类方法,以减少本地更新总数.具体来说,通过计算本地更新参数和全局模型参数在多维上的余弦相似度来判断本地更新是否是必要的,从而避免不必要的通信.同时,终端设备根据其所在的网络位置聚类,并通过移动边缘节点以簇的形式与云端通信,从而避免与服务器高并发访问相关的延迟.本文以Softmax回归和卷积神经网络实现MNIST手写数字识别为例验证了所提方法在提高通信效率上的有效性.实验结果表明,相比传统的联邦学习,本文提出的基于边缘的模型清洗和设备聚类方法减少了 60%的本地更新数,模型的收敛速度提高了 10.3%.更多还原

【Abstract】 The end devices participating in federated learning train the local model on their local datasets and collaboratively learn a global prediction model with the server,so federated learning can achieve the purpose of machine learning without sharing private and sensitive data.In fact,federated learning typically takes several iterations between the terminal device and the cloud server to reach the target accuracy.Therefore,when a large number of end devices communicate with servers,the limited network bandwidth between the server and terminal devices will inevitably lead to a large model transmission delay.In addition,due to the heterogeneity of end devices and the non-independent identically distributed characteristics of local data,local models may be malicious models that converge in the opposite direction to the global model.These models not only poison the accuracy of the global model but increase additional communication costs.Therefore,reducing network occupancy and improving the communication efficiency of federated learning becomes crucial.The existing research mainly focuses on reducing communication rounds or cleaning dirty data from local.One of the studies is to calculate the number of identical symbolic parameters between the global model and the local update to determine the importance of the local update,ultimately reducing the communication rounds.It only considers the difference in the direction of model parameters and does not consider the parameter deviation between the global model and the local model.Different from the existing work,this paper proposes an edge-based model cleaning and device clustering method to reduce the number of local updates.Specifically,we calculate the cosine similarity between the local update parameters and the global model parameters to determine whether the local update is necessary to be uploaded.If the cosine similarity between the two is less than the set threshold,the update will not be uploaded to the server for global aggregation,thereby avoiding unnecessary communication.Meanwhile,end devices clustered according to their network locations and communicate with the cloud in the form of clusters through mobile edge nodes,thereby avoiding the delay associated with high concurrent access to the server.Considering that model updates usually contain large gradient vectors,a large amount of data needs to be transferred by mobile edge nodes,which may increase model transmission time.Therefore,before the model is transferred to the cloud,the mobile edge node uses its computing resources to first aggregate the local updates in the cluster and then transmits the aggregated cluster model to the cloud server for global aggregation.Each edge aggregation consumes computing resources from mobile edge nodes,and each global aggregation consumes network communication resources.This paper takes Softmax regression and convolutional neural networks to realize MNIST handwritten digit recognition as an example to verify the effectiveness of the proposed method in improving communication efficiency.The experimental results show that compared with the traditional federated learning,the edge-based model cleaning and device clustering method proposed in this paper reduces the local update of the Softmax regression model by 60%,and the convergence speed of the model increases by 10.3%.更多还原