【作者】 王杰；

【导师】 张璇；

【作者基本信息】 云南大学 ， 人工智能与机器学习， 2023， 硕士

【摘要】 联邦学习是一种分布式机器学习方法,可以保证本地数据安全的同时与其它参与方共同建模完成机器学习任务,在当今数据驱动的时代,联邦学习已经成为了一种重要的机器学习方法,常应用于对数据安全具有高敏感性的行业中,如金融和医疗行业等。联邦学习允许多个参与方在保护数据隐私的同时,共同训练一个高质量的模型,联邦学习现在正迅速成为深度学习中分布式训练研究的标准,旨在解决本地数据安全问题,增强本地数据的安全性,并降低服务器成本,然而,随着数据量的增加,如何在保证隐私安全的同时,最大化模型的训练性能变得越来越重要。在联邦学习中,安全和效率是重要的维度,但绝对安全和效率不能同时存在。如果想同时提高效率和安全性,我们就需要在效率和安全之间找到平衡,同时,异构边缘网络的通信效率和安全性作为联邦学习的一个研究瓶颈,限制了模型容量、用户参与度和设备安全性,所以我们在研究联邦学习系统部署时需要同时考虑安全和效率问题,并在安全和效率之间找到一个平衡点来解决这个问题,即需要在具有分布式数据隐私方法的联邦学习场景下快速训练模型。在本文中,针对联邦学习训练过程中存在的外部威胁问题,本文首先提出了一种新技术基于差分隐私的轻量化建模联邦学习方法(Fed-Dpcm),它利用了差分隐私和轻量级建模的思想。将噪声添加到全局模型中,并在每个训练循环中压缩客户端训练参数,以减轻模型训练的负担,最终,可以在安全的环境中以更少的训练时间获得高精度的模型。可是本文的方法在针对内部威胁问题时缺乏相应的应对手段,这便是研究过程中的另一挑战。在上述工作的基础上,本文进一步探索解决联邦学习内部威胁的有效方法。在本文后续研究工作中,提出了一种新的无服务器联邦学习框架——基于优化委员会机制的高效联邦学习框架(HEFL),该框架借助区块链可以在保证快速收敛的情况下确保算法的鲁棒性并有效抵御内部威胁本文通过广泛的实验表明,HEFL与现有的联邦学习方法相比可以有更快的收敛速度,同时以分散方法的方式获得比传统拜占庭容忍算法更好的鲁棒性。更多还原

【Abstract】 Federated learning is a distributed machine learning approach that can secure local data while modeling the machine learning task while modeling with other participants to complete machine learning tasks.In today’s data-driven era,federated learning has become an important machine learning approach that is often used in industries with high sensitivity to data security,such as finance and healthcare.Federated learning allows multiple participants to jointly train a high-quality model while protecting data privacy,and it is now rapidly becoming the standard for distributed training research in deep learning to address local data security,enhance local data security.However,with the increase of data volume,how to ensure privacy security while maximizing the training performance of models has become increasingly important.In federated learning,security and efficiency are important dimensions,however absolute security and efficiency cannot exist at the same time.If we want to improve efficiency and security at the same time,we need to find a balance between efficiency and security.Meanwhile,the communication efficiency and security of heterogeneous edge networks,as a research bottleneck in federated learning,limit the model capacity,user engagement,and device security,Thus consider both security and efficiency when studying the deployment of federated learning systems,and find a balance between security and efficiency to solve this problem,which means it is necessary to quickly train the model in federated learning scenarios with distributed data privacy methods.In this paper,in response to the external threats in the training process of federated learning,a new technology based on differential privacy lightweight modeling federated learning method(Fed-Dpcm)is first proposed,which utilizes the ideas of differential privacy and lightweight modeling.Add noise to the global model and compress client training parameters in each training cycle to reduce the burden of model training,Ultimately,high-precision models can be obtained in a secure environment with less training time.However,the approach in this paper lacks corresponding countermeasures for the insider threat problem,which is another challenge in the research process.Based on the above work,this paper further explores effective methods to address the insider threat of federated learning.In the subsequent research work of this paper,a new serverless federated learning framework,the High Efficiency Federated Learning Framework(HEFL)based on the optimization committee mechanism,is proposed,With the help of blockchain,this framework can ensure the robustness of the algorithm and effectively resist Insider threat while ensuring rapid convergence.This paper shows that HEFL can have a faster convergence rate than the existing federated learning methods through extensive experiments,Simultaneously achieving better robustness than traditional Byzantine tolerance algorithms through decentralized methods.更多还原