【作者】 胡涛；

【导师】 常征；

【作者基本信息】 电子科技大学 ， 电子信息（专业学位）， 2023， 硕士

【摘要】 随着物联网设备数量的不断增长,巨量数据被产生,为机器学习提供了丰富的数据资源,然而在利用这些数据进行训练的时候也存在着很多问题。在传统的中心化机器学习模式中,所有数据都集中在一个中心节点进行处理,这不仅存在着隐私泄露的风险,还需要大量的集中式计算资源。为了解决传统中心化机器学习所带来的问题,联邦学习应运而生,作为新兴的机器学习技术,可以在保护个人隐私的前提下,利用终端设备的计算能力和本地数据来进行模型训练。然而,联邦学习也存在一些问题,因为设备本身与设备网络环境的异构性,尤其是在涉及大量设备的情况下,现有的联邦学习算法容易产生高昂的通信成本,在参数上传阶段需要极大的上行带宽与较低的通信延迟。为了解决这一瓶颈问题,本文提出了一种被称为CE-Fed PA的通信高效的联邦学习方法,该方法由数据压缩、设备选择策略两部分组成。在数据压缩部分,为了避免数据压缩后模型信息的丢失与收敛速度的降低,引入Adam优化算法。对Adam优化算法产生的参数矩阵、一阶力矩矩阵、二阶力矩矩阵采用稀疏、量化、Rice编码,并对量化后的联邦优化做了收敛性分析。随后的定量分析结果表明,对比传统联邦学习方法,压缩率对于不同的稀疏率可以达到2-15倍,对比使用Adam优化算法的传统联邦学习方法而言,压缩率对于不同的稀疏率可以达到6-45倍,从而减少了客户端上传的数据量。在设备选择策略部分,使用基于本文提出的指标——设备贡献效应因子作为优先级来动态选择设备集合。设备贡献效应因子由设备数据质量与运算能力影响。继而提出了关于设备贡献效应的多约束线性规划数学模型,通过动态规划算法求解该问题,最后对算法的正确性与有效性做了一定分析,通过设备选择策略,避免了通信网络质量很差的设备参与,从而降低了通信消耗。为了验证本系统的通信高效性,在人工智能领域常用的公开数据集上的实验中,本文探索了独立同分布和非独立同分布两种数据分布,使用仿真环境与实机环境均开展了大量的实验。仿真实验结果表明本文提出的CE-Fed PA算法比传统联邦学习算法在通信效率上表现更加优越,达到相同的模型精度所需要的通信时间分别减少1.8倍和5.8倍。在实机实验中,由四个安卓手机与一台电脑作为服务器组成的端对端真实物联网环境下,本系统的通信时间也减少了2.2倍。结果表明本系统在模型精度基本未受到影响的前提下可以大大减少上传数据量,降低通信时延。更多还原

【Abstract】 As the number of Io T devices continues to grow,huge amounts of data are being generated,providing huge amounts of raw materials for machine learning,yet there are many problems to be addressed in the utilization of this data.In the traditional centralized machine learning model,all data are collected in a central node for processing,which not only has the risk of privacy leakage,but also requires a large amount of centralized computing resources.In order to solve the problems caused by traditional centralized machine learning,federation learning has emerged as an emerging machine learning technique that can utilize the computing power of end devices and local data for model training while protecting personal privacy.However,federation learning also has some problems because of the heterogeneity of the device itself and the network environment,especially when a large number of devices are involved,existing federation learning algorithms are prone to high communication costs,such as the need for great uplink bandwidth with low communication latency.To address this bottleneck,this thesis proposes a communication-efficient federation learning method called CE-Fed PA.The method consists of two parts: data compression,and device selection.In the data compression part,the Adam optimization algorithm is introduced to avoid the loss of model information and the reduction of convergence speed after data compression.The parameter matrix,first-order moment matrix,and second-order moment matrix generated by the Adam optimization algorithm are sparse,quantized,and Rice coded,and the convergence analysis of the quantized federal optimization is done.The subsequent quantitative analysis results show that the compression rate can reach 2-15 times for different sparse rates compared to the traditional federation learning method,and 6-45 times for different sparse rates compared to the traditional federation learning method using the Adam optimization algorithm,thus reducing the amount of data uploaded by the client.In the device selection strategy section,the set of devices is dynamically selected using the device contribution effect factor,a metric based on the one proposed in this thesis,as the priority.The device contribution effect factor is influenced by the device data quality and computing power.Then,a multi-constrained linear programming mathematical model of the device contribution effect is proposed,and the problem is solved by a dynamic programming algorithm,and finally,the correctness and effectiveness of the algorithm are analyzed,and the device selection strategy avoids the participation of of devices with poor communication network quality,thus reducing the communication consumption.In order to verify the communication efficiency of this system,both independent and non-independent and identically distributed are explored in experiments on publicly available datasets commonly used in the field of artificial intelligence,and extensive experiments are conducted using both simulation and real-world environments.The results of simulation experiments show that the CE-Fed PA algorithm proposed in this thesis performs better than the traditional federation learning algorithm in terms of communication efficiency,requiring 1.8 times and 5.8 times less communication time to achieve the same model accuracy,respectively.In the real-world experiment,the communication time of this system is also reduced by 2.2 times in an end-to-end real Io T environment consisting of four Android phones and one computer as a server.The results show that the system can greatly reduce the amount of uploaded data and reduce the communication delay under the premise that the accuracy of the model is basically not affected.更多还原