节点文献
批量归一化的自适应联邦学习算法
Adaptive Federated Learning Algorithm with Batch Normalization
【摘要】 针对联邦学习模型在训练过程中出现的客户端漂移以及协变量偏移的问题,提出一种基于批量归一化的自适应联邦学习算法。该算法融合参数自适应更新与批量归一化。在迭代的过程中,客户端本地模型通过自适应参数不断优化,从而缓解客户端漂移。通过批量归一化约束模型复杂度,算法收敛速度显著加快。使用时装数据集以及图像10分类数据集分别在卷积神经网络以及多层感知机网络模型上进行实验。结果表明,相较于经典的联邦平均算法,提出的算法在提升精度的同时加快了30%以上的收敛速度。在非独立同分布的数据实验中,该算法在设备低参与率的情况下也能够达到预期的效果。
【Abstract】 Aiming at client drift and covariate shift during the training of federated learning,we proposed an adaptive federated learning method based on batch normalization. The method combines adaptive updating of parameters and batch normalization techniques. In the iteration process,client local models were gradually optimized by adaptively adjusting variable factors to mitigate client drift. The convergence rate was significantly enhanced by using batch normalization to constrain the complexity of models. With fashionmixed national institute of standards and technology database and Canadian institute for advanced research-10datasets,experiments were performed on both convolutional neural network and multi-layer perceptron network models. Compared with the classical federated average methods,experimental results demonstrate that the proposed method achieves higher accuracy and its convergence rate is enhanced by more than 30%.While it can also achieve promising results with low participation rates of devices in non-identicalindependent-distribution data experiments.
【Key words】 federated learning; adaptive parameters; batch normalization; rate of convergence;
- 【文献出处】 武汉工程大学学报 ,Journal of Wuhan Institute of Technology , 编辑部邮箱 ,2023年05期
- 【分类号】TP181
- 【下载频次】23