【摘要】 目的 在联邦学习场景中，由于各客户端数据分布的不一致，会导致各客户端的局部目标之间偏差较大，以及全局平均模型偏离全局最优，影响模型训练的收敛速度和模型精度。针对非独立同分布数据导致的全局模型收敛缓慢以及模型准确率较低的问题，提出一种联合动态校正的联邦学习算法（federated learning algorithm for joint dynamic correction, FedJDC），分别从客户端和服务器端进行优化。方法 为了降低局部模型更新偏移的影响，定义累积偏移度来衡量各参与客户端的数据非独立同分布程度，并在本地损失函数中引入动态约束项，根据累积偏移度动态调整约束项大小，可自动适应不同程度的非独立同分布数据，减小局部模型的更新方向不一致性，从而提高模型准确率及通信效率；其次，针对全局模型聚合偏移，将参与客户端上传的累积偏移度作为全局模型聚合权重，从而动态更新全局模型，大幅减少通信轮数。结果 本文在3个真实数据集上的实验结果表明，与4种不同的联邦学习算法相比，在多种不同非独立同分布程度的情况下，FedJDC可以平均减少62.29%、20.90%、24.93%和20.47%的通信轮次，平均提高5.48%、1.62%、2.10%和2.28%的模型准确率。结论 本文提出的联邦学习中局部和全局偏移的联合动态校正算法从局部模型更新和全局模型聚合两方面进行改进，降低了通信轮次，提高了准确率，取得了良好的收敛效果。更多还原

【Abstract】 Objective Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. In practical applications, the data between nodes usually follow a non-independent identical distribution(non-IID). In the local update, each client model will be optimized toward its local optima(i. e., fitting its individual feature distribution) instead of the global optimal objective and raises a client update drift. Meanwhile, in global updates that aggregate these diverged local models, the server model is further distracted by the set of mismatching local optima, which subsequently leads to a global drift at the server model. To solve the problems of slow global convergence and increasing number of training communication rounds caused by non-IID data, this paper proposes a joint dynamic correction federated learning algorithm(FedJDC) that is optimized from the client and server.Method To reduce the influence of non-IID on federated learning, this paper carries out a joint optimization from the two aspects of local model update and global model update and proposes the FedJDC algorithm. This paper then uses the cosine similarity between the local and global update directions to measure the offset of each participating client. Afterward, given that each client has a different degree of non-IID, if the degree of the model offset is only determined by the cosine similarity calculated in this round, then the model update may become unstable. Therefore, the FedJDC algorithm defines the cumulative offset and introduces the attenuation coefficient ρ. In calculating the cumulative offset of the model, the current and historical cumulative offsets are taken into account. In addition, by changing ρ to reduce the proportion of the cumulative offset of the current round, the influence of the offset of the current round on the final result can be reduced. This paper also proposes a strategy for dynamically adjusting the constraint terms for local model update offset. Specifically, the constraint terms of the local loss function are dynamically adjusted according to the calculated cumulative offset of the local model, and the algorithm is automatically adapted to various non-IID settings without a careful selection of hyperparameters, thus improving the flexibility of the algorithm. To dynamically change the weight of global model aggregation in each round and effectively improve the convergence speed and model accuracy, this paper also designs a dynamic weighted aggregation strategy that takes the accumulated offset uploaded by all clients as the weight of global model aggregation in each round of communication.Result The proposed method is tested on three dataset using different deep learning models. LeNeT5, the VGG16network model, and the ResNet18 network model are used for training in the MNIST, FMNIST, and CIFAR10 datasets, respectively. Four experiments are designed to prove the effectiveness of the proposed algorithm. To verify the accuracy of FedJDC at different degrees of non-IID, the hyperparameter β of the Dirichlet distribution is varied, and the performance of different algorithms is compared. Experimental results show that FedJDC can improve the model accuracy by 5. 48%, 1. 62%, 2. 10%, and 2. 28% on average compared with FedAvg, FedProx, FedAdp, and FedLAW, respectively. To evaluate the communication efficiency of FedJDC, the number of communication rounds is counted as FedJDC reaches a target accuracy, and this number is compared with that obtained by other algorithms. Experimental results show that under different degrees of non-IID, FedJDC can reduce communication rounds by 62. 29%, 20. 90%, 24. 93%, and 20. 47% on average compared with FedAvg, FedProx, FedAdp, and FedLAW, respectively. This paper also investigates the effect of the number of local epochs on the accuracy of the final model. Experimental results show that FedJDC outperforms the other four methods under different epochs in terms of final model accuracy. FedJDC also demonstrates better robustness against the larger offset caused by more local update epochs. Ablation experiments also show that each optimization method performs well on all datasets, and FedJDC combines these two strategies to achieve the global optimal performance.Conclusion This paper optimizes the local and global model offsets from two aspects and proposes a joint dynamic correction algorithm for these offsets in federated learning. The cumulative offset is defined, and the attenuation coefficient is introduced into the calculation of the cumulative offset. Considering the historical and current offset information, the size of the cumulative offset is dynamically adjusted to ensure the stability of the training parameter update. The dynamic constraint strategy takes the cumulative offset calculated by the client in each round as the constraint parameter of the client model. The dynamic weighted aggregation strategy changes the weight of each local model during the global model aggregation based on the cumulative offset of each participating client so as to dynamically update the global model in each round.The combination of the two optimization strategies has achieved good results, effectively alleviated the performance degradation of the federated learning model caused by non-IID data, and provided a good foundation for the further implementation of federated learning in this field.更多还原