An adaptive federated learning scheme with differential privacy preserving

Abstract

Abstract Driven by the upcoming development of the sixth-generation communication system (6G), the distributed machine learning schemes represented by federated learning has shown advantages in data efficient utilization and multi-party cooperative modeling. The total communication cost of federated learning is related to the number of communication rounds, the communication consumption of each participant, the setting of reasonable learning rate and the guarantee of calculation fairness have important influence on the control of total cost. In addition, the simple data isolation strategy in the federated learning framework cannot completely guarantee the privacy security of users. Motivated by the above problems, this paper proposed a federated learning scheme combined with the adaptive gradient descent strategy and differential privacy mechanism, which is suitable for multi-party collaborative modeling scenarios. To ensure that federated learning scheme can train efficiently with limited communications costs, the learning rate adaptive algorithm is innovatively used to adjust the gradient descent process to avoid model overfitting and fluctuation phenomena, so as to improve the modeling efficiency and model performance in multi-party calculation scenarios. Furthermore, aiming at adapting to the ultra-large-scale distributed secure computing scenario, this research introduces a differential privacy mechanism to resist various background attacks. Experimental results demonstrate that the proposed adaptive federated learning model performs better than the traditional models at fixed communication costs. This novel modeling scheme also shows robustness to different super-parameter settings and provides stronger quantifiable privacy preserving for federated learning process.