Masked Diffusion Strategy for Privacy-Preserving Distributed Learning

To protect both local gradients and estimated parameters in distributed learning, this paper introduces a masked diffusion (MD) strategy, leading to two algorithms: the MD stochastic gradient (MD-SG) and the MD primal-dual stochastic gradient (MPD-SG). The two algorithms distinguish themselves from existing privacy diffusion methods by incorporating two mechanisms: non-zero mean protection noise and a random matrix step-size. The first mechanism ensures the confidentiality of the transmitted values, while the second protects the gradient information. We analyze the mean-square stability and privacy of the proposed methods under standard assumptions. The results indicate that the MPD-SG algorithm, with a sufficiently small parameter γ, can achieve better steady-state performance than the MD-SG algorithm in heterogeneous data scenarios. Finally, simulations illustrate the effectiveness of the proposed algorithms and support the theoretical analysis.