SCMoE-PFL: A soft-clustering mixture-of-experts framework for personalized federated learning

Abstract

Traditional federated learning (FL) methods rely on a single global model, which often collapses under heterogeneous and non-IID client data distributions. Personalized federated learning (PFL) alleviates this limitation, yet existing approaches either overfit to local data or fail to exploit shared knowledge effectively. To address these challenges, this paper presents SCMoE-PFL, a personalized federated learning framework that integrates soft clustering and a mixture-of-experts (MoE) mechanism to reconcile global generalization with local personalization. First, we introduce a multi-center threshold-based soft clustering (MCTC) method that enables clients to participate in multiple clusters, improving data utilization and cluster quality. Second, intra-cluster aggregation yields a set of expert models, while each client separately trains a private model on its high-sensitivity data, ensuring privacy preservation. Finally, a lightweight energy-aware gating network adaptively fuses expert and private models. By calibrating initial feature-matching weights with energy-based predictive confidence, this dual-check mechanism effectively prevents over-reliance on uncertain experts, thereby producing highly reliable personalized predictions. Experiments on four benchmark datasets demonstrate that SCMoE-PFL substantially improves accuracy, convergence, and fairness under both moderate and extreme heterogeneity, achieving maximum accuracy improvements of 24.71 and 26.01 percentage points over FedAvg, respectively. Theoretical analysis further establishes performance lower bounds and clarifies the framework’s advantages in privacy protection, computational efficiency, and system reliability. These results show that SCMoE-PFL offers a robust and flexible solution for personalized federated learning in heterogeneous environments.