Federated Learning

Data ownership, populations and objectives

  • Federated setting - Clients train collaboratively while keeping raw training data decentralized. Kairouz et al. distinguish statistical, system and privacy challenges; retaining data locally does not prevent leakage through model updates. Kairouz et al. 2021.
  • Cross-device versus cross-silo - Many intermittently available devices and a small set of participating organizations impose different participation, trust and resource constraints. State the deployment regime before comparing federated algorithms. Kairouz et al. 2021.
  • Vertical federated learning - Parties hold different features for overlapping entities, rather than separate examples with a shared feature schema. Entity alignment, label ownership and intermediate-information leakage become central design questions. Liu et al. 2024.

Global-model optimization under heterogeneity

  • FedAvg - Selected clients train locally and the server averages their models, typically weighted by local example counts. More local work reduces synchronization frequency but can amplify differences between client objectives. McMahan et al. 2017.
  • FedProx - Adds a proximal penalty toward the current global model and permits variable local work. Li et al. analyze statistical and system heterogeneity; the penalty constrains local drift rather than making client data identical. Li et al. 2020.
  • SCAFFOLD - Uses control variates to correct client drift caused by heterogeneous local objectives. Karimireddy et al.’s convergence analysis distinguishes this correction from merely shortening each client’s local training. Karimireddy et al. 2020.
  • FedNova - Normalizes client updates to address objective inconsistency caused by unequal amounts of local computation. Its target is aggregation bias, distinct from feature shift or adversarial clients. Wang et al. 2020.
  • Adaptive federated optimization - FedAdagrad, FedAdam and FedYogi apply adaptive updates at the server while clients perform local optimization. Reddi et al. analyze nonconvex heterogeneous objectives; client and server learning rates play different roles. Reddi et al. 2021.

Client participation and asynchronous updates

  • Power-of-Choice selection - Prefers clients with high local loss. Cho et al. analyze a tradeoff between convergence speed and solution bias; selection changes the effective optimization process, so comparisons must state the target client weights. Cho et al. 2020.
  • Oort - Selects training clients using estimated data utility and execution speed to improve time to accuracy. Its testing selector separately targets a requested data distribution; fast training participation and representative evaluation are different objectives. Lai et al. 2021.
  • FedBuff - Buffers several asynchronous client updates before applying a server update. This reduces dependence on slow clients while allowing aggregated updates; buffer size, staleness and secure-aggregation protocol are distinct design choices. Nguyen et al. 2022.

Personalization and distributional objectives

  • MOCHA - Learns related client-specific linear predictors with a multi-task regularizer. Its distributed optimizer accommodates variable local work and dropped updates; the convex task losses and learned task relationships distinguish it from shared-model FedAvg. Smith et al. 2017.
  • Agnostic federated learning - Optimizes performance against mixtures of client distributions rather than only their observed average. Mohri et al. derive bounds and convex optimization results; robustness is relative to the specified mixture family. Mohri et al. 2019.
  • pFedMe - Uses Moreau-envelope objectives to separate personalized optimization from global learning. Dinh et al. formulate a bilevel problem; a shared initialization and a personalized final model serve different objectives. Dinh et al. 2020.
  • IFCA clustered federation - Alternates client-to-cluster assignments with training one model per cluster. Convergence results assume suitable initialization and structured losses; neural experiments test a broader setting than the strongly convex theory. Ghosh et al. 2020.
  • FedBN - Keeps batch-normalization components local while aggregating other model components. Li et al. target feature-distribution shifts across clients; local normalization is not a general solution to every form of non-IID data. Li et al. 2021.
  • Ditto - Learns personalized models regularized toward a global model. Li et al. study fairness across devices and poisoning robustness; these depend on the client population and threat model used in evaluation. Li et al. 2021.

Heterogeneous models and adapters

  • FedDF - Distills client-model predictions on unlabeled server data into a central model. This permits different client architectures where weight averaging is undefined; the distillation data and server computation are additional resources. Lin et al. 2020.
  • FLoRA - Stacks low-rank adapter factors to aggregate their products, allowing different client ranks. Averaging the factors separately introduces cross-client terms; exact update aggregation here does not imply privacy or robustness to malicious clients. Wang et al. 2024.

Privacy mechanisms and integrity risks

  • Secure aggregation - Bonawitz et al. let a server recover an aggregate without revealing individual client contributions under protocol assumptions. This protects individual updates; it does not itself limit information in the released aggregate or model. Bonawitz et al. 2017.
  • User-level differential privacy - McMahan et al. modify FedAvg with bounded client contributions, noise and privacy accounting. The protected unit is a user; record sampling and the privacy budget rather than labeling all federated training private. McMahan et al. 2018.
  • Robust gradient aggregation - Yin et al. analyze coordinate-wise medians and trimmed means against Byzantine workers. Guarantees assume independently sampled data from a common distribution; arbitrary non-IID client updates require additional analysis. Yin et al. 2018.
  • Gradient leakage - Zhu et al. reconstruct inputs from shared gradients in studied vision and language settings. This demonstrates an attack surface; success depends on the information exposed and does not imply recovery from every aggregated update. Zhu et al. 2019.
  • Model-replacement backdoors - Bagdasaryan et al. show malicious clients can implant targeted behavior through submitted updates. Confidential aggregation and resistance to poisoning address different threats; one does not establish the other. Bagdasaryan et al. 2020.

Benchmarks and evaluation

  • LEAF - Provides federated datasets, reference implementations and an evaluation framework emphasizing client heterogeneity. Real client partitions test different conditions from randomly dividing a centralized dataset. Caldas et al. 2018.
  • FedScale - Combines realistic client datasets with a runtime for system-aware FL evaluation. Lai et al. benchmark model and system performance together; communication rounds alone omit client speed and availability effects. Lai et al. 2022.
  • Motley personalization benchmark - FedAvg plus fine-tuning has the best mean client accuracy on three of four cross-device datasets. Yet fine-tuning hurts 40% of clients on TedMulti-EnEs; average improvement and individual benefit are separate outcomes. Wu et al. 2022.

Software

  • Flower - Framework for federated experiments across simulated and heterogeneous real clients. Supports studying deployment differences; a simulation result does not measure performance on the intended device fleet. Beutel et al. 2020.
  • TensorFlow Federated - Provides federated computations and learning algorithms with simulation support. Client-data partitioning and aggregation are explicit components; privacy guarantees require an appropriate configured mechanism. docs.