This repository is primarily based on our survey paper 📚🔍:
Synergizing Foundation Models and Federated Learning: A Survey
Unlike smaller models, Foundation Models (FMs), such as LLMs and VLMs, are built upon vast amounts of training data 📊. While general FMs can use public data, domain-specific FMs require proprietary data for pre-training and fine-tuning, raising privacy concerns 🔒. Federated Learning (FL) 🤝💻, a compelling privacy-preserving approach, enables collaborative learning across distributed datasets while maintaining data privacy🛡️. Synergizing FM and FL offers a promising way to address data availability and privacy challenges in FM development, potentially revolutionizing large-scale machine learning in sensitive domains.
🙏If you find this survey useful for your research, please consider citing:
@misc{li2024synergizing,
title={Synergizing Foundation Models and Federated Learning: A Survey},
author={Shenghui Li and Fanghua Ye and Meng Fang and Jiaxu Zhao and Yun-Hin Chan and Edith C. -H. Ngai and Thiemo Voigt},
year={2024},
eprint={2406.12844},
archivePrefix={arXiv}
}
Table of Contents
| Title | Venue | Year | GitHub |
|---|---|---|---|
| FLM-TopK: Expediting Federated Large Language Model Tuning by Sparsifying Intervalized Gradients | INFOCOM | 2025-06 | |
| Federated Adaptive Fine-Tuning of Large Language Models with Heterogeneous Quantization and LoRA | INFOCOM | 2025-05 | |
| Promoting Data and Model Privacy in Federated Learning through Quantized LoRA | EMNLP | 2024-11 |
| Title | Venue | Year | GitHub |
|---|---|---|---|
| FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF | AAMAS | 2025-05 | |
| Towards Federated RLHF with Aggregated Client Preference for LLMs | ICLR | 2025-04 |  |
| Federated Fine-Tuning of Large Language Models: Kahneman-Tversky vs. Direct Preference Optimization | arXiv | 2025-02 |
| Title | Venue | Year | GitHub |
|---|---|---|---|
| FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models | ICML | 2024-07 | |
| Personalized Federated Learning for Text Classification with Gradient-Free Prompt Tuning | NAACL | 2024-06 | |
| ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning | arXiv | 2023-10 | |
| Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models | arXiv | 2023-10 |
| Title | Venue | Year | GitHub |
|---|---|---|---|
| Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks | ICASSP | 2024-03 | |
| Joint Federated Learning and Personalization for on-Device ASR | ASRU | 2023-12 | |
| Importance of Smoothness Induced by Optimizers in Fl4Asr: Towards Understanding Federated Learning for End-To-End ASR | ASRU | 2023-12 | |
| Federated Learning for Speech Recognition: Revisiting Current Trends Towards Large-Scale ASR | FL@FM-NeurIPS | 2023-12 |
| Title | Venue | Year | GitHub | Developed by |
|---|---|---|---|---|
| FederatedScope: A Flexible Federated Learning Platform for Heterogeneity | VLDB | 2023-09 |  |
 |
| FedLab: A Flexible Federated Learning Framework | JMLR | 2023-01 |  |
 |
| OpenFL: the open federated learning library | PMB | 2022-10 |  |
 |
| NVIDIA FLARE: Federated Learning from Simulation to Real-World | FL@NeurIPS | 2022-07 |  |
 |
| FedScale: Benchmarking Model and System Performance of Federated Learning at Scale | ICML | 2022-07 |  |
 |
| Scalable federated machine learning with FEDn | CCGrid | 2022-05 |  |
 |
| FLUTE: A Scalable Extensible Framework for High-Performance Federated Learning Simulations | FL@NeurIPS | 2022-03 |  |
 |
| FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection | JMLR | 2021-08 |  |
 |
| Pysyft: A library for easy federated learning | FLS | 2021-06 |  |
 |
| Flower: A friendly federated learning research framework | Arxiv | 2020-07 |  |
 |
| FedML: A Research Library and Benchmark for Federated Machine Learning | SpicyFL | 2020-07 |  |
 |
