FedLGT (Language-Guided Transformer for Federated Multi-Label Classification)
Paper: I-Jieh Liu et al., AAAI 2024
Intuition:
FedLGT is designed to handle multi-label image classification in federated learning settings, which is more challenging than single-label classification due to partial label correlations observed by local clients. This algorithm leverages the power of transformers to transfer knowledge across different clients, improving the performance of the global model even when dealing with heterogeneous data distributions (GitHub).
FedCompass (Efficient Cross-Silo Federated Learning)
Paper: Presented at ICLR 2024
Intuition:
FedCompass introduces a semi-asynchronous federated learning approach that aims to address the inefficiencies of both synchronous and asynchronous methods. It uses a computing power-aware scheduler (COMPASS) to adaptively assign local training steps to clients based on their computational capabilities. This method helps synchronize the arrival of client models, reducing staleness and improving overall efficiency and performance (GitHub).
FedDyn (Federated Dynamics)
Overview:
FedDyn is noted for its dynamic adjustment of learning rates and model parameters during the training process. This algorithm aims to balance computational and communication efficiency while maintaining high accuracy. It adapts to client heterogeneity and data distribution changes dynamically, making it robust for various federated learning scenarios.
FedYogi and FedAdam
Overview:
These algorithms adapt techniques from centralized optimization (like the Adam optimizer) to federated settings. They aim to improve convergence rates and accuracy by adjusting learning rates based on gradient updates. They tend to perform better than traditional methods like FedAvg, especially in non-IID data settings (Papers with Code) .
FOCUS (Federated Online Client Update Scheduling)
Overview:
FOCUS schedules client updates dynamically to minimize communication overhead while maximizing learning efficiency. It adapts the training process in real-time, based on the current state of the client models and network conditions, improving both convergence speed and model accuracy.
These newer algorithms show significant improvements over traditional federated learning methods, particularly in handling client and data heterogeneity, improving convergence rates, and maintaining model performance. Each algorithm brings unique advantages depending on the specific requirements and constraints of the federated learning setup.
FedLGT (Language-Guided Transformer for Federated Multi-Label Classification)
Paper: I-Jieh Liu et al., AAAI 2024
Intuition:
FedLGT is designed to handle multi-label image classification in federated learning settings, which is more challenging than single-label classification due to partial label correlations observed by local clients. This algorithm leverages the power of transformers to transfer knowledge across different clients, improving the performance of the global model even when dealing with heterogeneous data distributions (GitHub).
FedCompass (Efficient Cross-Silo Federated Learning)
Paper: Presented at ICLR 2024
Intuition:
FedCompass introduces a semi-asynchronous federated learning approach that aims to address the inefficiencies of both synchronous and asynchronous methods. It uses a computing power-aware scheduler (COMPASS) to adaptively assign local training steps to clients based on their computational capabilities. This method helps synchronize the arrival of client models, reducing staleness and improving overall efficiency and performance (GitHub).
FedDyn (Federated Dynamics)
Overview:
FedDyn is noted for its dynamic adjustment of learning rates and model parameters during the training process. This algorithm aims to balance computational and communication efficiency while maintaining high accuracy. It adapts to client heterogeneity and data distribution changes dynamically, making it robust for various federated learning scenarios.
FedYogi and FedAdam
Overview:
These algorithms adapt techniques from centralized optimization (like the Adam optimizer) to federated settings. They aim to improve convergence rates and accuracy by adjusting learning rates based on gradient updates. They tend to perform better than traditional methods like FedAvg, especially in non-IID data settings (Papers with Code) .
FOCUS (Federated Online Client Update Scheduling)
Overview:
FOCUS schedules client updates dynamically to minimize communication overhead while maximizing learning efficiency. It adapts the training process in real-time, based on the current state of the client models and network conditions, improving both convergence speed and model accuracy.
These newer algorithms show significant improvements over traditional federated learning methods, particularly in handling client and data heterogeneity, improving convergence rates, and maintaining model performance. Each algorithm brings unique advantages depending on the specific requirements and constraints of the federated learning setup.