Skip to content

Latest commit

 

History

History
136 lines (99 loc) · 6.15 KB

peft.md

File metadata and controls

136 lines (99 loc) · 6.15 KB

LoRA (Low-Rank Adaptation):

  • Core Idea: LoRA introduces trainable low-rank matrices into the model, targeting specific weight matrices (e.g., attention layers). Instead of fine-tuning the full model, LoRA adds a low-rank decomposition to certain layers and fine-tunes only these low-rank matrices.

  • Implementation:

    • Adds two small low-rank matrices ( A ) and ( B ) to the original weights ( W ) of the model, where ( \Delta W = AB^T ).
    • The original pre-trained weights ( W ) remain frozen during training, and only ( A ) and ( B ) are updated.

  • Advantages:

    • Memory efficient: Fewer parameters are updated.
    • Doesn't require modifying the model architecture significantly.
    • Easy to integrate into transformer-based models.

  • Use Case:

    • Primarily used for fine-tuning language models (e.g., LLaMA, GPT).
    • Common in NLP and generative tasks.

Adapters:

  • Core Idea: Adapters are small trainable modules inserted into the layers of a pre-trained model. These modules are trained while keeping the original model weights frozen.

  • Implementation:

    • Adapters are typically lightweight neural network modules (e.g., feedforward layers) inserted between layers of a model, such as attention or feedforward blocks.
    • During training, only the parameters of the adapters are updated, while the rest of the model remains fixed.

  • Advantages:

    • Modular: Different tasks can have separate adapters without modifying the original model.
    • Memory efficient: Reduces the need to fine-tune the entire model.
    • Easy task-switching by replacing adapters.

  • Use Case:

    • Popular in multi-task learning and scenarios requiring task-specific fine-tuning.
    • Useful in both NLP and multimodal applications.

Key Differences:

Feature LoRA Adapters
Architecture Adds low-rank matrices to weight updates. Inserts small modules between layers.
Frozen Parameters Original model weights are frozen. Original model weights are frozen.
Parameter Updates Updates low-rank matrices ( A, B ). Updates adapter parameters only.
Overhead Minimal: modifies specific weight matrices. Moderate: introduces new modules into the model.
Use Cases NLP fine-tuning, generative tasks. Multi-task learning, task-specific adaptation.

Both methods are highly efficient and suitable for scenarios where training large models directly is infeasible. However, the choice depends on your use case, such as the number of tasks, modularity requirements, and computational constraints.

1. LoRA for Fine-Tuning

Advantages:

  1. Efficiency:
    • LoRA modifies only a small subset of trainable parameters (the low-rank matrices), making it highly parameter-efficient.
  2. Minimal Overhead:
    • No additional modules are inserted into the architecture; only the weight matrices of certain layers are modified.
  3. Better for Single Tasks:
    • Works well for scenarios where you want to fine-tune on a single or specific domain/task without modularity concerns.
  4. Speed:
    • Training and inference are faster compared to adapters due to the lightweight nature of LoRA’s modifications.

Use Cases:

  • Single-task fine-tuning.
  • Resource-constrained environments (e.g., limited GPU memory).
  • Generative tasks such as text generation or summarization.

Limitations:

  • Less modular: Difficult to manage multiple tasks or transfer fine-tuned components across models.
  • Potentially less effective when task-switching is required.

2. Adapters for Fine-Tuning

Advantages:

  1. Modularity:
    • Adapters are ideal for multi-task setups since each task can have its own adapter, enabling seamless task switching.
  2. Transfer Learning:
    • Adapters trained on one task can be reused or adapted for related tasks, making them versatile.
  3. Isolation:
    • Fine-tuning with adapters avoids interference between tasks, which is especially useful in multi-task or federated learning setups.

Use Cases:

  • Multi-task learning or scenarios requiring task switching.
  • Incremental fine-tuning on new tasks/domains.
  • Applications where modularity and reusability of components are important.

Limitations:

  • Higher computational overhead compared to LoRA due to added modules.
  • Slightly more complex integration into existing architectures.

Comparison Table:

Feature LoRA Adapters
Parameter Efficiency High Moderate
Modularity Low High
Fine-Tuning Overhead Minimal Moderate
Use Case Single-task fine-tuning Multi-task or modular setups
Scalability Limited for multiple tasks Scalable for multi-task setups
Inference Efficiency Higher Lower due to added modules

What’s Better?

  • LoRA is generally better if:

    • You are working on a single task or domain.
    • You prioritize efficiency and minimal computational overhead.
    • You have limited resources (e.g., GPU memory).

  • Adapters are better if:

    • You need to fine-tune on multiple tasks or domains.
    • Task modularity and reusability are important.
    • You want a framework that supports incremental learning.

Best Practices:

  • Experimentation: If resources allow, experiment with both methods to see which performs better for your specific task.
  • Hybrid Approaches: Recent work combines both methods for more efficient and effective fine-tuning.
  • Task-Specific Considerations: Consider the complexity of the task, the expected level of generalization, and memory constraints.