Will Brooks
TornButter
AI & ML interests
None yet
Recent Activity
reacted
to
MoritzLaurer's
post
with 🔥
2 days ago
The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
Read the release notes and other resources here 👇
Release: https://github.com/huggingface/trl/releases/tag/v0.13.0
Mergekit: https://github.com/arcee-ai/mergekit
Mixture of judges paper: https://huggingface.co/papers/2409.20370
liked
a model
3 days ago
hexgrad/Kokoro-82M
liked
a model
4 days ago
kudzueye/boreal-flux-dev-v2
Organizations
None yet
TornButter's activity
Not able to use this
5
#59 opened 4 months ago
by
Cutekameena612
Zephyr 7b 128k?
1
#31 opened about 1 year ago
by
TornButter
koboldcpp thinks it is a GPT-NEO-X model?
2
#3 opened over 1 year ago
by
TornButter
Vicuna 7B
4
#1 opened over 1 year ago
by
TornButter
filenames of shards in pytorch_model.bin.index.json
2
#4 opened over 1 year ago
by
h3ndrik
Error when launching
4
#5 opened over 1 year ago
by
pupdike
Is 64GB of RAM enough?
#1 opened over 2 years ago
by
TornButter
BLOOM models don't run on my GPU
1
#114 opened over 2 years ago
by
TornButter
BLOOM models don't run on my GPU
1
#114 opened over 2 years ago
by
TornButter
Is 64GB of RAM enough?
#1 opened over 2 years ago
by
TornButter