Post your loop here and I'll help you design the error signal #8
Pinned
fitz2882
started this conversation in
Error Signals
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
The one thing LoopGain needs from you is a single non-negative number per iteration that says how wrong the current output is. Lower is better, zero means done. Everything else is derived from that number's trajectory.
That's the whole integration — and it's also the part people get stuck on. So: describe your loop in a reply and I'll suggest a signal. No format required. "I have a RAG pipeline that re-retrieves when the answer looks incomplete" is enough to work with.
Starting points, if yours resembles one of these:
10 − quality_scoreobserve()takes a number or any sequence — hand it the raw list of failing tests and it uses the length. So often there's no conversion work at all.Two rules, and one escape hatch:
target_error=None. LoopGain then stops when the number stops improving, wherever that plateau lands, instead of waiting for a target that never arrives.The failure mode to avoid is a signal that's noisy but doesn't actually track quality — a judge score that wobbles ±2 on identical output will read as oscillation. If you're unsure whether yours is stable enough, say so in your reply and we'll look at it.
All reactions