Log exceptions escaping OnPagesClosedWorker

[Tiago Nápoli (tiagonapoli)]

Summary

Adds logging for exceptions escaping OnPagesClosedWorker. Observability only — no behavior change. Catch, log via the allocator's existing logger, rethrow.

Why

OngoingCloseUntilAddress is a work-ownership token. From OnPagesClosed:

if (_ongoingCloseUntilAddress >= newSafeHeadAddress) break;   // "someone owns my range"
if (CAS(ref OngoingCloseUntilAddress, newSafeHeadAddress, _ongoing) == _ongoing) {
    if (_ongoingCloseUntilAddress == 0)   // worker spawned ONLY on the 0 -> non-zero transition
        OnPagesClosedWorker();
    return;
}

A worker is spawned only on the 0 → non-zero transition, and the only normal exit from the worker loop is CAS(ref OngoingCloseUntilAddress, 0, closeEndAddress). So an exception unwinding the worker leaves the token non-zero with no live worker — a state the pipeline cannot leave. Every later OnPagesClosed sees a non-zero token, assumes an owner is running, extends the token and starts nothing. ClosedUntilAddress never advances again, and the unbounded while (...) Thread.Yield() waits depending on it spin at 100% of a core indefinitely.

Evidence

A node had a thread pegged inside ResetCore's wait for ClosedUntilAddress during a replica attach:

OS Thread Id: 0x1a1cd

System.Threading.Thread.YieldInternal()
AllocatorBase`2+<>c__DisplayClass54_2[...].<ResetCore>b__1()   Allocator/AllocatorBase.cs @ 407
LightEpoch.Drain(Int64)                                        Epochs/LightEpoch.cs @ 497
LightEpoch.BumpCurrentEpoch(System.Action)                     Epochs/LightEpoch.cs @ 430
AllocatorBase`2[...].ResetCore()                               Allocator/AllocatorBase.cs @ 389
AllocatorBase`2[...].Reset()                                   Allocator/AllocatorBase.cs @ 331
TsavoriteKV`2[...].Reset()                                     Index/Recovery/Recovery.cs @ 419
DatabaseManagerBase.ResetDatabase(GarnetDatabase)              libs/server/Databases/DatabaseManagerBase.cs @ 270
StoreWrapper.Reset(Int32)                                      libs/server/StoreWrapper.cs @ 537
ReplicationManager...g__ReplicaSyncAttachTaskAsync|0>d.MoveNext()   libs/cluster/.../ReplicaDiskbasedSync.cs @ 146

Allocator state from the dump: ClosedUntilAddress = 16,777,216 (frozen), OngoingCloseUntilAddress = 2,031,377,992, SafeHeadAddress = 2,031,377,992 — 1.88 GiB of log that could never be closed.

The logs from that node contain no Tsavorite entries at all, so which exception killed the worker is unknowable after the fact. That is the gap this PR closes.

[Copilot]

Pull request overview

Adds allocator-level logging for unhandled exceptions escaping the OnPagesClosedWorker background pipeline, improving Tsavorite observability of unrecoverable “page closing stalled” failures without altering control flow (exceptions are still rethrown).

Changes:

• Wrap OnPagesClosedWorker in a try/catch that logs a critical message with key allocator state and rethrows.
• Extract the existing worker loop into OnPagesClosedWorkerCore() to keep the wrapper minimal.