Reintroduce incremental checkpointing

[alessandropellegrini]

This PR reintroduces support for incremental checkpointing. Rather than basing it on the '09 DyMeLoR implementation, it is now based on the buddy system that is backing LP allocation.

There's no support for instrumentation, in line with the "core" approach: if incremental checkpointing is enabled but no __write_mem() calls are placed in the code, it's undefined behaviour and nothing will work.

There are some aspects to document related to the different checkpointing schemes we currently support. None of these are correctness issues, but are rather performance corner cases that should be documented.

Interaction with autonomic checkpointing

When both incremental_ckpt = true and ckpt_interval = 0 (autonomic mode), the autonomic formula uses checkpoint cost statistics that mix full and incremental checkpoints, leading to a biased estimate of the optimal interval.

A lower ckpt_avg_cost causes the current autonomic formula to compute a shorter optimal checkpoint interval (more frequent checkpoints). This is arguably desirable: incremental checkpoints should be cheap (although model dependent), so taking them more often should be efficient.

However, the model is imprecise: it treats the mix of full and incremental checkpoints as if they were a homogeneous cost, when in reality every full_ckpt_period-th checkpoint has a significantly higher cost.

This is am ok-ish side effect. The autonomic mechanism will converge to a workable interval. The bias is in the "correct" direction (more frequent checkpointing when checkpoints are cheap). A more precise model would account for the amortized cost of the periodic full checkpoint, but this is an optimization, not a correctness issue.

Fossil collection

When incremental checkpointing is active, model_allocator_fossil_lp_collect walks backward from the fossil collection target until it finds a full checkpoint. It retains that full checkpoint (and all incremental checkpoints after it) because the incremental chain needs its base.

If full_ckpt_period is large (e.g., 100), and the autonomic mechanism selects a short ckpt_interval (e.g., 5), then between two full checkpoints there are up to 99 incremental checkpoints. Fossil collection cannot free ANY of them until the next full checkpoint is committed. This creates a memory sawtooth: checkpoint memory grows during the incremental chain and is only reclaimed in bulk when the full checkpoint falls behind the GVT.

Users should be aware that full_ckpt_period controls the maximum length of the non-reclaimable checkpoint chain. A very high value increases peak memory usage. I'll try to document this in the upcoming months (years?).

Behaviour on frequent rollbacks

If an LP experiences frequent rollbacks, ckpt_since_last_full may never reach full_ckpt_period. This means full checkpoints are taken less often than the configured period, prolonging the incremental chain and delaying its release by fossil collection.

The autonomic mechanism registers each rollback as a bad event, which increases the rollback probability estimate and shortens the checkpoint interval. More frequent checkpoints + infrequent full checkpoints = a long chain of incremental checkpoints that cannot be fossil-collected.

In practice, LPs experiencing many rollbacks will eventually advance past a GVT boundary and receive a full checkpoint. But the worst case (high rollback rate + high full_ckpt_period

• short autonomic interval) could lead to unexpectedly high memory consumption from checkpoint storage.

[codecov]

Codecov Report

❌ Patch coverage is 95.83815% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.60%. Comparing base (d6568f6) to head (19fd504).

Files with missing lines	Patch %	Lines
src/mm/buddy/checkpoint.c	47.22%	19 Missing ⚠️
.../integration/correctness/application_incremental.c	92.62%	8 Missing and 1 partial ⚠️
src/mm/checkpoint/full.c	90.74%	4 Missing and 1 partial ⚠️
test/integration/phold_incremental.c	92.30%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #125      +/-   ##
===========================================
+ Coverage    85.51%   89.60%   +4.08%     
===========================================
  Files           46       49       +3     
  Lines         1553     2308     +755     
  Branches        39       40       +1     
===========================================
+ Hits          1328     2068     +740     
- Misses         209      223      +14     
- Partials        16       17       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Piccions]

Some issues to fix, but otherwise looks good!

[alessandropellegrini]

I should have addressed all the issues. One of them motivated me to write additional tests, so I'll push later once I'm done.