Skip to content

Commit 8ea5f07

Browse files
committed
Persist: clarify reduced semantics to refer only to spine structure
The exert function is called exclusively by the catalog force compaction code path. In the case that there was only one spine batch with multiple hollow batches, the exert function would incorrectly introduce empty SpineBatches. The behavior of the `reduced` function was always using the number of HollowBatches as a proxy for the structural shape of the spine. With incremental compaction, there might be only one run with a number of empty HollowBatches. This is totally legal, and not a reason to try and force structural merges.
1 parent 0608d97 commit 8ea5f07

File tree

1 file changed

+12
-6
lines changed

1 file changed

+12
-6
lines changed

src/persist-client/src/internal/trace.rs

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1744,15 +1744,21 @@ impl<T: Timestamp + Lattice> Spine<T> {
17441744
self.introduce_batch(batch, usize::cast_from(index.trailing_zeros()), log);
17451745
}
17461746

1747-
/// True iff there is at most one HollowBatch in `self.merging`.
1747+
/// Returns true when the trace is considered *structurally reduced*.
17481748
///
1749-
/// When true, there is no maintenance work to perform in the trace, other
1750-
/// than compaction. We do not yet have logic in place to determine if
1751-
/// compaction would improve a trace, so for now we are ignoring that.
1749+
/// Reduced == the total number of runs (across every
1750+
/// `SpineBatch` and all of their inner hollow batches) is < 2. In other
1751+
/// words, there are either zero runs (fully empty) or exactly one logical
1752+
/// run of data remaining.
17521753
fn reduced(&self) -> bool {
17531754
self.spine_batches()
1754-
.flat_map(|b| b.parts.as_slice())
1755-
.count()
1755+
.map(|b| {
1756+
b.parts
1757+
.iter()
1758+
.map(|p| p.batch.run_meta.len())
1759+
.sum::<usize>()
1760+
})
1761+
.sum::<usize>()
17561762
< 2
17571763
}
17581764

0 commit comments

Comments
 (0)