-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
I was debugging an issue with addIndexes
, turned on IndexWriter
's InfoStream
logging, and ConcurrentMergeScheduler
periodically logs juicy details like this:
MS 0 [2025-07-24T19:54:48.516458703Z; pool-4-thread-2]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
merge thread Lucene Merge Thread #19 estSize=1137.2 MB (written=287.1 MB) runTime=15.6s (stopped=0.0s, paused=0.0s) rate=unlimited
leave running at Infinity MB/sec
merge thread Lucene Merge Thread #20 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
leave running at Infinity MB/sec
merge thread Lucene Merge Thread #21 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
leave running at Infinity MB/sec
merge thread Lucene Merge Thread #22 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
leave running at Infinity MB/sec
The top merge (Lucene Merge Thread #19
) was a natural merge and it correctly sets the estSize
(OneMerge.estimatedMergeBytes
), here.
But the other merges all (incorrectly) show estSize=0
because their merges bypass IndexWriter.registerMerge
and instead register here (in AddIndexesMergeSource
).
This estimated size is useful for debugging ... e.g. segment tracing tools use it. Also, ConcurrentMergeScheduler
uses this when it throttles merges.
I think it should be a simple fix -- maybe factor out that code that computes the estimatedMergeSize
for natural merges today, so it can be shared for merges triggered by addIndexes
too?
Version and environment details
10.2.1 -- NOTE it is possible this has been fixed in 10.2.2 / 10.x -- I have not checked!