Skip to content

Merges kicked off by IndexWriter.addIndexes(CodecReader[]) fail to set estimated size #14993

@mikemccand

Description

@mikemccand

I was debugging an issue with addIndexes, turned on IndexWriter's InfoStream logging, and ConcurrentMergeScheduler periodically logs juicy details like this:

MS 0 [2025-07-24T19:54:48.516458703Z; pool-4-thread-2]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
merge thread Lucene Merge Thread #19 estSize=1137.2 MB (written=287.1 MB) runTime=15.6s (stopped=0.0s, paused=0.0s) rate=unlimited
  leave running at Infinity MB/sec
merge thread Lucene Merge Thread #20 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
  leave running at Infinity MB/sec
merge thread Lucene Merge Thread #21 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
  leave running at Infinity MB/sec
merge thread Lucene Merge Thread #22 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
  leave running at Infinity MB/sec

The top merge (Lucene Merge Thread #19) was a natural merge and it correctly sets the estSize (OneMerge.estimatedMergeBytes), here.

But the other merges all (incorrectly) show estSize=0 because their merges bypass IndexWriter.registerMerge and instead register here (in AddIndexesMergeSource).

This estimated size is useful for debugging ... e.g. segment tracing tools use it. Also, ConcurrentMergeScheduler uses this when it throttles merges.

I think it should be a simple fix -- maybe factor out that code that computes the estimatedMergeSize for natural merges today, so it can be shared for merges triggered by addIndexes too?

Version and environment details

10.2.1 -- NOTE it is possible this has been fixed in 10.2.2 / 10.x -- I have not checked!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions