Skip to content

Add ElasticsearchIndexDeletionPolicy and EngineConfig policy wrapper #130442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Jul 2, 2025

This change adds a new abstract class ElasticsearchIndexDeletionPolicy that allows using different policy implementations for InternalEngine.

It also adds an index deletion policy wrapper to the EngineConfig to allow wrapping the default CombinedDeletionPolicy by inheritors of InternalEngine.

Relates ES-10929

@tlrx tlrx added >non-issue :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. labels Jul 2, 2025
@elasticsearchmachine elasticsearchmachine added the serverless-linked Added by automation, don't add manually label Jul 2, 2025
Copy link
Contributor

@kingherc kingherc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work. Some early comments.


public abstract boolean hasUnreferencedCommits();

public interface CommitsListener {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about getting rid of this interface since we don't use it anymore, wdyt @kingherc? we can do in a follow-up once all the pieces are in place.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fcofdez . Yes I believe it can be removed.

Separately, we were discussing whether IndexModule#setIndexCommitListener() could be removed in this thread with @tlrx , feel free to read to see as well. I had a question whether someone, e.g., externally, like a plugin writer user, might have been using it, but we think no. So that may also be considered for removal. However, maybe an optional follow-up.

@fcofdez fcofdez requested a review from kingherc July 16, 2025 13:08
@fcofdez fcofdez marked this pull request as ready for review July 16, 2025 13:08
@elasticsearchmachine elasticsearchmachine added the Team:Distributed Indexing Meta label for Distributed Indexing team label Jul 16, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing)

Copy link
Contributor

@kingherc kingherc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


public abstract boolean hasUnreferencedCommits();

public interface CommitsListener {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fcofdez . Yes I believe it can be removed.

Separately, we were discussing whether IndexModule#setIndexCommitListener() could be removed in this thread with @tlrx , feel free to read to see as well. I had a question whether someone, e.g., externally, like a plugin writer user, might have been using it, but we think no. So that may also be considered for removal. However, maybe an optional follow-up.

@@ -187,7 +180,7 @@ private void deleteCommit(IndexCommit commit) throws IOException {
assert commit.isDeleted() == false : "Index commit [" + commitDescription(commit) + "] is deleted twice";
logger.debug("Delete index commit [{}]", commitDescription(commit));
commit.delete();
assert commit.isDeleted() : "Deletion commit [" + commitDescription(commit) + "] was suppressed";
// assert commit.isDeleted() : "Deletion commit [" + commitDescription(commit) + "] was suppressed";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit Reminder to remove or adjust (maybe leave the assertion in stateful)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. >non-issue serverless-linked Added by automation, don't add manually Team:Distributed Indexing Meta label for Distributed Indexing team v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants