Skip to content

Conversation

@tvaron3
Copy link
Member

@tvaron3 tvaron3 commented Sep 29, 2025

Description

Change feed processor will now stop renewing a lease if after 25 * the lease renewal interval there haven't been any results processed. This should allow for another worker to pick up this lease.

Implementation

PartitionSupervisorImpl already has the logic for stopping the renew and processing threads for a partition for exceptions. This was extended to stop both the processing and renewing threads when a change hasn't being processed after 25 * the lease renewal interval. The last processed time tracked is taking after we receive a response from change feed.

jeet1995 and others added 30 commits September 11, 2025 21:51
* add cfp logs

* react to comments

* react to comments

* react to comments

* react to comments

* add log line for release errors
…edRangeCacheRefresh

[Spark]SupportSystemPropertyForFeedRangeCacheRefreshInterval
…range read feed (Azure#21)

* Added a way to limit prefetch and page size for ReadFeed of PartitionKeyRange.

* Added additional logging.

* Added additional logging.

* Added additional logging.

* DOUBLE CHECK IMPACT - comment out feed range refresh interval setting

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Disable fault injection CFP test.

* Addressing review comments.
…geCacheForCFP

AddWireup using partition key range cache for cfp
…zure#23)

* Added a way to limit prefetch and page size for ReadFeed of PartitionKeyRange.

* Added additional logging.

* Added additional logging.

* Added additional logging.

* DOUBLE CHECK IMPACT - comment out feed range refresh interval setting

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Addressing review comments.

* Disable fault injection CFP test.

* Addressing review comments.

* Revert Spark test fixes.
…Planning-dev

improvePerfDuringPartitionPlanning
@tvaron3
Copy link
Member Author

tvaron3 commented Sep 29, 2025

/azp run java - cosmos - spark

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

2 similar comments
@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

}

return this.partitionKeyRangeCache.tryLookupAsync(BridgeInternal.getMetaDataDiagnosticContext(request.requestContext.cosmosDiagnostics), documentCollectionValueHolder.v.getResourceId(), null, null)
StringBuilder sb = new StringBuilder("RxDocumentClientImpl.createDocumentInternal:").append(",");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think we want to check in the log change in master, maybe have one PR only includes the defensive mechanism change?

@tvaron3
Copy link
Member Author

tvaron3 commented Sep 30, 2025

/azp run java - cosmos - tests

@tvaron3
Copy link
Member Author

tvaron3 commented Sep 30, 2025

/azp run java - cosmos - kafka

@tvaron3
Copy link
Member Author

tvaron3 commented Sep 30, 2025

/azp run java - cosmos - spark

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

2 similar comments
@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tvaron3
Copy link
Member Author

tvaron3 commented Oct 1, 2025

/azp run java - cosmos - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

1 similar comment
@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tvaron3
Copy link
Member Author

tvaron3 commented Oct 1, 2025

/azp run java - cosmos - kafka

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tvaron3
Copy link
Member Author

tvaron3 commented Oct 1, 2025

/azp run java - cosmos - spark

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tvaron3 tvaron3 closed this Oct 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants