Skip to content

Optimize Kafka Connect operations for large numbers of connectors #1334

@germanosin

Description

@germanosin

Issue submitter TODO list

  • I've searched for an already existing issues here
  • I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

No

Describe the feature you're interested in

Currently, when requesting Kafka Connect statuses or lists, multiple requests are made per connector to enrich the information. Since KIP-465 introduced a unified endpoint that provides this data in a single query, we should leverage it to improve performance.
To implement this:

  • Split logic into a versioned service.
  • Collect version statistics in memory (similar to clusters).
  • Use the unified query to gather connector and task information.

Describe alternatives you've considered

No

Version you're running

1.3

Additional context

No response

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions