feat(ticdc): prevent using the same TiDB cluster as both upstream and downstream

[wlwilliamx]

What problem does this PR solve?

Issue Number: close #12062, #11767

What is changed and how it works?

Overview

This PR implements a mechanism to prevent TiCDC from using the same TiDB cluster as both upstream and downstream. The approach relies on retrieving the Cluster ID from mysql.tidb in TiDB and comparing it with the upstream Cluster ID obtained from PD.

Changes in This PR

• Fetch the upstream Cluster ID from PD via gRPC.
• Query the downstream Cluster ID from TiDB’s mysql.tidb.
• Compare the Cluster IDs during changefeed creating, resuming and updating operation.
• Reject changefeed operation if the upstream and downstream Cluster IDs match.
• Add error handling and logging.

Check List

Tests

• Manual test (add detailed scripts or steps below)

1. Create a changefeed using cmd: cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri="mysql://root:@127.0.0.1:8300/" --changefeed-id="create-cmd", results:

   Error: [CDC:ErrSameUpstreamDownstream]TiCDC does not support creating a changefeed with the same TiDB cluster as both the source and the target for the changefeed.
   

2. Create a changefeed using Open API V1: curl -X POST -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v1/changefeeds -d '{"changefeed_id":"create-v1","sink_uri":"mysql://root:@127.0.0.1:4000/"}', results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support creating a changefeed with the same TiDB cluster as both the source and the target for the changefeed.",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

3. Create a changefeed using Open API V2: curl -X POST -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v2/changefeeds -d '{"changefeed_id":"create-v2","sink_uri":"mysql://root:@127.0.0.1:4000/"}', results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support creating a changefeed with the same TiDB cluster as both the source and the target for the changefeed.",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

4. Update a changefeed using cmd: cdc cli changefeed update --changefeed-id update-cmd --sink-uri="mysql://root:@127.0.0.1:4000/", results:

   Error: [CDC:ErrSameUpstreamDownstream]TiCDC does not support updating a changefeed with the same TiDB cluster as both the source and the target for the changefeed.
   

5. Update a changefeed using Open API V1: curl -X PUT -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v1/changefeeds/update-v1 -d '{"sink_uri":"mysql://root:@127.0.0.1:4000/"}', results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support updating a changefeed with the same TiDB cluster as both the source and the target for the changefeed",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

6. Update a changefeed using Open API V2: curl -X PUT -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v2/changefeeds/update-v2 -d '{"sink_uri":"mysql://root:@127.0.0.1:4000/"}', results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support updating a changefeed with the same TiDB cluster as both the source and the target for the changefeed",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

7. Resume a changefeed using cmd: cdc cli changefeed resume --changefeed-id resume-cmd, results:

   Error: [CDC:ErrSameUpstreamDownstream]TiCDC does not support resuming a changefeed with the same TiDB cluster as both the source and the target for the changefeed.
   

8. Resume a changefeed using Open API V1: curl -X POST http://127.0.0.1:8300/api/v1/changefeeds/resume-v1/resume, results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support resuming a changefeed with the same TiDB cluster as both the source and the target for the changefeed.",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

9. Resume a changefeed using Open API V2: curl -X POST http://127.0.0.1:8300/api/v2/changefeeds/resume-v2/resume -d '{}', results:

   {
       "error_msg": "[CDC:ErrSameUpstreamDownstream]TiCDC does not support resuming a changefeed with the same TiDB cluster as both the source and the target for the changefeed.",
       "error_code": "CDC:ErrSameUpstreamDownstream"
   }
   

Questions

Will it cause performance regression or break compatibility?

No.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note

TiCDC now prevents using the same TiDB cluster as both upstream and downstream, ensuring data consistency and avoiding replication loops.  

[wlwilliamx]

/check-issue-triage-complete

[wlwilliamx]

/CC @benmeadowcroft

[ti-chi-bot]

@wlwilliamx: GitHub didn't allow me to request PR reviews from the following users: benmeadowcroft.

Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/CC @benmeadowcroft

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[wlwilliamx]

/CC @lidezhu

[wlwilliamx]

/retest

[wlwilliamx]

/retest

[wlwilliamx]

/test pull-cdc-integration-mysql-test

[wlwilliamx]

/test pull-cdc-integration-mysql-test

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2025-02-16 22:51:48.946639445 +0000 UTC m=+828951.342861507: ☑️ agreed by benmeadowcroft.
• 2025-02-17 10:31:38.824144801 +0000 UTC m=+870941.220366863: ☑️ agreed by 3AceShowHand.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand, asddongmen, benmeadowcroft

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [3AceShowHand,asddongmen]
• cdc/api/v2/OWNERS [benmeadowcroft]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[wlwilliamx]

/retest