Prepare docs for the unified Redpanda Migrator

[JakeSCahill]

Description

Do not merge until 4.67.0 is released.

Resolves https://redpandadata.atlassian.net/browse/DOC-1671
Review deadline: October 17

This PR prepares changes to the migrator cookbook and adds a migration guide from the legacy migrator components to the new unified ones.

When 4.67.0 is released, a branch called auto-docs/update-rpcn-connector-docs is created and a PR is submitted against main to add reference docs for the new migrator.

When this happens, please:

1. Review both PRs together. Ensure the reference docs from the automated PR align with the conceptual and guide updates in this PR.

2. Add cross-links. In the automated PR, add links from the new migrator reference pages to the relevant guides introduced in this PR.

3. Check that the automated PR correctly references 4.67.0 and that links and headings in this PR remain valid.

Merge order:

1. Merge this PR first.

2. Then merge the automated reference PR, once validation checks pass.

After both are merged, verify that the docs site builds successfully and the new migrator docs appear under the correct section.

Page previews

Checks

• New feature
• Content gap
• Support Follow-up
• Small fix (typos, links, copyedits, etc)

[netlify]

✅ Deploy Preview for redpanda-connect ready!

Name	Link
🔨 Latest commit	8dc9d5e
🔍 Latest deploy log	https://app.netlify.com/projects/redpanda-connect/deploys/68e6619af03d6500084bca35
😎 Deploy Preview	https://deploy-preview-311--redpanda-connect.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough

Walkthrough

The navigation adds a cross-reference to a new guide: guides:migration:migrate-unified-redpanda-migrator.adoc. The cookbook page redpanda_migrator.adoc is updated to replace references to the bundled migrator with the unified Redpanda Migrator, revising terminology, configuration examples, component descriptions, flow steps, and notes. Config examples switch from redpanda_migrator_bundle.yaml to redpanda_migrator.yaml with label-based input/output pairing and updated mappings. The migration guide updates minimum/available versions to Redpanda Connect 4.67.0+ and marks legacy migrator components as deprecated in 4.67.0.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    actor User
    participant Migrator as Unified Redpanda Migrator
    participant Src as Source Kafka Cluster
    participant Dst as Destination Redpanda Cluster
    participant SR as Schema Registry (Src/Dst)
    participant ACL as ACL/Authorization Layer

    Note over Migrator: Config: input[label], output[label], topic & group mappings

    rect rgba(200,200,255,0.2)
    note right of Migrator: Initialization
    User->>Migrator: Start pipeline with redpanda_migrator.yaml
    Migrator->>Src: Connect (TLS/SASL)
    Migrator->>Dst: Connect (TLS/SASL)
    Migrator->>SR: Sync schemas (as configured)
    end

    rect rgba(200,255,200,0.2)
    note right of Migrator: Provisioning
    Migrator->>Dst: Create topics (from mappings)
    Migrator->>ACL: Ensure ACLs (if enabled)
    end

    rect rgba(255,255,200,0.2)
    note right of Migrator: Streaming & Translation
    loop For each topic/partition
        Migrator->>Src: Consume messages
        Migrator->>Dst: Produce messages (name interpolation)
    end
    Migrator->>Dst: Translate/commit offsets (exclude migrator group)
    end

    alt Success
        Migrator-->>User: Status: completed
    else Error
        Migrator-->>User: Status: error with diagnostics
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

• Jeffail
• rockwotj

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title clearly and concisely summarizes the main change in this pull request, indicating that documentation is being prepared for the unified Redpanda Migrator. It is specific to the primary focus of the changeset and avoids unnecessary detail or generic wording. A teammate scanning the history can understand the main intent without ambiguity.
Linked Issues Check	✅ Passed	This PR fully addresses the objectives of DOC-1671 by documenting the new unified Redpanda Migrator, updating legacy migrator references in the cookbook in place, and adding a dedicated migration guide while marking the old components as deprecated. The changes remove outdated bundle references and introduce the unified migrator flow and updated version requirements as specified in the linked issue.
Out of Scope Changes Check	✅ Passed	All modifications in this pull request are focused on updating or adding documentation related to the unified Redpanda Migrator as required by the linked issue and there are no unrelated or out-of-scope changes present.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Description Check	✅ Passed	The pull request description clearly outlines the purpose of the changes, references the related Jira issue, provides instructions for merging after the 4.67.0 release, and describes the coordination steps for cross-linking and validation, demonstrating a direct connection to the documented changes in the migrator guides and cookbooks.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Jira integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between a0e8195 and c5ad816.

📒 Files selected for processing (3)

• modules/ROOT/nav.adoc (1 hunks)
• modules/cookbooks/pages/redpanda_migrator.adoc (9 hunks)
• modules/guides/pages/migration/migrate-unified-redpanda-migrator.adoc (1 hunks)

[mmatczuk]

Components are paired using matching label fields for coordination. this is no longer mandatory that is to say if you have one input and output is works without explicitly setting labels.

[mmatczuk]

Changing topic configurations, such as partition count, isn’t currently supported. - it would be supported when we release. We may add it later.

[mmatczuk]

The max_in_flight: 1 setting is required to preserve message ordering at the partition level. - this is no longer max_in_flight setting.

[mmatczuk]

start_from_oldest: true is a deprecated flag

[mmatczuk]

topic: '${! @kafka_topic }' this can be removed unless you want to add a prefix or modify the destination topic

[mmatczuk]

exclude: [ "migrator" ] is not needed we now do that automatically - maybe worth noting