Update contract changes for KIP-848 regex subscription

[pranavrth]

No description provided.

[confluent-cla-assistant]

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

[Copilot]

Pull request overview

This PR documents contract changes related to KIP-848's regex subscription handling, specifically the shift from client-side to broker-side regex matching with a different regex engine.

• Adds documentation for regex subscription behavior changes in the KIP-848 consumer protocol
• Documents the migration from libc regex engine (client-side) to Google RE2/J engine (broker-side)
• Provides a concrete example of regex pattern incompatibility between protocols

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

[nitpick] The sentence structure is unclear and could be improved for better readability. Consider rephrasing to: 'A known incompatibility case: Given topics topic-1 and topic-2, the pattern ^topic* works in the classic protocol but fails in the consumer protocol. Use ^topic.* instead.'

Suggested change

	Known case which would fail with the new protocol - We have `topic-1` and `topic-2`. If we subscribe with `^topic*` in the `classic` protocol, it works but it won't work in the `consumer` protocol. Use `^topic.*` instead.
	A known incompatibility case: Given topics `topic-1` and `topic-2`, the pattern `^topic*` works in the `classic` protocol but fails in the `consumer` protocol. Use `^topic.*` instead.

[emasab]

Thanks for the clarification I proposed some changes.
There's also a change to the example regex ^topic given using ^topic* is also matching topi and we don't want to add this element that creates confusion about the proposed example change.

[emasab]

Suggested change

Known case which would fail with the new protocol - We have `topic-1` and `topic-2`. If we subscribe with `^topic*` in the `classic` protocol, it works but it won't work in the `consumer` protocol. Use `^topic.*` instead.

A known case which would fail with the new protocol: we have topics `topic-1` and `topic-2`. If we subscribe with `^topic` in the `classic` protocol, it's assigned partitions from both topics but no partitions is assigned with the `consumer` protocol. That's because the broker side regex implementation as well as the Java classic protocol one require that regexes match the complete topic while `libc` one finds the pattern into the topic, that could be present as a prefix. To obtain the same result with this example you'd use `^topic.*` instead.