Add missing endpoints (/schemas, /subjects) to SchemaRegistryClient #2017

fangnx · 2025-08-08T21:32:13Z

What

Add the missing endpoints and update the existing ones (some have missing/incorrect fields) to SR client. This PR is only addressing endpoints with /schemas and /subjects prefixes. I will start a follow-up PR to add the rest.

Checklist

Contains customer facing changes? Including API/behavior changes
Did you add sufficient unit test and/or integration test coverage for this PR?
- If not, please explain why it is not required

References

JIRA: https://confluentinc.atlassian.net/browse/DGS-21591

Issues: #1429

Test & Review

Open questions / Follow-ups

confluent-cla-assistant · 2025-08-08T21:32:23Z

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

MSeal

Minor suggestions, looks clean and ready as you can choose which of those comments to apply

MSeal · 2025-08-12T23:15:02Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            `GET Schema API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--schemas-ids-int-%20id-schema>`_
+        """  # noqa: E501
+
+        query = {'subject': subject_name} if subject_name is not None else None


It can be slightly cleaner to do else {} or two lines with query = {} and if subject_name: so you don't have to do the if query is not None later on

Good suggestion! Refactored

MSeal · 2025-08-12T23:17:54Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects-(string-%20subject)-versions-(versionId-%20version)-schema>`_
+        """  # noqa: E501
+
+        query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}


Interesting that deleted isn't in the doc parameters. Do we need to update to docs page later on or is this intentionally a hidden field?

Yes, I will make sure the docs are updated

MSeal · 2025-08-12T23:22:17Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

+            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects-(string-%20subject)-versions-(versionId-%20version)-schema>`_
+        """  # noqa: E501
+
+        query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}


Same here on deleted not being in docs reference, are there other routes we need to add deleted to that have it missing?

MSeal · 2025-08-12T23:23:44Z

src/confluent_kafka/schema_registry/common/schema_registry_client.py

+
+    @classmethod
+    def from_dict(cls: Type[T], src_dict: Dict[str, Any]) -> T:
+        d = src_dict.copy()


Kinda unnecessary, this is a shallow copy action that we immediately throw away. Could just make this a one-liner of return cls(subject=src_dict.get('subject'), version=src_dict.get('version'))

Copilot

Pull Request Overview

This PR adds missing endpoints to the SchemaRegistryClient for /schemas and /subjects prefixes, expanding the available functionality for interacting with the Schema Registry. The implementation also corrects the return type of the get_schema method from Schema to RegisteredSchema which is a breaking change that requires updates to existing client code.

Added new endpoints: get_schema_string, get_schema_types, get_schema_versions, get_version_schema_string, and get_referenced_by
Enhanced existing endpoints with additional optional parameters (deleted, subject_prefix)
Updated return type of get_schema from Schema to RegisteredSchema

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/schema_registry/conftest.py	Added mock implementations and regex patterns for new endpoints
tests/schema_registry/_sync/test_api_client.py	Added unit tests for new endpoints and updated existing tests for return type changes
tests/schema_registry/_async/test_api_client.py	Added async unit tests for new endpoints and updated existing tests for return type changes
tests/integration/schema_registry/_sync/test_proto_serializers.py	Updated test to access schema through `.schema` property due to return type change
tests/integration/schema_registry/_async/test_proto_serializers.py	Updated async test to access schema through `.schema` property due to return type change
src/confluent_kafka/schema_registry/common/schema_registry_client.py	Added SchemaVersion class and refactored RegisteredSchema field ordering
src/confluent_kafka/schema_registry/_sync/schema_registry_client.py	Implemented new endpoints and updated existing methods with additional parameters
src/confluent_kafka/schema_registry/_async/schema_registry_client.py	Implemented async versions of new endpoints and updated existing methods
DEVELOPER.md	Removed trailing whitespace

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-14T21:06:31Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

+            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects-(string-%20subject)-versions-(versionId-%20version)-schema>`_
+        """  # noqa: E501
+
+        query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}


[nitpick] This conditional query construction pattern is inconsistent with the cleaner approach used in other methods. Consider using the same pattern as in get_schema_string where the query is built incrementally.

Suggested change

query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}

query = {'deleted': deleted}

if fmt is not None:

query['format'] = fmt

Copilot · 2025-08-14T21:06:32Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects-(string-%20subject)-versions-(versionId-%20version)-schema>`_
+        """  # noqa: E501
+
+        query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}


[nitpick] This conditional query construction pattern is inconsistent with the cleaner approach used in other methods. Consider using the same pattern as in get_schema_string where the query is built incrementally.

Suggested change

query = {'deleted': deleted, 'format': fmt} if fmt is not None else {'deleted': deleted}

query = {'deleted': deleted}

if fmt is not None:

query['format'] = fmt

rayokota

Thanks @fangnx . Actually let's revert changing the Schema return type to RegisteredSchema as that's a breaking API change

Copilot

Pull Request Overview

This PR adds missing endpoints to the SchemaRegistryClient for both schemas and subjects APIs, expanding the functionality to support additional Schema Registry operations. The implementation includes new endpoints for retrieving schema metadata, versions, types, and references.

Add new /schemas endpoints: /schemas/ids/{id}/schema, /schemas/ids/{id}/versions, /schemas/ids/{id}/subjects, and /schemas/types
Add new /subjects endpoints: /subjects/{subject}/versions/{version}/schema and /subjects/{subject}/versions/{version}/referencedby
Update existing endpoints with additional parameters and improved query parameter handling
Add comprehensive test coverage for all new endpoints in both sync and async clients

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/schema_registry/conftest.py	Add mock handlers and regex patterns for new schema and subject endpoints
tests/schema_registry/_sync/test_api_client.py	Add test cases for new sync client endpoints with proper assertions
tests/schema_registry/_async/test_api_client.py	Add test cases for new async client endpoints mirroring sync tests
src/confluent_kafka/schema_registry/common/schema_registry_client.py	Add SchemaVersion data class and simplify SchemaReference.from_dict implementation
src/confluent_kafka/schema_registry/_sync/schema_registry_client.py	Implement new endpoints and update existing ones with enhanced parameter support
src/confluent_kafka/schema_registry/_async/schema_registry_client.py	Mirror sync implementation for async client with proper await handling
DEVELOPER.md	Remove trailing whitespace

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-18T19:56:31Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

+
+        TODO: add API reference
+        """
+        query = {'offset': offset, 'limit': limit}


The query parameters are being added unconditionally, but the limit parameter should only be added if it's not negative according to the docstring. This could result in unexpected behavior when limit is -1.

Suggested change

query = {'offset': offset, 'limit': limit}

query = {'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:32Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

+            `GET Schema Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--schemas-ids-int-%20id-versions>`_
+        """  # noqa: E501
+
+        query = {'offset': offset, 'limit': limit}


Same issue as above - the limit parameter should only be added to the query if it's not negative, as stated in the docstring.

Suggested change

query = {'offset': offset, 'limit': limit}

query = {'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:32Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

@@ -805,7 +943,10 @@ def get_subjects(self) -> List[str]:
            `GET subjects API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects>`_
        """  # noqa: E501

-        return self._rest_client.get('subjects')
+        query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}


The limit parameter should only be added to the query if it's not negative, as documented in the method's docstring.

Suggested change

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:32Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

@@ -970,7 +1174,8 @@ def get_versions(self, subject_name: str) -> List[int]:
            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#post--subjects-(string-%20subject)-versions>`_
        """  # noqa: E501

-        return self._rest_client.get('subjects/{}/versions'.format(_urlencode(subject_name)))
+        query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}


The limit parameter should only be added to the query if it's not negative, as documented in the method's docstring.

Suggested change

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:33Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+
+        TODO: add API reference
+        """
+        query = {'offset': offset, 'limit': limit}


The query parameters are being added unconditionally, but the limit parameter should only be added if it's not negative according to the docstring.

Suggested change

query = {'offset': offset, 'limit': limit}

query = {'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:33Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            `GET Schema Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--schemas-ids-int-%20id-versions>`_
+        """  # noqa: E501
+
+        query = {'offset': offset, 'limit': limit}


Same issue as above - the limit parameter should only be added to the query if it's not negative, as stated in the docstring.

Suggested change

query = {'offset': offset, 'limit': limit}

query = {'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:33Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

@@ -805,7 +943,10 @@ async def get_subjects(self) -> List[str]:
            `GET subjects API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#get--subjects>`_
        """  # noqa: E501

-        return await self._rest_client.get('subjects')
+        query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}


The limit parameter should only be added to the query if it's not negative, as documented in the method's docstring.

Suggested change

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset}

if limit >= 0:

query['limit'] = limit

Copilot · 2025-08-18T19:56:33Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

@@ -970,7 +1174,8 @@ async def get_versions(self, subject_name: str) -> List[int]:
            `GET Subject Versions API Reference <https://docs.confluent.io/current/schema-registry/develop/api.html#post--subjects-(string-%20subject)-versions>`_
        """  # noqa: E501

-        return await self._rest_client.get('subjects/{}/versions'.format(_urlencode(subject_name)))
+        query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}


The limit parameter should only be added to the query if it's not negative, as documented in the method's docstring.

Suggested change

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset, 'limit': limit}

query = {'deleted': deleted, 'deleted_only': deleted_only, 'offset': offset}

if limit >= 0:

query['limit'] = limit

MSeal · 2025-08-18T21:39:04Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+        Raises:
+            SchemaRegistryError: if subjects can't be found
+
+        TODO: add API reference


Leftover TODO

Thanks! Will remove it, and add the API reference link once technical writers added it to https://docs.confluent.io/platform/current/schema-registry/develop/api.html#sr-api-compatibility page (separate task for SR team)

MSeal · 2025-08-18T21:40:50Z

src/confluent_kafka/schema_registry/_sync/schema_registry_client.py

+        Raises:
+            SchemaRegistryError: if subjects can't be found
+
+        TODO: add API reference


Ditto on reference TODO

rayokota

Thanks @fangnx , left some comments

rayokota · 2025-08-18T21:41:07Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

@@ -704,6 +715,35 @@ async def get_schema(

        return registered_schema.schema

+    async def get_schema_string(


Let's remove this method. It's a convenience method mainly for use with curl or PostMan and is otherwise redundant

rayokota · 2025-08-18T21:43:14Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

@@ -953,12 +1096,73 @@ async def get_version(

        return registered_schema

-    async def get_versions(self, subject_name: str) -> List[int]:
+    async def get_version_schema_string(


Let's remove this method as well, as it's another convenience method

rayokota · 2025-08-18T21:48:43Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

-        self, schema_id: int, subject_name: Optional[str] = None, fmt: Optional[str] = None
+        self, schema_id: int, subject_name: Optional[str] = None,
+        fmt: Optional[str] = None, reference_format: Optional[str] = None,
+        find_tags: Optional[List[str]] = None, fetch_max_id: bool = False


Let's remove find_tags and fetch_max_id for now as those are for internal use.

Will remove them. Out of curiosity, looks like those query params (and their usage) are accessible to external customers as well. Will that become an issue at some point?

rayokota · 2025-08-18T21:49:19Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            subject_name (str): Subject name the schema is registered under.
+            fmt (str): Desired output format, dependent on schema type.
+            reference_format (str): Desired output format for references.
+            find_tags (list[str]): Find tagged entities for the given tags or * for all tags.


Remove find_tags and fetch_max_id

rayokota · 2025-08-18T21:49:32Z

src/confluent_kafka/schema_registry/_async/schema_registry_client.py

+            query['format'] = fmt
+        if reference_format is not None:
+            query['reference_format'] = reference_format
+        if find_tags is not None:


sonarqube-confluent · 2025-08-18T22:57:35Z

66.70% Coverage on New Code (is less than 80.00%)

Analysis Details

3 Issues

0 Bugs
0 Vulnerabilities
3 Code Smells

Coverage and Duplications

66.70% Coverage (66.10% Estimated after merge)
No duplication information (5.80% Estimated after merge)

Project ID: confluent-kafka-python

View in SonarQube

updaet

b0410dd

This comment has been minimized.

Sign in to view

fangnx added 3 commits August 11, 2025 17:39

update

3212e55

tests

356f3ca

fix integration tests

884c35e

MSeal approved these changes Aug 12, 2025

View reviewed changes

fangnx added 2 commits August 12, 2025 21:37

regenerate sync code + fix it

a30561b

fix some unit tests broken from unasync

30a97e5

This comment has been minimized.

Sign in to view

missing params and address feedback

f1029a9

This comment has been minimized.

Sign in to view

lint

c465d04

fangnx changed the title ~~Add missing endpoints to SchemaRegistryClient~~ Add missing endpoints (/schemas, /subjects) to SchemaRegistryClient Aug 13, 2025

fangnx marked this pull request as ready for review August 13, 2025 20:04

Copilot AI review requested due to automatic review settings August 13, 2025 20:04

fangnx requested review from a team as code owners August 13, 2025 20:04

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

Merge branch 'master' into sr-endpoints-add-1

363c7af

This comment has been minimized.

Sign in to view

fangnx requested review from MSeal and Copilot August 14, 2025 21:05

Copilot AI reviewed Aug 14, 2025

View reviewed changes

rayokota requested changes Aug 14, 2025

View reviewed changes

revert breaking chamges

3115286

fangnx requested review from Copilot and rayokota August 15, 2025 19:34

update

e812f1c

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

fangnx requested a review from Copilot August 18, 2025 18:18

This comment was marked as outdated.

Sign in to view

fangnx added 2 commits August 18, 2025 14:22

Merge branch 'master' into sr-endpoints-add-1

6f5a209

update

00db3ae

fangnx requested a review from Copilot August 18, 2025 19:54

Copilot AI reviewed Aug 18, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

MSeal approved these changes Aug 18, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

rayokota requested changes Aug 18, 2025

View reviewed changes

address feedback

aeb14fe

-        query = {'offset': offset, 'limit': limit}
+        query = {'offset': offset}
+        if limit >= 0:
+            query['limit'] = limit

		@@ -704,6 +715,35 @@ async def get_schema(

		return registered_schema.schema

		async def get_schema_string(

Add missing endpoints (/schemas, /subjects) to SchemaRegistryClient #2017

Are you sure you want to change the base?

Add missing endpoints (/schemas, /subjects) to SchemaRegistryClient #2017

Conversation

fangnx commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Checklist

References

Test & Review

Open questions / Follow-ups

Uh oh!

confluent-cla-assistant bot commented Aug 8, 2025

Uh oh!

This comment has been minimized.

MSeal left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

Uh oh!

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

rayokota left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

fangnx commented Aug 8, 2025 •

edited

Loading

rayokota left a comment •

edited

Loading