schema registry protobuf serialization exception after upgrading to 2.7.0

[jbouricius]

Running version 2.6.1 of the library, this is approximately my code:

class FooSerializer:

    def __init__(self, topic: str, schema_registry: ConfluentSchemaRegistryClient):
        self._topic = topic
        self._protobuf_serializer = ProtobufSerializer(
            Foo,
            schema_registry,
            conf={
                "auto.register.schemas": False,
                "use.deprecated.format": False,
                "use.latest.version": True,
            },
        )

    def serialize(self, foo: Foo) -> bytearray:
        return self._protobuf_serializer(foo, SerializationContext(self._topic, MessageField.VALUE))

where Foo is some proto class already registered as the schema for the topic. This works exactly as I expect.

After upgrading to 2.7.x or 2.8.0, the same exact code hits an exception ending in this traceback:

    return self._protobuf_serializer(foo, SerializationContext(self._topic, MessageField.VALUE))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/testrunner/.local/lib/python3.11/site-packages/confluent_kafka/schema_registry/protobuf.py", line 582, in __call__
    fd_proto, pool = self._get_parsed_schema(latest_schema.schema)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/testrunner/.local/lib/python3.11/site-packages/confluent_kafka/schema_registry/protobuf.py", line 610, in _get_parsed_schema
    _resolve_named_schema(schema, self._registry, pool)
  File "/home/testrunner/.local/lib/python3.11/site-packages/confluent_kafka/schema_registry/protobuf.py", line 207, in _resolve_named_schema
    _resolve_named_schema(referenced_schema.schema, schema_registry_client, pool, visited)
  File "/home/testrunner/.local/lib/python3.11/site-packages/confluent_kafka/schema_registry/protobuf.py", line 208, in _resolve_named_schema
    file_descriptor_proto = _str_to_proto(ref.name, referenced_schema.schema.schema_str)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/testrunner/.local/lib/python3.11/site-packages/confluent_kafka/schema_registry/protobuf.py", line 182, in _str_to_proto
    raise SerializationError(str(e))
confluent_kafka.serialization.SerializationError: Error parsing message

I didn't see anything in the changelogs that indicated there was a breaking change here. Am I missing a change or is there some other problem? Any other information I can provide?

[franciellyferreira]

Hi, I have the same problem, to temporary solve it I fixed the version 2.6.1 in the project.

[rayokota]

For Protobuf, this library needs changes from confluentinc/schema-registry#3276, which will be in the next CP patch releases.