SystemError when a callback raises

[spenczar]

Description

I'd like to raise an exception when the error_cb callback is triggered. If I do so, two exceptions are raised. First the one I asked for, and second is a SystemError.

I'm working on a command-line tool. I really want to crash in case of an error.

The SystemError is making it difficult for me to write tests around my error reporting. It also makes the error messages much harder to interpret for users of my command-line tool.

How to reproduce

I think this is a minimal reproducer - it will try to connect to a broker that doesn't exist so it will trigger the callback:

#!/usr/bin/env python
import confluent_kafka


def demo():
    consumer = confluent_kafka.Consumer({
        "bootstrap.servers": "nonexistent",
        "group.id": "demo",
        "error_cb": error_cb,
    })
    consumer.subscribe(["test"])
    return consumer.consume(1, timeout=1)


def error_cb(error):
    raise(confluent_kafka.KafkaException(error))


if __name__ == "__main__":
    demo()

If I run this, this is what I get:

Traceback (most recent call last):
  File "minimal_reproducer.py", line 17, in error_cb
    raise(confluent_kafka.KafkaException(error))
cimpl.KafkaException: KafkaError{code=_RESOLVE,val=-193,str="nonexistent:9092/bootstrap: Failed to resolve 'nonexistent:9092': Temporary failure in name resolution (after 9ms in state CONNECT)"}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "minimal_reproducer.py", line 21, in <module>
    demo()
  File "minimal_reproducer.py", line 13, in demo
    return consumer.consume(1, timeout=1)
  File "minimal_reproducer.py", line 17, in error_cb
    raise(confluent_kafka.KafkaException(error))
SystemError: <class 'cimpl.KafkaException'> returned a result with an error set

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):

>>> import confluent_kafka
>>> confluent_kafka.version()
('1.4.1', 17039616)
>>> confluent_kafka.libversion()
('1.4.0', 17039615)

• Apache Kafka broker version:
  n/a
• Client configuration: {...}
  in reproducer above
• Operating system:
  Ubuntu 20.04, Linux 5.4.0
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue

[spenczar]

Possibly related: #729