[BUG]: Telemetry dependency collection massively consumes CPU at 1 minute mark impacting our production services

[kravnaut]

Tracer Version(s)

2.19.3

Python Version(s)

3.12.8

Pip Version(s)

24.3.1

Bug Report

Recently (or at least in this version we just upgraded to) added feature gated by DD_TELEMETRY_DEPENDENCY_COLLECTION_ENABLED seems to be

We have profiled the application and upon learning it was this feature we saw in your code it was gated using env var DD_TELEMETRY_DEPENDENCY_COLLECTION_ENABLED which we have since engaged to disable it. This resolved the problem for us.

Reproduction Code

CPU spikes to 100% and stays there for 5+ seconds impacting ongoing request handling in the container.

Error Logs

No response

Libraries in Use

No response

Operating System

No response

[kravnaut]

Here is austin profiler snapshot of the process running this code in ddtrace's thread with -s setting on to only measure non-idle frames

The heaviest path:

get_module_distribution_versions (/usr/local/lib/python3.12/site-packages/ddtrace/internal/packages.py:77)
> update_imported_dependencies (/usr/local/lib/python3.12/site-packages/ddtrace/internal/telemetry/data.py:76)
> TelemetryWriter._app_dependencies_loaded_event (/usr/local/lib/python3.12/site-packages/ddtrace/internal/telemetry/writer.py:419)
> TelemetryWriter.periodic (/usr/local/lib/python3.12/site-packages/ddtrace/internal/telemetry/writer.py:627)

[P403n1x87]

@kravnaut thanks for reporting this. We have merged a fix for this recently (#12327) and it should become available with the next patch releases.

[joshfermin]

@P403n1x87 was this part of the 2.21.2 release? we are still seeing significant CPU performance issues as well on that version.