v1.67.0

Summary

In this release, Application Performance Monitoring (APM) introduces optimized W3C context propagation, reduced lock contention on Git metadata tags and support for span events in OpenTelemetry. It improves multiple contribs, see below, and releases a new one for uptrace/bun. It also introduces multiple fixes, improves support for Orchestrion and deprecates hostname detection, now available behind a compatibility environment variable DD_TRACE_CLIENT_HOSTNAME_COMPAT=v1.66.0.

Contribs updated in this release:

• gocql/gocql: support for MapScanCAS and the Observer API, and bug fixed in cluster and datacenter tag values, old behaviour available behind compatibility environment variable DD_TRACE_GOCQL_COMPAT=v1.65.
• gorm.io/gorm.v1: adds parent span propagation.
• IBM/sarama.v1: support for DSM.
• net/http: support for rwUnwrapper interface.
• uptrace/bun: new contrib available.

Application Security Management (ASM) improves WAF run operations and upgrades go-libddwaf.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: setup GLS-stored context by @eliottness in #2761
• tracer: allow x-datadog headers to pass as header tags by @mtoffl01 in #2744
• internal: cache build info to reduce lock contention in GetGitMetadataTags by @darccio in #2770
• ddtrace/tracer,profiler: share agent default URL resolution logic by @nsrip-dd in #2719
• ddtrace/tracer: put hostname detection behind a compat environment variable by @darccio in #2786
• ddtrace/tracer: optimize textmap.composeTracestate by @darccio in #2794
• ddtrace/tracer: noDebugStack config reports to telemetry under distinct key debug_stack_enabled by @mtoffl01 in #2798
• ddtracer/tracer: fix appending 'tracer_version' to the statsd tags by @nsrip-dd in #2801
• Mtoff/span events by @mtoffl01 in #2780
• ddtrace/mocktracer: fix SetUser & WithUserMetadata by @eliottness in #2787
• ddtrace/tracer: Add test to ensure global config inherits statsd address when configured with tracer.WithDogstatsdAddress by @mtoffl01 in #2683
• ddtrace/tracer: update the tag naming convention for otel env var telemetry metrics by @khanayan123 in #2760
• ddtrace/tracer: update the expected telemetry tags for OTel env-var mapping by @mcculls in #2797
• [contrib/gocql] adding support for MapScanCAS in gocql (cassandra) by @ufoot in #2752
• contrib/uptrace/bun: initial implementation by @darccio in #2771
• contrib/net/http: implement rwUnwrapper by @rarguelloF in #2775
• [contrib/gocql] reporting both cluster and datacenter by @ufoot in #2577
• contrib/IBM/sarama.v1: add DSM support by @rarguelloF in #2788
• contrib/gorm.io/gorm.v1: propagate parent span by @System-Glitch in #2759
• contrib/jackc/pgx.v5: acquire release by @ngavinsir in #2722
• contrib/jackc/pgx.v5: publish pgxpool stats to statsd client by @bengentree in #2692
• contrib/gocql/gocql: implement observer api based tracing (#2805)

Application Security Management (ASM)

• internal/appsec: refactor simple WAF run operations by @eliottness in #2776
• upgrade go-libddwaf & appsec-internal-go and remove occurences of go 1.20 by @eliottness in #2782

General

• build(deps): bump github.com/vektah/gqlparser/v2 from 2.5.8 to 2.5.15 by @dependabot in #2755
• build(deps): bump github.com/gofiber/fiber/v2 from 2.52.1 to 2.52.5 by @dependabot in #2767
• build(deps): bump github.com/hashicorp/go-retryablehttp from 0.7.4 to 0.7.7 by @dependabot in #2757
• internal/exectracetest: fix TestExecutionTraceSpans flakes by @nsrip-dd in #2769
• Add datadog static analyzer by @juli1 in #2741
• ci: upgrade golangci-lint 1.52.2 -> 1.59.1 by @eliottness in #2777
• Add instruction how to update Go version for macrobenchmarks by @ddyurchenko in #2793

New Contributors

• @juli1 made their first contribution in #2741
• @System-Glitch made their first contribution in #2759
• @ngavinsir made their first contribution in #2722
• @bengentree made their first contribution in #2692
• @mcculls made their first contribution in #2797

Full Changelog: v1.66.0...v1.67.0

v1.66.0

Summary

In this release, Application Performance Monitoring (APM) adds support for log/slog instrumentation.
And Application Security Management (ASM) now monitors protects SQL queries performed via database/sql against SQL injections.

This release aligns our Go support policy with the official policy of the Go project, brings several bug fixes to the tracer, contrib packages and appsec.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: add propagation style info to the startup logs by @khanayan123 in #2748
• contrib: add log/slog instrumentation by @rarguelloF in #2765
• ddtrace/opentelemetry: Fix http remapping to account for int values by @zarirhamza in #2737
• ddtrace/tracer: update sample rate configuration parsing to avoid logging a warning when empty by @zacharycmontoya in #2751
• contrib/99designs/gqlgen: additional options to reduce noise by @samsullivan in #2695
• ddtracer/tracer: fix appending 'tracer_version' to the statsd tags by @dianashevchenko in #2784

Application Security Management (ASM)

• appsec: support SQLi Exploit Prevention (RASP) by @Hellzy in #2730

General

• datastreams: Remove one colon check for well-formed edge tags by @juliannzhou in #2729
• internal/telemetry: don't send app-dependencies-loaded in the absence of Go deps by @Julio-Guerra in #2740
• readme: drop go 1.19 from support table by @eliottness in #2758
• all: align Go support with upstream policy by @nsrip-dd in #2705
• internal/datastreams: Always try sending data stream stats by @piochelepiotr in #2756
• .gitlab/macrobenchmarks: Upgrade macrobenchmarks setup, add CPU / RSS utilization overhead measurements by @ddyurchenko in #2763
• .github/workflows: run smoke tests on latest Go 1.23 release candidate by @darccio in #2764

New Contributors

• @zacharycmontoya made their first contribution in #2751
• @khanayan123 made their first contribution in #2748

Full Changelog: v1.65.1...v1.66.0

v1.65.1

Summary

This patch release provide two small fixes:

• Workaround of a bug found in go 1.20 that concerns all users of the appsec/events.IsSecurityError() that made the function always return false. Seeing spans in the DataDog's UI containing errors with the following message can also be a symptom of this issue: request blocked by WAF
• Removal of a unsolicitated warning log in the form of: ignoring DD_TRACE_SAMPLE_RATE, error: strconv.ParseFloat: parsing "": invalid syntax

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: update sample rate configuration parsing to avoid logging a warning when empty in #2751

Application Security Management (ASM)

• appsec: fix IsSecurityError in #2746

Full Changelog: v1.65.0...v1.65.1

v1.65.0

Summary

In this release, Application Performance Monitoring (APM) improved trace and span IDs generation using math/rand/v2 for Go 1.22 and later. APM introduced support for Go 1.22 ServeMux patterns and OTel env vars and better OTel HTTP status code remapping. Also, APM fixed a race condition when using spans concurrently and how dropped traces were reported to the agent.

Application Security Management (ASM) introduces its new "Exploit Prevention" in public beta, a new type of in-app security monitoring that detects and blocks vulnerability exploits. This release includes Exploit Prevention of Server-Side Request Forgeries (SSRF) for HTTP client requests.

Data Streams Monitoring (DSM) updates sketch mapping to improve accuracy and moves to google.golang.org/protobuf.

And last, multiple dependencies have been upgraded to fixed versions for reported vulnerabilities and to reduce our dependency on github.com/golang/protobuf.

Changes

Application Performance Monitoring (APM)

• go.mod: update github.com/DataDog/sketches-go to v1.4.5 by @darccio in #2698
• contrib/database/sql: allow peer.service to be set explicitly and add into propagated context by @tabgok in #2679
• contrib/net/http: Support go1.22 ServeMux patterns by @felixge in #2716
• ddtrace/tracer: use math/rand/v2 for SpanID and TraceID by @knusbaum in #2689
• ddtrace/tracer: fix concurrent map writes when applying trace sampling rules and setting tags concurrently by @darccio in #2727
• w3c: ensure _dd.parent_id is set using x-datadog-parent-id by @mabdinur in #2696
• ddtrace/tracer: increment droppedP0Traces iff it's not going to be sent by @darccio in #2713
• ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec by @darccio in #2623
• ddtrace/tracer: only finish execution trace task, restore pprof labels once by @nsrip-dd in #2708
• ddtrace/tracer: add support for OTEL env vars by @mtoffl01 in #2715
• ddtrace/tracer: report trace and span sampling rules separately by @darccio in #2718
• ddtrace/opentelemetry: adds http status code remapping for otel trace metrics by @zarirhamza in #2728

Application Security Management (ASM)

• appsec: support for SSRF Exploit Prevention by @Hellzy in #2707
• appsec/grpc: fix rpc message blocking by @Julio-Guerra in #2723

Data Streams Monitoring

• Move to google.golang.org/protobuf by @piochelepiotr in #2694
• datastreams: Update mapping by @piochelepiotr in #2703
• [DSMS-19] Fix SNS topic format issue by @juliannzhou in #2725

General

• internal: stacktrace: skip internal packages by @eliottness in #2697
• go.mod: upgrade github.com/mattn/go-sqlite3 up to v1.14.18 (#2693) by @darccio in #2714
• go.mod: upgrade github.com/go-jose/go-jose/v3 to v3.0.3 by @darccio in #2700
• go.mod: upgrade google.golang.org/protobuf to v1.33.0 by @darccio in #2699
• go.mod: upgrade richardartoul/molecule by @darccio in #2724

New Contributors

• @juliannzhou made their first contribution in #2725

Full Changelog: v1.64.0...v1.65.0

v1.64.1

This patch release brings a fix for a data race in ddtrace/tracer when applying trace sampling rules and setting tags concurrently.

Full Changelog: v1.64.0...v1.64.1

v1.64.0

Summary

In this release, the tracer adds support for tag-based sampling in sampling rules set via remote configuration. See https://docs.datadoghq.com/tracing/trace_collection/runtime_config/ for more on remote configuration.
Additionally, the tracer now supports matching numeric values via span sampling rules, in addition to string values.
For example, [{"error_code": "4??", "sampling_rate":"0.01"}] will now match both the integer 403 and
the string `"403". String matches are now case-insensitive.

This release brings several bug fixes to the tracer, contrib packages, and remote configuration.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: added support for metrics tag sampling by @dianashevchenko in #2662
• ddtrace/tracer: adding support of tags in remote sampling rules. by @yuanyuanzhao3 in #2676
• ddtrace/tracer: added a WithAgentTimeout option to reduce the test suite duration with no agent by @dianashevchenko in #2661

Fixes

Application Performance Monitoring (APM)

• docs: fix dbStats example in contrib/database/sql by @mtoffl01 in #2669
• contrib/dimfeld/httptreemux.v5: fix route and name for 30X redirects by @darccio in #2685
• contrib/database/sql: Disable DBStats if statsd client initialization fails by @mtoffl01 in #2682
• fix: Don't call c.Error(...) inside echotrace middlewares by @nakkamarra in #2609
• {ddtrace/tracer,datastreams}: set default agent TO to 10s by @ahmed-mez in #2655
• internal/remoteconfig: fixes data race when accessing capabilities by @SvenGasterstaedt in #2652
• ddtrace/tracer: fix calculation of tracestate length by @bouwkast in #2585
• ddtracer/trace: if tracing disabled don't call agent to get features by @bstncartwright in #2482
• ddtrace/tracer: fix sampling rule override for manual keep by @dianashevchenko in #2666
• internal: Fixing _dd.p.dm decision maker collision on number 10. by @yuanyuanzhao3 in #2672
• ddtrace/tracer: remove agent_psr when rule_psr is set by @dianashevchenko in #2668

New Contributors

• @nakkamarra made their first contribution in #2609
• @SvenGasterstaedt made their first contribution in #2652
• @bouwkast made their first contribution in #2585
• @bstncartwright made their first contribution in #2482

Full Changelog: v1.63.1...v1.64.0

v1.63.1

Summary

Fixed _dd.p.dm decision maker collision on number 10.

Changes

Fixes

• [v1.63.1] internal: Fixing _dd.p.dm decision maker collision on number 10 by @darccio in #2673

Full Changelog: v1.63.0...v1.63.1

v1.63.0

What's Changed

Application Performance Monitoring (APM)

Summary: database/sql integration allows to send DB stats metrics with WithDBStats, new options added to 99designs/gqlgen and segmentio/kafka.go, and google.golang.org/grpc.v12 compilation errors fixed. Also introduces support for remote sampling rules consistent with the current implementation, except sampling by tags.

• ddtrace/tracer: populate and propagate remote rules decision maker by @yuanyuanzhao3 in #2643
• ddtrace/tracer: support for remote sampling rules by @yuanyuanzhao3 in #2596
• contrib/99designs/gqlgen: add WithCustomTag option by @samsullivan in #2598
• contrib/database/sql: implementation of DB Stats feature with unique statsd client by @mtoffl01 in #2629
• contrib/database/sql: use tracer-level tags on contrib's statsd.Client by @mtoffl01 in #2635
• contrib/google.golang.org/grpc.v12: replace missing {From,New}Context with equivalent functions by @darccio in #2624
• contrib/segmentio/kafka.go.v0: add DSM support by @adrien-f in #2625

Application Security Management (ASM)

• Stricter time budget per request by @eliottness in #2613
• Collect edge waf headers by @Julio-Guerra in #2637
• Allow enabling SCA with DD_APPSEC_SCA_ENABLED by @Julio-Guerra in #2634
• contrib/chi.v5: add a response header reader option by @turettn in #2640

Profiling

• profiler/internal/fastdelta: remove Go 1.18 + windows fuzz test workaround by @nsrip-dd in #2639

General

• go.mod: rotate go 1.19 from supported versions by @eliottness in #2617
• internal: remove pre-Go 1.18 workaround for reading build info by @nsrip-dd in #2638

Fixes

Application Performance Monitoring (APM)

• internal/remoteconfig/: Fixes capability APM_TRACING_SAMPLING_RULES to use the correct reserved number 29. by @yuanyuanzhao3 in #2620
• internal/samplernames: fixes a colliding sampler name by @yuanyuanzhao3 in #2627

Application Security Management (ASM)

• Handle bad redirect action parameters by @Hellzy in #2604
• Make redirect actions blocking by @Hellzy in #2628
• Don't enable blocking-related RC features when using local security rules by @Hellzy in #2626

New Contributors

• @samsullivan made their first contribution in #2598
• @adrien-f made their first contribution in #2625
• @dmellonielet made their first contribution in #2644
• @turettn made their first contribution in #2640

Full Changelog: v1.62.0...v1.63.0

v1.62.0

Summary

In this release, Application Performance Monitoring (APM) improves header tag normalization, expands support for dynamic instrumentation

Application Security Management (ASM) adds support for passlist security events on gRPC.

What's Changed

Application Performance Monitoring (APM)

• [DSM] Close span on produce error in ckgo by @mborst in #2558
• contrib/database/sql: add in ddh, dddb propagation by @tabgok in #2550
• contrib/google.golang.org/grpc: fix flaky tests by removing arbitrary sleeps by @Julio-Guerra in #2584
• contrib/gofiber/fiber.v2: add possibility to exclude spans generation for specific requests by @nsakharenko in #2583
• ddtrace/tracer: added UnmarshalJSON method to sampling rules by @dianashevchenko in #2563
• tracer/remote-config: Subscribe to dynamic instrumentation configs via remote config by @grantseltzer in #2510
• ddtrace/tracer: go tracer tests lint. by @yuanyuanzhao3 in #2587
• statsdtest: Move mock statsd client for testing into its own package by @mtoffl01 in #2564
• normalizer: expand "header tag" normalization by @mtoffl01 in #2549

Application Security Management (ASM)

• appsec: fix remote security rules update by @Hellzy in #2568
• appsec: remove byte slices from WAF input by @eliottness in #2591
• appsec: DD_APPSEC_WAF_TIMEOUT default value: 4ms -> 1ms @eliottness in #2591
• contrib/google.golang.org/grpc: security rule passlist support by @Julio-Guerra in #2589

General

• Replace Go version 1.18+1.19 with 1.19+1.22.0 by @ddyurchenko in #2572

Fixes

• contrib/gorm.io/gorm.v1: do not panic on open by @bendiknesbo in #2560

New Contributors

• @mborst made their first contribution in #2558
• @bendiknesbo made their first contribution in #2560
• @tabgok made their first contribution in #2550
• @grantseltzer made their first contribution in #2510
• @yuanyuanzhao3 made their first contribution in #2587
• @nsakharenko made their first contribution in #2583

Full Changelog: v1.61.0...v1.62.0

v1.61.0

Summary

In this release, Application Performance Monitoring (APM) adds support for Span Links (a highly requested feature!). This feature is currently supported within Datadog's OpenTelemetry API implementation.

The default trace context propagation order, which is used for traces in distributed workflows, will become datadog,tracecontext (previously it was tracecontext,datadog). This is not a breaking change, and customers should not experience any negative changes in behavior. If you experience any issues, please reach out to Datadog support.

Other APM features include out-of-the-box library integration support for github.com/jackc/pgx/v5 and the ability to ignore specific error types in the github.com/labstack/echo/v4 integration.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: Switch default context propagation order by @JianyiGao in #2368
• tracing: Adds support for Span Links by @mabdinur in #2502
• ddtrace/tracer: added tracing_enabled option to remote config by @dianashevchenko in #2513
• contrib/labstack/echo.v4: add option to ignore errors by @mrkagelui in #1567
• contrib/jackc/pgx.v5: add pgx support by @renanferr in #2410
• dyngo: dynamically register listeners only if they're needed by @RomainMuller in #2394

Application Security Management (ASM)

ASM Customers upgrading to Go 1.22 should upgrade dd-trace-go or at least upgrade github.com/DataDog/go-libddwaf to version v2.3.1. Otherwise ASM Threats won't start with the following error:

appsec: threats detection cannot be enabled for the following reasons: 1 error occurred:
	* unsupported Go version: go1.22.0

• appsec: support for go 1.22 by @Julio-Guerra in go-libddwaf#64
• appsec: DataDog's WAF ignore field with ddwaf:"ignore" by @eliottness in go-libddwaf#68
• chore: go-libddwaf v2.3.1 by @eliottness (Release Notes)

Profiling

• profiler: add pgo tag by @felixge in #2556

Data Streams Monitoring (DSM)

• [data streams] Track high watermark offsets by @piochelepiotr in #2511

General

• profiler: skip flaky TestExecutionTraceRandom by @nsrip-dd in #2531
• ci/appsec: refresher by @Julio-Guerra in #2537
• internal/namingschema: simplify the namingschema by @knusbaum in #2129
• interna/version: bump version.go to v1.61.0-dev by @katiehockman in #2501
• .github/workflows: fixes apm:ecosystem label for issues and PRs by @katiehockman in #2525
• opentelemetry: refactor span links code for clarity by @katiehockman in #2538
• chore: make the smoke-test workflow usable as go-libddwaf integ test by @RomainMuller in #2546
• chore: fix smoke test workflow_call situation by @RomainMuller in #2547
• chore: make git available in smoke-test docker image by @RomainMuller in #2548
• ci/system-tests: add graphql system tests by @Julio-Guerra in #2554

New Contributors

• @Juneezee made their first contribution in #2437
• @mabdinur made their first contribution in #2502
• @renanferr made their first contribution in #2410

Full Changelog: v1.60.3...v1.61.0