Skip to content

bug(pipes): mutation pipe results are cached and coalesced — the write silently drops on repeat calls #386

Description

@taitelee

Summary

/v1/pipes/{name} runs its SQL through the pipe cache + singleflight with no mutation guard, even though the shared executor deliberately supports mutation-verb SQL. A pipe that writes returns a cached [] on repeat calls and never re-executes.

Detail

  • internal/api/clickhouse_exec.go:26-31 dispatches mutation-verb SQL through conn.Exec, so an INSERT/ALTER pipe does execute the first time.
  • internal/api/pipes.go:176-221 caches the result under queryCacheKey(sql, params) with cache.QueryTimeToTTL (floor 10s, up to 1h) and wraps execution in singleflight — with no isMutation(sql) check on either the Put (store) or execute path.

Scenario: an admin authors INSERT INTO audit_log VALUES ({{msg}}, now()) with allowed_roles: ["writer"]. First call inserts and caches []. Every identical call within the TTL returns 200 + X-Cache: HIT and never reaches ClickHouse; concurrent identical calls are collapsed by singleflight to a single insert.

Impact

Silent data loss on mutation pipes. Docs frame pipes as the cached read path but never forbid mutation SQL, and the exec layer explicitly supports it.

Fix direction

Either reject mutation SQL at pipes.Store.Put, or bypass cache + singleflight when isMutation(sql).

Found in a repo-wide audit; verified by code trace. Distinct from #321/#365.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cacheLocal / shared / tiered cachingarea/pipesNamed query pipesbugSomething isn't working

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions