pkg: add reusable opencost + prom modules for FinOps v2#492
Draft
nadaverell wants to merge 2 commits into
Draft
Conversation
nadaverell
force-pushed
the
feature/finops-v2
branch
from
e43924c to
4ded49b
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Moves the Prometheus client + cost math out of internal/ and into pkg/ so koala-backend and skyhook-connector can share the code for the FinOps v2 Costs Explorer. pkg/prom — extracted from internal/prometheus: - Client, HTTPTransport, Discover(), and the metric-query builders. - Optional Headers map on HTTPTransport for auth-protected backends (Authorization, X-Scope-OrgID, ...) — applied after Accept. - Memory-scrape dedupe (max-by container inside the outer sum) folded into BuildQuery / BuildNamespaceQuery / BuildClusterQuery so callers do not double-count series from multiple scrape jobs. - internal/prometheus is now a thin facade re-exporting via aliases.go; client.go and discovery.go delegate, queries.go moved under pkg/. pkg/opencost has two paths into the same data: - REST (ComputeCostSummary, ComputeCostTrend) hits OpenCost's /allocation API. windowHours() normalizes /allocation.totalCost (summed over the window) to dollars/hr so callers project monthly via x 730 regardless of window. Coarse default steps (24h->6h, 7d->1d, 30d->2d) fit CAC's 30s proxy budget. Includes pod/controller aggregation, client-side namespace post-filter for OpenCost versions that silently ignore the REST filter param, removal of synthetic __unallocated__ drilldown rows, and per-row NetworkCost surfacing. - PromQL (ComputeCostSummaryFromProm, ComputeCostTrendFromProm, ComputeWorkloadsFromProm, ComputeNodeCosts) extracted from internal/opencost/handlers.go. The workloads path takes a PodOwnerLookup callback so pkg/opencost stays free of k8s.io/client-go; radar supplies it from its informer cache, koala-backend / connector can supply it from CAC-served pod metadata. internal/opencost/handlers.go shrinks 688 -> 147 lines; the four endpoints (/summary, /workloads, /trend, /nodes) keep their JSON shape unchanged.
nadaverell
force-pushed
the
feature/finops-v2
branch
from
4ded49b to
226ff2b
Compare
The endpoint accepted ?url=<any HTTP(S) URL> and called SetURL() to redirect all subsequent Prometheus queries to that host. The scheme was validated but the host was not, and radar binds to 0.0.0.0 by default — so anyone reachable on the listen port could exfiltrate the configured --prometheus-header credentials to an arbitrary host or port-scan internal services via Prometheus query latency. The override was undocumented and unused by the radar UI (it only POSTs /prometheus/connect with no body or params). Operators set the Prometheus URL via --prometheus-url at startup; there's no UX cost to removing the per-request override. Also removes the now-orphaned (*Client).SetURL method. Flagged by CodeQL go/request-forgery (alert #330).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves the Prometheus client + cost math out of
internal/and intopkg/so koala-backend and skyhook-connector can share the code for the FinOps v2 Costs Explorer. Net result:internal/prometheus/{client,discovery,queries}.goshrinks substantially (queries.go moved underpkg/prom/).internal/opencost/handlers.goshrinks 688 → 147 lines — handlers are now thin HTTP wrappers aroundpkg/opencost./summary,/workloads,/trend,/nodes) keep their JSON shape unchanged.pkg/prom— extracted frominternal/prometheusClient,HTTPTransport,Discover(), and all metric-query builders.Headers map[string]stringonHTTPTransportfor auth-protected backends (Authorization,X-Scope-OrgID, …) — applied afterAccept.BuildQuery/BuildNamespaceQuery/BuildClusterQueryso callers don't double-count series from multiple scrape jobs.internal/prometheusis now a thin facade re-exporting viaaliases.go;client.goanddiscovery.godelegate,queries.gomoved underpkg/.pkg/opencost— two paths into the same dataREST (
ComputeCostSummary,ComputeCostTrend) — new code path, hits OpenCost's/allocationAPI:windowHours()normalizes/allocation.totalCost(summed over the window) to $/hr so callers project monthly via × 730 regardless of window. Regression test pins this.SummaryOptions.Aggregate.filterparam.__unallocated__controller/pod drilldown rows so child totals don't exceed the parent namespace.NetworkCostper row +TotalNetworkCostin the summary.PromQL (
ComputeCostSummaryFromProm,ComputeCostTrendFromProm,ComputeWorkloadsFromProm,ComputeNodeCosts) — extracted frominternal/opencost/handlers.go:PodOwnerLookupcallback sopkg/opencoststays free ofk8s.io/client-go. radar supplies it from its informer cache; koala-backend / connector can supply it from CAC-served pod metadata.Tag published:
pkg/v1.4.4. Consumed by:Test plan
go test ./...in both root andpkg/submodules — all green (includes window-normalization regression, header pass-through, memory-scrape dedupe)./api/opencost/{summary,workloads,trend,nodes}endpoints continue to return the same JSON shape after extraction.