fix: treat prometheus counters as rates in autoscaling signals by nXtCyberNet · Pull Request #1042 · volcano-sh/kthena

nXtCyberNet · 2026-05-13T16:34:19Z

What type of PR is this?
/kind bug
/kind enhancement
What this PR does / why we need it:

Prometheus counter metrics are monotonically increasing cumulative values.
The autoscaler was treating these raw cumulative values as instantaneous load
signals, causing two critical failures:

Scale-up runaway: A pod that handled 50 total requests since startup would
trigger scaling to 5 replicas (ceil(50 / target)), regardless of current traffic.
No scale-down: Even after traffic stops, the counter value persists at 50,
preventing the autoscaler from ever scaling back down.

The fix: Track per-pod counter snapshots across scrape cycles and compute the
rate of change (delta/elapsed_seconds) instead of the raw cumulative value.
This correctly reflects instantaneous demand and enables both scale-up and scale-down.

Implementation Details:

Added CounterMap and ScrapeTimestamp fields to HistogramInfo to maintain
per-pod/metric baseline state across scrape cycles.
New rate calculation in metric collection:

rate = (current_value - previous_value) / elapsed_seconds

Counter resets (current < previous) detected and handled by clamping rate to 0.
First scrape returns rate=0 until baseline is established.
Added GetLastUnfreshSnapshotWithTimestamp() to SnapshotSlidingWindow
to expose precise per-pod scrape timestamps (more accurate than window-level timestamps).
Backward compatibility: Nil CounterMap guards protect in-memory snapshots
created before this change.

Which issue(s) this PR fixes:
Fixes #1037

Special notes for your reviewer:

Why per-pod timestamps? The sliding-window-level timestamp is too coarse;
per-pod scrape times give us the actual elapsed duration for each counter,
improving rate precision when scrape intervals vary.
Counter reset handling: If a pod restarts, its counter resets to 0.
Detecting current < previous avoids reporting a massive negative rate;
returning 0 is safe and gives the pod a grace period to re-accumulate load signals.
Backward compatibility: Any in-memory snapshots from before this change
will have CounterMap == nil. These are safely handled with a nil-check guard.
Testing recommendation: Add bench tests for rate calculation under
varying scrape intervals and counter reset scenarios.

Does this PR introduce a user-facing change?:
Yes. Autoscaling behavior for counter-based metrics will change—scaling will now
respond to instantaneous rate of change rather than cumulative totals, enabling
proper scale-down behavior.

Fix autoscaler counter metric handling: Prometheus counter metrics are now 
correctly treated as rates (delta/elapsed_seconds) instead of raw cumulative 
values. This fixes runaway scale-up and enables proper scale-down when traffic stops.

volcano-sh-bot · 2026-05-13T16:34:27Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign git-malu for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

pkg/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

gemini-code-assist

Code Review

This pull request updates the metric collector to calculate rates for Prometheus counters by comparing current values with previous snapshots, introducing a new GetLastUnfreshSnapshotWithTimestamp method in the sliding window structure to track scrape intervals. The reviewer suggested improving the precision of rate calculations by using a single consistent timestamp per pod scrape rather than calling the timestamp function multiple times, and recommended refactoring GetLastUnfreshSnapshot to utilize the new method to reduce code duplication.

gemini-code-assist · 2026-05-13T16:37:46Z

+			collector.processPrometheusString(result, pastHistogramMap, pastCounterMap, currentHistogramMap, currentCounterMap, pastScrapeTimestamp, instanceInfo.MetricsMap)
 			(*currentHistograms)[pod.Name] = HistogramInfo{
-				PodStartTime: pod.Status.StartTime,
-				HistogramMap: currentHistogramMap,
+				PodStartTime:    pod.Status.StartTime,
+				HistogramMap:    currentHistogramMap,
+				CounterMap:      currentCounterMap,
+				ScrapeTimestamp: util.GetCurrentTimestamp(),
 			}


For improved precision in rate calculation, it's better to determine the current scrape timestamp once per pod and use it consistently. Currently, util.GetCurrentTimestamp() is called inside processPrometheusString for each metric, and again when creating HistogramInfo. This can introduce minor inaccuracies because the timestamp used for the rate calculation (now) will be slightly different from the timestamp stored for the next cycle (ScrapeTimestamp).

To improve this, you can get the timestamp once before processing the metrics and use it in both places. This ensures the elapsed_seconds for the rate calculation is based on the exact interval between the stored scrape timestamps.

Example of the proposed change:

// In fetchMetricsFromPods: ... result := string(bodyStr) now := util.GetCurrentTimestamp() collector.processPrometheusString(result, pastHistogramMap, pastCounterMap, currentHistogramMap, currentCounterMap, pastScrapeTimestamp, now, instanceInfo.MetricsMap) (*currentHistograms)[pod.Name] = HistogramInfo{ PodStartTime: pod.Status.StartTime, HistogramMap: currentHistogramMap, CounterMap: currentCounterMap, ScrapeTimestamp: now, } ... // And update processPrometheusString signature and body: func (c *MetricCollector) processPrometheusString(..., pastScrapeTimestamp int64, now int64, instanceMetricMap algorithm.Metrics) { // ... // inside case ..._COUNTER: // remove: now := util.GetCurrentTimestamp() // ... }

gemini-code-assist · 2026-05-13T16:37:46Z

+func (window *SnapshotSlidingWindow[T]) GetLastUnfreshSnapshotWithTimestamp() (value T, timestamp int64, ok bool) {
+	if window.freshMilliseconds == 0 {
+		return value, 0, false
+	}
+	currentTimestamp := window.getCurrentTimestamp()
+	window.expire(currentTimestamp)
+	if window.pool.Len() == 0 {
+		return value, 0, false
+	}
+	front := window.pool.Front()
+	if isFresh(window.freshMilliseconds, currentTimestamp, front.timestamp) {
+		return value, 0, false
+	}
+	return front.value, front.timestamp, true
+}


The logic inside this new function is nearly identical to GetLastUnfreshSnapshot. To reduce code duplication and improve maintainability, GetLastUnfreshSnapshot could be refactored to call this new function and discard the returned timestamp. This would make the code more aligned with the DRY (Don't Repeat Yourself) principle.

hzxuzhonghu · 2026-05-18T02:20:39Z

Thanks for the analysis, can you add some test coverage

hzxuzhonghu · 2026-05-18T02:21:38Z

From another view, it reflects the current api does not fit all.

// AutoscalingPolicyMetric defines a metric and its target value for scaling decisions.
type AutoscalingPolicyMetric struct {
	// MetricName defines the name of the metric to monitor for scaling decisions.
	MetricName string `json:"metricName"`
	// TargetValue defines the target value for the metric that triggers scaling operations.
	TargetValue resource.Quantity `json:"targetValue"`
}

nXtCyberNet · 2026-05-18T04:24:37Z

Hi @hzxuzhonghu, thanks for the review.
I apologize—I didn't fully consider the design implications. Before I proceed, I want to understand the roadmap better. Do you think adding a MetricType field to the API would be a good long-term solution? If you believe it's worth doing, I'm happy to include it in this PR.
However, if adding MetricType would conflict with the existing design (where histogram is the default), I think the best approach is to close this PR for now. Adding counter support without explicit API-level type declaration could introduce ambiguity and break existing behavior.
What are your thoughts on the right path forward?

hzxuzhonghu · 2026-05-18T07:13:33Z

@nXtCyberNet I donot have a good suggestion now, but will deep dive into other scalers first

nXtCyberNet · 2026-05-18T07:23:28Z

Okay, I'll wait for your response. In the meantime, I'll add the test coverage you requested. Thanks!

nXtCyberNet · 2026-06-15T04:54:26Z

@hzxuzhonghu any updates ?

nXtCyberNet · 2026-06-23T13:08:56Z

@kube-gopher hi , just want to know your thoughts on this , since you already the Observability part of the project , Thanks!

kube-gopher · 2026-06-23T13:52:24Z

@nXtCyberNet It looks feasible for now, but I haven't looked into it deeply yet. I'm planning to add a feature to the scaler later—I'll open an issue when the time comes, and we can discuss it together then!

hzxuzhonghu · 2026-06-24T08:59:21Z

can we finish it and make it release in v1.0 by end of this month

kube-gopher · 2026-06-24T14:45:09Z

@hzxuzhonghu That should work. I'll run a test tomorrow and see how it goes

nXtCyberNet · 2026-06-24T14:50:32Z

@hzxuzhonghu @kube-gopher , Yup okay I will complete it asap, Thanks for the confirmation.

nXtCyberNet · 2026-06-25T06:35:18Z

Hi @hzxuzhonghu, thanks for the review. I apologize, I didn't fully consider the design implications. Before I proceed, I want to understand the roadmap better. Do you think adding a MetricType field to the API would be a good long-term solution? If you believe it's worth doing, I'm happy to include it in this PR. However, if adding MetricType would conflict with the existing design (where histogram is the default), I think the best approach is to close this PR for now. Adding counter support without an explicit API-level type declaration could introduce ambiguity and break existing behavior. What are your thoughts on the right path forward?

I think this issue may already be solved. Looking at https://github.com/volcano-sh/kthena/blob/main/docs/kthena/docs/reference/crd/workload.serving.volcano.sh.md#metricsource, it seems we already have a way to specify the required behavior.

nXtCyberNet · 2026-06-25T07:32:07Z

@hzxuzhonghu could you please take a look. Thanks!

kube-gopher · 2026-06-25T13:58:50Z

-	currentHistogramMap := currentHistograms[pod.Name].HistogramMap
-	if currentHistogramMap == nil {
-		currentHistogramMap = make(map[string]*histogram.Snapshot)
-	}


From a visual standpoint, the current design isn't as good as before

yup i agree , however it didnot create very big change just reuse the already created map , i.e its just a memory optimization. But i will do this , Thanks for pointing out ,

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

kube-gopher · 2026-07-07T23:34:32Z

 	}
 	currentHistograms := make(map[string]HistogramInfo)

+	pastCounters, ok := collector.PastCounters.GetLastUnfreshSnapshot()


Have you considered this issue: when counting in a window-based manner, within the first minute after the collector starts or the Pod restarts, even though the counter itself is increasing, all auto-scaling metrics based on that counter will be reported as 0, potentially causing scaling-out to be skipped?

My suggestion is to have the counter immediately obtain the baseline from the last scrape

@nXtCyberNet I suggest that you ultimately submit a clean commit history

yup i agree i willl rebase it later in single commit

but i want when the full system starts then the baseline is also 0 so why did it a problem , but if you thinking if the router is restarted then also since this metrics is gone so how can it get it and if the vllm pod crashed then also all the metrics get flushed out so , isnt it a current tradeoff , and also we are finding average rate over a minute not the instantaneus rate . so could you explain a bit more about this . and also didnot it gonna break the autoscaler algo as the rate we get is a noise
cc @hzxuzhonghu

@nXtCyberNet I don't quite understand your language. Please express your thoughts in an affirmative tone or a declarative mood

i will use ai to finetune this ,

Thanks for the suggestion, but I have a few concerns:

On the "baseline = 0" issue: When the whole system starts fresh, the baseline is also 0 — so why is that actually a problem in that case?
On restarts/crashes: If the router restarts, its metrics are gone anyway. Same if the vLLM pod crashes — all its metrics get flushed. Isn't this just an inherent tradeoff we already have, regardless of the baseline approach? Could you explain in more detail what specific case you're worried about?
On time drift: If there's any time drift, the counter resets back to 0 as well — this happens in both of the above cases too.
On averaging: We're computing the average rate over a full minute, not the instantaneous rate. Won't that introduce noise into what the autoscaler sees, and potentially break its scaling logic?

cc @hzxuzhonghu

hzxuzhonghu · 2026-07-08T08:59:44Z

+				currentCounterMap[policyKey] = v
+				rate := 0.0
+				now := util.GetCurrentTimestamp()
+				if pastCounterMap != nil && pastScrapeTimestamp > 0 {


With this logic, the first scrape after the collector starts or a pod restarts always reports 0 for counter metrics because there is no previous baseline yet. If traffic is already increasing during that first window, scale-out can be skipped until the next scrape. Please seed the baseline from the first scrape while still allowing the next scrape to produce a rate, or document why this delay is acceptable for autoscaling.

@nXtCyberNet This is the best repair solution

@nXtCyberNet Based on your previous response, I think you should be able to understand the maintainer's suggestion

@kube-gopher i think i got it now ,

I'll state my view directly: seed on first scrape, compute from the next scrape, and keep the 60s
GetLastUnfreshSnapshot() logic only for histogram/SLO metrics

okay i have done this even with 15 sec scrap time the rate became noisey but that i think is not a big issue . now you can take a look . Thanks for help

@nXtCyberNet The repair is incomplete

kube-gopher · 2026-07-08T12:17:55Z

@nXtCyberNet Here's a question for you to think about: Prometheus has a default scrape interval of 15 seconds, and the counter snapshot staleness window only reverts to a "stale" baseline after 60 seconds

What rate does the autoscaler report at t=15s, t=30s, t=45s, and t=60s while the counter is increasing?
What rate does it report on the first reconcile after the baseline finally becomes eligible, for example at t=75s?
During that initial 60s period, is the autoscaler observing real load or treating the metric as zero?

You may use AI to assist you, but for the final answer, I want you to phrase it in your own words for me

nXtCyberNet · 2026-07-08T13:29:50Z

Hi, so for your answer, considering the case where the router pod restarts, this is how I understand the timeline:

t=0    collector starts
t=15   first snapshot stored (no baseline)
t=30   no baseline
t=45   no baseline
t=60   no baseline (the t=15 sample is only 45s old)
t=75   the t=15 sample becomes 60s old, so this is the first valid baseline

rate = (counter75 - counter15) / 60

So you're right that during this initial ~60s period the autoscaler will see the request rate as 0.

My question is whether this is actually a problem or simply the expected trade-off of the current design.

On a router restart, the collector loses its in-memory sliding window, so there is no previous sample available to compute a rate from. Similarly, if the vLLM pod restarts, the Prometheus counters themselves reset. In both cases, there isn't enough historical information to compute a meaningful rate immediately after restart.

Also, since this metric is intentionally defined as a rolling one-minute average rather than an instantaneous 15s rate, waiting until a full one-minute baseline exists seems consistent with what the metric is trying to represent.

So is the concern specifically that the autoscaler observes 0 instead of "unknown" during that warm-up period? Or is there another issue I'm missing?

kube-gopher · 2026-07-08T13:57:42Z

Hi, so for your answer, considering the case where the router pod restarts, this is how I understand the timeline:
t=0    collector starts
t=15   first snapshot stored (no baseline)
t=30   no baseline
t=45   no baseline
t=60   no baseline (the t=15 sample is only 45s old)
t=75   the t=15 sample becomes 60s old, so this is the first valid baseline

rate = (counter75 - counter15) / 60
So you're right that during this initial ~60s period the autoscaler will see the request rate as 0.

My question is whether this is actually a problem or simply the expected trade-off of the current design.

On a router restart, the collector loses its in-memory sliding window, so there is no previous sample available to compute a rate from. Similarly, if the vLLM pod restarts, the Prometheus counters themselves reset. In both cases, there isn't enough historical information to compute a meaningful rate immediately after restart.

Also, since this metric is intentionally defined as a rolling one-minute average rather than an instantaneous 15s rate, waiting until a full one-minute baseline exists seems consistent with what the metric is trying to represent.

So is the concern specifically that the autoscaler observes 0 instead of "unknown" during that warm-up period? Or is there another issue I'm missing?

Currently, the issue revolves around the fact that it returns 0 instead of unknown

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

kube-gopher

you are adding a unit test case yourself

kube-gopher · 2026-07-09T12:15:18Z

+			if mf.GetType() == io_prometheus_client.MetricType_COUNTER {
+				currentCounterMap[policyKey] = v
+				rate := 0.0
+				now := util.GetCurrentTimestamp()
+				if pastCounterMap != nil && pastScrapeTimestamp > 0 {
+					if prev, ok := pastCounterMap[policyKey]; ok {
+						if v >= prev {
+							elapsedSec := float64(now-pastScrapeTimestamp) / 1000.0
+							if elapsedSec > 0 {
+								rate = (v - prev) / elapsedSec
+							}
+						}
+					}
+				}
+				values[policyKey] += rate
+			} else {
+				if gotValue {
+					values[policyKey] += v
+				}


if mf.GetType() == io_prometheus_client.MetricType_COUNTER { if !gotValue { continue } currentCounterMap[policyKey] = v prev, ok := pastCounterMap[policyKey] if !ok || pastScrapeTimestamp <= 0 { continue } if v < prev { continue } elapsedSec := float64(util.GetCurrentTimestamp()-pastScrapeTimestamp) / 1000.0 if elapsedSec <= 0 { continue } values[policyKey] += (v - prev) / elapsedSec } else { if gotValue { values[policyKey] += v } }

yup i agree 0 could cause fake baseline , instead insufficient metrics as it will already managed by the getDesiredInstancesForSingleInstanceMetric

kube-gopher · 2026-07-09T12:24:06Z

+			err = collector.collectPodMetrics(context.Background(), pod, podSource, wanted, values, pastHistograms, currentHistograms, pastCounters, currentCounters)
+			require.NoError(t, err)
+
+			assert.Equal(t, 0.0, values["my_policy_key"])


_, ok := values["my_policy_key"] assert.False(t, ok)

kube-gopher · 2026-07-09T12:26:44Z

+			err = collector.collectPodMetrics(context.Background(), pod, podSource, wanted, values, pastHistograms, currentHistograms, pastCounters, currentCounters)
+			require.NoError(t, err)
+
+			assert.InDelta(t, tc.want.rate, values["my_policy_key"], 1e-2)


add

} else { _, ok := values["my_policy_key"] assert.False(t, ok) }

kube-gopher · 2026-07-09T12:30:19Z

@nXtCyberNet Here's my thinking

kube-gopher · 2026-07-10T00:29:32Z

+
+	currentCounters[pod.Name] = CounterInfo{
+		PodStartTime:    pod.Status.StartTime,
+		CounterMap:      currentCounterMap,
+		ScrapeTimestamp: util.GetCurrentTimestamp(),
+	}


Counter-rate calculations are inaccurate when a pod is scraped through more than one metric source because the implementation retains only one timestamp per pod. suggest optimizing it

I agree this would make the collector more robust. I do have one question, though.

My understanding was that for autoscaling, the counter metric would typically come directly from the vLLM metrics endpoint, which acts as the single source of truth. In that case, I would expect each pod to expose only one relevant counter source for the autoscaler.

Could you elaborate on the scenarios where we'd intentionally scrape counter metrics from multiple endpoints of the same pod? If both the vLLM container and a sidecar exposed a similar counter, wouldn't we generally want to rely on the vLLM metric as the authoritative source? I'm wondering if I'm missing a valid use case or misunderstanding the intended design.

I understand that the API allows this configuration, but I'm trying to understand the motivation and intended deployment scenarios for supporting it. I may be missing a use case here.

I refuse to respond to AI-generated portions. Please directly store the timestamp and its corresponding counter value for each policy key/source combination

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

hzxuzhonghu · 2026-07-10T09:47:10Z

+				currentCounterMap[policyKey] = v
+
+				prev, ok := pastCounterMap[policyKey]
+				if !ok || pastScrapeTimestamp <= 0 {


When one selected pod has no baseline, this only skips that pod. values can still contain the same policy key from another pod, so the autoscaler emits a partial rate as though every ready replica were represented. Please suppress the policy metric for the whole pod group until every selected pod has a valid delta, and cover the mixed baseline/no-baseline case.

@nXtCyberNet It is worth repairing

can i add this as check for all the signals including histogram and gauge or only for counter by updating collectPodMetricsGroup

kube-gopher

After fixing these two issues, it can be merged

kube-gopher · 2026-07-10T12:38:37Z

It might be an indentation issue. I suggest running the tests locally now

Copilot AI review requested due to automatic review settings May 13, 2026 16:34

volcano-sh-bot added kind/bug kind/enhancement New feature or request labels May 13, 2026

volcano-sh-bot requested review from YaoZengzeng and git-malu May 13, 2026 16:34

volcano-sh-bot added the size/M label May 13, 2026

Copilot started reviewing on behalf of nXtCyberNet May 13, 2026 16:34 View session

Copilot AI reviewed May 13, 2026

gemini-code-assist Bot reviewed May 13, 2026

View reviewed changes

hzxuzhonghu mentioned this pull request Jun 24, 2026

Release v1.0.0 #1251

Open

7 tasks

nXtCyberNet force-pushed the issue/counter branch from 17fb007 to a328a23 Compare June 25, 2026 06:03

volcano-sh-bot added size/L and removed size/M labels Jun 25, 2026

kube-gopher reviewed Jun 25, 2026

View reviewed changes

volcano-sh-bot added the do-not-merge/contains-merge-commits label Jun 25, 2026

added counter delta

925c5f4

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

nXtCyberNet added 6 commits June 25, 2026 22:22

added counter delta

159dba5

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

.

5a9fd85

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

added test

3186790

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

refactor to made counter and histogram different

065291a

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

updated lint

a4a7c18

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

restore the metric lookup

ec0bc62

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

nXtCyberNet force-pushed the issue/counter branch from fafd93f to ec0bc62 Compare June 25, 2026 16:56

volcano-sh-bot removed the do-not-merge/contains-merge-commits label Jun 25, 2026

hzxuzhonghu requested a review from kube-gopher July 7, 2026 08:55

kube-gopher reviewed Jul 7, 2026

View reviewed changes

hzxuzhonghu reviewed Jul 8, 2026

View reviewed changes

nXtCyberNet requested a review from hzxuzhonghu July 8, 2026 14:00

first metrics as baseline

8624ca9

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

Copilot AI review requested due to automatic review settings July 8, 2026 14:51

Copilot AI reviewed Jul 8, 2026

kube-gopher reviewed Jul 9, 2026

View reviewed changes

kube-gopher reviewed Jul 10, 2026

View reviewed changes

removed baseline as 0

45173ff

Signed-off-by: nXtCyberNet <rohantech2005@gmail.com>

Copilot AI review requested due to automatic review settings July 10, 2026 08:50

Copilot AI reviewed Jul 10, 2026

hzxuzhonghu reviewed Jul 10, 2026

View reviewed changes

kube-gopher reviewed Jul 10, 2026

View reviewed changes

Uh oh!

Conversation

nXtCyberNet commented May 13, 2026

Uh oh!

volcano-sh-bot commented May 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

hzxuzhonghu commented May 18, 2026

Uh oh!

hzxuzhonghu commented May 18, 2026

Uh oh!

nXtCyberNet commented May 18, 2026

Uh oh!

hzxuzhonghu commented May 18, 2026

Uh oh!

nXtCyberNet commented May 18, 2026

Uh oh!

nXtCyberNet commented Jun 15, 2026

Uh oh!

nXtCyberNet commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kube-gopher commented Jun 23, 2026

Uh oh!

hzxuzhonghu commented Jun 24, 2026

Uh oh!

kube-gopher commented Jun 24, 2026

Uh oh!

nXtCyberNet commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nXtCyberNet commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nXtCyberNet commented Jun 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nXtCyberNet Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nXtCyberNet Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nXtCyberNet Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nXtCyberNet commented Jun 23, 2026 •

edited

Loading

nXtCyberNet commented Jun 24, 2026 •

edited

Loading

nXtCyberNet commented Jun 25, 2026 •

edited

Loading

nXtCyberNet Jul 8, 2026 •

edited

Loading

nXtCyberNet Jul 8, 2026 •

edited

Loading

nXtCyberNet Jul 8, 2026 •

edited

Loading

kube-gopher Jul 8, 2026 •

edited

Loading

nXtCyberNet Jul 8, 2026 •

edited

Loading

kube-gopher commented Jul 8, 2026 •

edited

Loading

nXtCyberNet Jul 10, 2026 •

edited

Loading