diff --git a/docs/usage.md b/docs/usage.md
index 54d8ef39..b8949250 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -285,6 +285,147 @@ Today is 27 Aug:
 
 If a changepoint is detected in the first 5 data points, Orion expands the lookback window, re-runs the analysis, and reports based on that expanded result.
 
+## Confidence Indicators
+
+Every detected changepoint is automatically annotated with a statistical confidence indicator that combines two measures:
+
+- **Cohen's d effect size** (primary) — magnitude of the difference relative to data variability. This drives the label classification.
+- **Welch's t-test p-value** (descriptive) — probability of observing a difference this extreme if there were no real before/after change. Reported alongside the effect size but does not gate the label.
+
+Cohen's d is used as the primary indicator because the changepoint was already detected by the analysis algorithm — re-testing the same data with a t-test would inflate significance (post-selection bias). The effect size describes *how large* the shift is, independent of the detection method.
+
+These produce a human-readable label shown in the "Affected Metrics" summary table and embedded in JSON output.
+
+### Label Format
+
+Labels are driven by Cohen's d effect size thresholds (Cohen 1988: 0.2 small, 0.5 medium, 0.8 large). The p-value is included as descriptive context only — it is **not** calibrated for post-selection inference (the changepoint was already selected from this same data):
+
+| Label | Meaning |
+|-------|---------|
+| `Large shift (d=1.20, p=0.001)` | d >= 0.8 — strong evidence of a meaningful metric shift |
+| `Moderate shift (d=0.60, p=0.03)` | 0.5 <= d < 0.8 — moderate-magnitude shift |
+| `Small shift (d=0.30, p=0.02)` | 0.2 <= d < 0.5 — small but detectable effect |
+| `Negligible shift (d=0.10, p=0.01)` | d < 0.2 — negligible practical impact |
+| `Degenerate variance — shift detected but effect size undefined` | Both segments have zero variance with different means; Cohen's d is undefined |
+| `Insufficient data` | Fewer than 2 data points on either side of the changepoint |
+| `Anomaly detection — shift confidence not applicable` | IsolationForest detects outliers, not sustained shifts |
+
+The p-value is formatted with `:.2g` to preserve precision (e.g., `5e-10` instead of `0.00`).
+
+### Effect Size (Cohen's d)
+
+Cohen's d is computed using a pooled standard deviation, which assumes the before and after segments have roughly similar variance. This is appropriate for CI performance data where the variance structure tends to be stable across a shift. An alternative (Glass's delta, using only the before-segment standard deviation) could be considered if shifts are expected to also change variance.
+
+When both segments have zero variance but different means, Cohen's d is mathematically undefined. In this case, `cohens_d` is reported as `null` with a "Degenerate variance" label.
+
+### How Segments Are Split
+
+The before/after data segments depend on the algorithm:
+
+- **Hunter (E-Divisive)**: segments are bounded by neighboring changepoints. For a changepoint at index *i* with previous changepoint at *p* and next changepoint at *n*, the before-segment is `data[p:i]` and the after-segment is `data[i:n]`. If there is no previous changepoint, `p` defaults to 0 (start of data). If there is no next changepoint, `n` defaults to the end of data. This prevents a later recovery or reversal from masking an earlier genuine shift.
+- **CMR**: all previous runs vs. the most recent run (CMR typically produces "Insufficient data" since the after segment has only one point)
+- **Anomaly Detection (IsolationForest)**: confidence is not computed. IsolationForest identifies unusual individual observations, not sustained before/after shifts. A separate "Anomaly detection" label is assigned.
+
+### Text Output
+
+The "Affected Metrics" summary table includes a Confidence column:
+
+```text
+Affected Metrics
++---------+-------+----------+------------------------------------------+--------+
+| Metric  | Value | % Change | Confidence                               | Labels |
++---------+-------+----------+------------------------------------------+--------+
+| ovnCPU  | 2.43  | 64.14%   | Large shift (d=1.20, p=0.001)            | [infra] |
+| etcdCPU | 3.50  | 1.18%    | Negligible shift (d=0.15, p=0.42)        | [etcd] |
++---------+-------+----------+------------------------------------------+--------+
+```
+
+### JSON Output
+
+Each changepoint entry in JSON output includes a `confidence` object (illustrative values):
+
+```json
+{
+  "is_changepoint": true,
+  "metrics": {
+    "ovnCPU_avg": {
+      "value": 2.43,
+      "percentage_change": 64.14,
+      "confidence": {
+        "p_value": 0.001,
+        "cohens_d": 1.2,
+        "label": "Large shift (d=1.20, p=0.001)",
+        "sufficient_data": true,
+        "sample_size_before": 15,
+        "sample_size_after": 5,
+        "mean_before": 1.48,
+        "mean_after": 2.43,
+        "std_before": 0.12,
+        "std_after": 0.18,
+        "ci_95": [0.82, 1.08]
+      }
+    }
+  }
+}
+```
+
+The `ci_95` field is always present. When confidence cannot be computed, it is `null`:
+
+```json
+{
+  "confidence": {
+    "p_value": null,
+    "cohens_d": null,
+    "label": "Insufficient data",
+    "sufficient_data": false,
+    "sample_size_before": 3,
+    "sample_size_after": 1,
+    "mean_before": 10.5,
+    "mean_after": 20.0,
+    "std_before": 0.5,
+    "std_after": null,
+    "ci_95": null
+  }
+}
+```
+
+### Confidence Fields Reference
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `p_value` | float\|null | Welch's t-test p-value (descriptive only — not calibrated for post-selection inference). Null when insufficient data or degenerate variance. |
+| `cohens_d` | float\|null | Cohen's d effect size (pooled std). Measures the magnitude of the shift relative to data variability. Null when insufficient data, degenerate variance, or IsolationForest. |
+| `label` | string | Human-readable confidence label (see Label Format above). |
+| `sufficient_data` | bool | Whether both segments had at least 2 data points for statistical computation. |
+| `sample_size_before` | int | Number of data points before the changepoint (after NaN removal). |
+| `sample_size_after` | int | Number of data points from the changepoint onward (after NaN removal). |
+| `mean_before` | float\|null | Mean of the before-segment values. |
+| `mean_after` | float\|null | Mean of the after-segment values. |
+| `std_before` | float\|null | Sample standard deviation of the before-segment (ddof=1). Null if fewer than 2 points. |
+| `std_after` | float\|null | Sample standard deviation of the after-segment (ddof=1). Null if fewer than 2 points. |
+| `ci_95` | [float, float]\|null | 95% confidence interval for the mean difference (mean_after − mean_before), computed using the Welch-Satterthwaite degrees of freedom. Descriptive only — not calibrated for post-selection inference. Null when insufficient data or when standard error is zero. Always present in the output (never omitted). |
+
+The `mean_before`, `mean_after`, `std_before`, and `std_after` fields are the exact values used to compute both `p_value` and `cohens_d`. Combined with the sample sizes, any consumer can independently reproduce the pooled standard deviation, t-statistic, and confidence interval.
+
+The `ci_95` and `p_value` fields are provided as descriptive context. Because the changepoint was selected from the same data, these values are subject to post-selection bias and should not be interpreted as calibrated hypothesis tests. Use Cohen's d (effect size) as the primary indicator of shift magnitude.
+
+### Standalone Reports
+
+When generating reports from JSON files with `--report`, confidence data is propagated from the JSON into the summary tables automatically — no additional flags needed.
+
+### Interpreting Results
+
+Use confidence indicators to triage changepoints for evidence of meaningful metric shifts:
+
+1. **Large shift** — investigate immediately; strong evidence of a meaningful metric shift
+2. **Moderate shift** — investigate; meaningful shift that warrants attention
+3. **Small shift** — detectable but small; may be acceptable depending on the metric's sensitivity
+4. **Negligible shift** — the detected change is too small to matter in practice
+5. **Insufficient data** — not enough data points to compute statistics (common with CMR or very recent runs)
+6. **Anomaly detection** — IsolationForest results; the algorithm detects outliers, not sustained shifts
+
+A confidence label indicates evidence of a metric shift, not necessarily that product code caused a regression. Environment changes, workload variations, or measurement differences could also explain the shift — always investigate the cause.
+
 ## Node Count Filtering
 
 ### Relaxed Matching
diff --git a/orion/algorithms/algorithm.py b/orion/algorithms/algorithm.py
index e876190b..978686d1 100644
--- a/orion/algorithms/algorithm.py
+++ b/orion/algorithms/algorithm.py
@@ -22,6 +22,15 @@ def __init__(
         self.regression_flag = False
         self._acked_logged = False
         self._cached_analysis = None
+        self.raw_change_points_by_metric = None
+        self._original_dataframe = None
+
+    @property
+    def original_dataframe(self):
+        """Pre-analysis dataframe. CMR sets _original_dataframe before collapsing."""
+        if self._original_dataframe is not None:
+            return self._original_dataframe
+        return self.dataframe
 
     def get_analysis_results(self):
         """Return (series, change_points_by_metric) from _analyze(),
diff --git a/orion/algorithms/cmr/cmr.py b/orion/algorithms/cmr/cmr.py
index eb6417f6..181b07f0 100644
--- a/orion/algorithms/cmr/cmr.py
+++ b/orion/algorithms/cmr/cmr.py
@@ -2,7 +2,6 @@
 
 # pylint: disable = line-too-long
 import pandas as pd
-import numpy
 
 from otava.analysis import TTestStats
 from otava.series import  ChangePoint
@@ -37,6 +36,7 @@ def _analyze(self):
             series.data = self.dataframe
             return series, {}
         # if larger than 2 rows, need to get the mean of 0 through -2
+        self._original_dataframe = self.dataframe.copy()
         self.dataframe = self.combine_and_average_runs(self.dataframe)
 
         series= self.setup_series()
@@ -76,14 +76,19 @@ def run_cmr(self, dataframe_list: pd.DataFrame):
         change_points_by_metric={ k:[] for k in metric_columns }
 
         for column in metric_columns:
+            try:
+                m1 = float(dataframe_list[column][0])
+                m2 = float(dataframe_list[column][1])
+            except (ValueError, TypeError):
+                continue
 
             change_point = ChangePoint(metric=column,
                                 index=1,
                                 qhat=0.0,
                                 time=0,
                                 stats=TTestStats(
-                                        mean_1=dataframe_list[column][0],
-                                        mean_2=dataframe_list[column][1],
+                                        mean_1=m1,
+                                        mean_2=m2,
                                         std_1=0.0,
                                         std_2=0.0,
                                         pvalue=1.0
@@ -112,15 +117,17 @@ def combine_and_average_runs(self, dataFrame: pd.DataFrame):
 
         metric_columns = list(dataFrame.columns)
         for column in metric_columns:
-
-            if isinstance(dF.loc[0, column], (numpy.float64, numpy.int64)):
-                mean = dF[column].mean()
-                data2[column] = [mean]
-            else:
-                column_list = dF[column].tolist()
-                # Convert each item to string to handle lists, UUIDs, and other non-string types
-                non_numeric_joined_list = ','.join(str(item) for item in column_list)
-                data2[column] = [non_numeric_joined_list]
+            try:
+                numeric_col = pd.to_numeric(dF[column])
+                data2[column] = [numeric_col.mean()]
+            except (ValueError, TypeError):
+                numeric_col = pd.to_numeric(dF[column], errors='coerce')
+                if numeric_col.notna().any():
+                    data2[column] = [numeric_col.mean()]
+                else:
+                    column_list = dF[column].tolist()
+                    non_numeric_joined_list = ','.join(str(item) for item in column_list)
+                    data2[column] = [non_numeric_joined_list]
             i += 1
         df2 = pd.DataFrame(data2)
 
diff --git a/orion/algorithms/edivisive/edivisive.py b/orion/algorithms/edivisive/edivisive.py
index 19c76adb..9917b23d 100644
--- a/orion/algorithms/edivisive/edivisive.py
+++ b/orion/algorithms/edivisive/edivisive.py
@@ -38,6 +38,10 @@ def _analyze(self):
             logger.info("ACKed UUIDs: %s", ", ".join(acked_uuids))
             self._acked_logged = True
 
+        self.raw_change_points_by_metric = {
+            metric: list(cps) for metric, cps in change_points_by_metric.items()
+        }
+
         # filter by direction and ack'ed issues
         for metric, changepoint_list in change_points_by_metric.items():
             for i in range(len(changepoint_list)-1, -1, -1):
diff --git a/orion/confidence.py b/orion/confidence.py
new file mode 100644
index 00000000..dd1a71a8
--- /dev/null
+++ b/orion/confidence.py
@@ -0,0 +1,211 @@
+"""Statistical confidence indicators for changepoints."""
+
+import math
+from dataclasses import dataclass
+from typing import Optional
+
+import numpy as np
+from scipy import stats
+
+import orion.constants as cnsts
+
+
+@dataclass
+class ConfidenceResult: # pylint: disable=too-many-instance-attributes
+    """Statistical confidence for a single changepoint."""
+
+    p_value: Optional[float]
+    cohens_d: Optional[float]
+    confidence_label: str
+    sufficient_data: bool
+    sample_size_before: int
+    sample_size_after: int
+    mean_before: Optional[float] = None
+    mean_after: Optional[float] = None
+    std_before: Optional[float] = None
+    std_after: Optional[float] = None
+    ci_95: Optional[tuple] = None
+
+    def to_dict(self):
+        """Return dict suitable for JSON/regression-data output."""
+        return {
+            "p_value": self.p_value,
+            "cohens_d": self.cohens_d,
+            "label": self.confidence_label,
+            "sufficient_data": self.sufficient_data,
+            "sample_size_before": self.sample_size_before,
+            "sample_size_after": self.sample_size_after,
+            "mean_before": self.mean_before,
+            "mean_after": self.mean_after,
+            "std_before": self.std_before,
+            "std_after": self.std_after,
+            "ci_95": list(self.ci_95) if self.ci_95 is not None else None,
+        }
+
+
+def _map_label(p_value, cohens_d):
+    """Map Cohen's d to a human-readable effect-size label.
+
+    Cohen's d drives the label tier. p-value is included as
+    descriptive context only — it is NOT calibrated for post-selection
+    inference (the changepoint was selected from this same data).
+    """
+    if cohens_d is None:
+        return "Degenerate variance — shift detected but effect size undefined"
+
+    d_str = f"{cohens_d:.2f}"
+    p_str = f"{p_value:.2g}" if p_value is not None else "n/a"
+    if cohens_d >= 0.8:
+        return f"Large shift (d={d_str}, p={p_str})"
+    if cohens_d >= 0.5:
+        return f"Moderate shift (d={d_str}, p={p_str})"
+    if cohens_d >= 0.2:
+        return f"Small shift (d={d_str}, p={p_str})"
+    return f"Negligible shift (d={d_str}, p={p_str})"
+
+
+def _get_segments(algorithm_name, data, changepoint_index,
+                  prev_boundary=0, next_boundary=None):
+    """Split data into before/after segments based on algorithm type.
+
+    Segments are bounded by neighboring changepoints to prevent
+    a later recovery from masking an earlier genuine shift.
+    """
+    if next_boundary is None:
+        next_boundary = len(data)
+    if algorithm_name == cnsts.CMR:
+        return data[:-1], data[-1:]
+    return data[prev_boundary:changepoint_index], data[changepoint_index:next_boundary]
+
+
+def _compute_stats(before, after):
+    """Compute Welch's t-test and Cohen's d for two data segments."""
+    n_before = len(before)
+    n_after = len(after)
+
+    if n_before < 2 or n_after < 2:
+        return ConfidenceResult(
+            p_value=None,
+            cohens_d=None,
+            confidence_label="Insufficient data",
+            sufficient_data=False,
+            sample_size_before=n_before,
+            sample_size_after=n_after,
+            mean_before=float(np.mean(before)) if n_before > 0 else None,
+            mean_after=float(np.mean(after)) if n_after > 0 else None,
+            std_before=float(np.std(before, ddof=1)) if n_before > 1 else None,
+            std_after=float(np.std(after, ddof=1)) if n_after > 1 else None,
+        )
+
+    mean_before = np.mean(before)
+    mean_after = np.mean(after)
+    std_before = np.std(before, ddof=1)
+    std_after = np.std(after, ddof=1)
+    pooled_std = math.sqrt(
+        ((n_before - 1) * std_before ** 2 + (n_after - 1) * std_after ** 2)
+        / (n_before + n_after - 2)
+    )
+
+    if pooled_std == 0 and mean_before != mean_after:
+        p_value = None
+        cohens_d = None
+    elif pooled_std == 0:
+        p_value = 1.0
+        cohens_d = 0.0
+    else:
+        _, p_value = stats.ttest_ind(before, after, equal_var=False)
+        cohens_d = abs(mean_after - mean_before) / pooled_std
+        if math.isnan(p_value):
+            p_value = 1.0
+
+    label = _map_label(p_value, cohens_d)
+
+    ci_95 = None
+    se = math.sqrt(std_before ** 2 / n_before + std_after ** 2 / n_after)
+    if se > 0:
+        mean_diff = float(mean_after - mean_before)
+        df_num = (std_before ** 2 / n_before + std_after ** 2 / n_after) ** 2
+        df_den = (
+            (std_before ** 2 / n_before) ** 2 / (n_before - 1)
+            + (std_after ** 2 / n_after) ** 2 / (n_after - 1)
+        )
+        welch_df = df_num / df_den if df_den > 0 else 1.0
+        t_crit = stats.t.ppf(0.975, welch_df)
+        ci_95 = (mean_diff - t_crit * se, mean_diff + t_crit * se)
+
+    return ConfidenceResult(
+        p_value=p_value,
+        cohens_d=cohens_d,
+        confidence_label=label,
+        sufficient_data=True,
+        sample_size_before=n_before,
+        sample_size_after=n_after,
+        mean_before=float(mean_before),
+        mean_after=float(mean_after),
+        std_before=float(std_before),
+        std_after=float(std_after),
+        ci_95=ci_95,
+    )
+
+
+def compute_confidence(algorithm_name, dataframe, change_points_by_metric,
+                       raw_change_points_by_metric=None):
+    """Compute confidence indicators for all changepoints.
+
+    raw_change_points_by_metric: unfiltered detector boundaries. When
+    provided, segment boundaries are derived from ALL detector-found
+    changepoints so that a filtered recovery still bounds the window.
+
+    Returns dict keyed by metric name, index-aligned with
+    change_points_by_metric.
+    """
+    result = {}
+
+    if algorithm_name == cnsts.ISOLATION_FOREST:
+        for metric, cps in change_points_by_metric.items():
+            result[metric] = [
+                ConfidenceResult(
+                    p_value=None, cohens_d=None,
+                    confidence_label=(
+                        "Anomaly detection — shift confidence not applicable"
+                    ),
+                    sufficient_data=False,
+                    sample_size_before=0, sample_size_after=0,
+                ) for _ in cps
+            ]
+        return result
+
+    for metric, cps in change_points_by_metric.items():
+        if metric not in dataframe.columns:
+            result[metric] = [
+                ConfidenceResult(
+                    p_value=None, cohens_d=None,
+                    confidence_label="Insufficient data",
+                    sufficient_data=False,
+                    sample_size_before=0, sample_size_after=0,
+                ) for _ in cps
+            ]
+            continue
+
+        data = dataframe[metric].values
+        raw_cps = (raw_change_points_by_metric or {}).get(metric, cps)
+        all_boundaries = sorted(set(cp.index for cp in raw_cps))
+
+        metric_results = []
+        for cp in cps:
+            prev_boundary = 0
+            next_boundary = len(data)
+            for b in all_boundaries:
+                if b < cp.index:
+                    prev_boundary = b
+                elif b > cp.index:
+                    next_boundary = b
+                    break
+            before, after = _get_segments(
+                algorithm_name, data, cp.index, prev_boundary, next_boundary
+            )
+            before = before[~np.isnan(before)]
+            after = after[~np.isnan(after)]
+            metric_results.append(_compute_stats(before, after))
+        result[metric] = metric_results
+    return result
diff --git a/orion/pipeline/analysis_result.py b/orion/pipeline/analysis_result.py
index 019dfddd..371e095d 100644
--- a/orion/pipeline/analysis_result.py
+++ b/orion/pipeline/analysis_result.py
@@ -1,6 +1,6 @@
 """AnalysisResult dataclass and standalone utility functions."""
 
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from itertools import groupby
 from typing import Dict, List
 import pandas as pd
@@ -30,6 +30,7 @@ class AnalysisResult: # pylint: disable=too-many-instance-attributes
     version_field: str
     sippy_pr_search: bool
     github_repos: list
+    confidence_by_metric: dict = field(default_factory=dict)
 
 
 def group_change_points_by_time(
diff --git a/orion/pipeline/formatters/base.py b/orion/pipeline/formatters/base.py
index c0649987..ff985ba1 100644
--- a/orion/pipeline/formatters/base.py
+++ b/orion/pipeline/formatters/base.py
@@ -49,7 +49,7 @@ def extract_regression_data(self, data: AnalysisResult) -> list:
         github_client = self._get_github_client(data.github_repos)
 
         for metric, cps in data.change_points_by_metric.items():
-            for cp in cps:
+            for cp_idx, cp in enumerate(cps):
                 index = cp.index
                 percentage_change = (
                     (cp.stats.mean_2 - cp.stats.mean_1)
@@ -66,6 +66,10 @@ def extract_regression_data(self, data: AnalysisResult) -> list:
                                 "labels": data.metrics_config[metric].get(
                                     "labels") or [],
                             })
+                            self._add_confidence(
+                                data, metric, cp_idx,
+                                reg["metrics_with_change"][-1]
+                            )
                     continue
 
                 seen_indices.add(index)
@@ -96,6 +100,10 @@ def extract_regression_data(self, data: AnalysisResult) -> list:
                     "uuid": row.get(data.uuid_field),
                     "timestamp": row.get("timestamp"),
                 }
+                self._add_confidence(
+                    data, metric, cp_idx,
+                    doc["metrics_with_change"][-1]
+                )
                 prs = row.get("prs")
                 if prs is not None:
                     doc["prs"] = prs
@@ -128,6 +136,13 @@ def extract_regression_data(self, data: AnalysisResult) -> list:
 
         return regression_data
 
+    @staticmethod
+    def _add_confidence(data, metric, cp_idx, metric_entry):
+        """Add confidence data to a metric entry if available."""
+        confidences = data.confidence_by_metric.get(metric, [])
+        if cp_idx < len(confidences):
+            metric_entry["confidence"] = confidences[cp_idx].to_dict()
+
     @staticmethod
     def _get_github_client(
         github_repos: list,
diff --git a/orion/pipeline/formatters/json_formatter.py b/orion/pipeline/formatters/json_formatter.py
index c581cc84..4b9c1f1e 100644
--- a/orion/pipeline/formatters/json_formatter.py
+++ b/orion/pipeline/formatters/json_formatter.py
@@ -22,6 +22,7 @@ def format(self, data: AnalysisResult) -> dict:
                 key: {
                     "value": entry.pop(key),
                     "percentage_change": 0,
+                    "is_changepoint": False,
                     "labels": value["labels"] if value["labels"] else [],
                 }
                 for key, value in data.metrics_config.items()
@@ -31,7 +32,8 @@ def format(self, data: AnalysisResult) -> dict:
         github_client = BaseFormatter._get_github_client(data.github_repos)
 
         for key, value in data.change_points_by_metric.items():
-            for change_point in value:
+            confidences = data.confidence_by_metric.get(key, [])
+            for cp_idx, change_point in enumerate(value):
                 index = change_point.index
                 percentage_change = (
                     (change_point.stats.mean_2 - change_point.stats.mean_1)
@@ -40,7 +42,14 @@ def format(self, data: AnalysisResult) -> dict:
                 dataframe_json[index]["metrics"][key][
                     "percentage_change"
                 ] = percentage_change
+                dataframe_json[index]["metrics"][key][
+                    "is_changepoint"
+                ] = True
                 dataframe_json[index]["is_changepoint"] = True
+                if cp_idx < len(confidences):
+                    dataframe_json[index]["metrics"][key][
+                        "confidence"
+                    ] = confidences[cp_idx].to_dict()
                 if data.collapse:
                     if (
                         index > 0
diff --git a/orion/reporting/standalone.py b/orion/reporting/standalone.py
index 79471053..96982e34 100644
--- a/orion/reporting/standalone.py
+++ b/orion/reporting/standalone.py
@@ -80,13 +80,16 @@ def extract_regression_data(workload: str, data: list[dict]) -> list[dict]:
 
         metrics_with_change = []
         for name, info in entry.get("metrics", {}).items():
-            if info.get("percentage_change", 0) != 0:
-                metrics_with_change.append({
+            if info.get("is_changepoint", info.get("percentage_change", 0) != 0):
+                entry_dict = {
                     "name": name,
                     "value": info.get("value"),
                     "percentage_change": info.get("percentage_change", 0),
                     "labels": info.get("labels", ""),
-                })
+                }
+                if "confidence" in info:
+                    entry_dict["confidence"] = info["confidence"]
+                metrics_with_change.append(entry_dict)
 
         if not metrics_with_change:
             continue
diff --git a/orion/reporting/summary.py b/orion/reporting/summary.py
index 92a1a6c8..abeceef3 100644
--- a/orion/reporting/summary.py
+++ b/orion/reporting/summary.py
@@ -31,13 +31,19 @@ def print_regression_summary(regression_data) -> None:
             print(f"{'Build:':<20} {regression['build_url']}")
         print("\nAffected Metrics")
         if regression['metrics_with_change']:
-            table = [
-                [m['name'], m['value'], f"{m['percentage_change']:.2f}%", m.get('labels', '')]
-                for m in regression['metrics_with_change']
-            ]
+            table = []
+            for m in regression['metrics_with_change']:
+                conf = m.get('confidence', {})
+                table.append([
+                    m['name'], m['value'],
+                    f"{m['percentage_change']:.2f}%",
+                    conf.get('label', ''),
+                    m.get('labels', ''),
+                ])
             print(tabulate(
                 table,
-                headers=["Metric", "Value", "Percentage change", "Labels"],
+                headers=["Metric", "Value", "% Change",
+                         "Confidence", "Labels"],
                 tablefmt="outline"
             ))
 
diff --git a/orion/run_test.py b/orion/run_test.py
index a817061b..6f5616be 100644
--- a/orion/run_test.py
+++ b/orion/run_test.py
@@ -13,6 +13,7 @@
 from orion.github_client import GitHubClient
 from orion.visualization import VizData
 from orion.pipeline.analysis_result import AnalysisResult
+from orion.confidence import compute_confidence
 
 
 class TestResults(NamedTuple):
@@ -234,8 +235,25 @@ def analyze(test, kwargs, is_pull=False):
 
     logger.info("Comparison algorithm: %s", algorithm_name)
 
+    iforest_nan_cols = []
+    iforest_nan_configs = {}
     if algorithm_name == cnsts.ISOLATION_FOREST:
-        fingerprint_matched_df = fingerprint_matched_df.dropna().reset_index()
+        iforest_nan_cols = fingerprint_matched_df.columns[
+            fingerprint_matched_df.isna().all()
+        ].tolist()
+        if iforest_nan_cols:
+            logger.warning(
+                "Dropping all-NaN columns before Isolation Forest: %s",
+                iforest_nan_cols,
+            )
+            fingerprint_matched_df = fingerprint_matched_df.drop(
+                columns=iforest_nan_cols
+            )
+            for col in iforest_nan_cols:
+                if col in metrics_config:
+                    iforest_nan_configs[col] = metrics_config.pop(col)
+        fingerprint_matched_df = fingerprint_matched_df.dropna().reset_index(drop=True)
+        metrics = list(metrics_config.keys())
 
     algorithm_factory = AlgorithmFactory()
     algorithm = algorithm_factory.instantiate_algorithm(
@@ -293,8 +311,18 @@ def analyze(test, kwargs, is_pull=False):
             and expanded_points > len(fingerprint_matched_df)
         ):
             if algorithm_name == cnsts.ISOLATION_FOREST:
+                exp_nan = expanded_fingerprint_matched_df.columns[
+                    expanded_fingerprint_matched_df.isna().all()
+                ].tolist()
+                if exp_nan:
+                    expanded_fingerprint_matched_df = (
+                        expanded_fingerprint_matched_df.drop(columns=exp_nan)
+                    )
+                    iforest_nan_cols = list(
+                        set(iforest_nan_cols) | set(exp_nan)
+                    )
                 expanded_fingerprint_matched_df = (
-                    expanded_fingerprint_matched_df.dropna().reset_index()
+                    expanded_fingerprint_matched_df.dropna().reset_index(drop=True)
                 )
 
             expanded_algorithm = algorithm_factory.instantiate_algorithm(
@@ -363,6 +391,11 @@ def analyze(test, kwargs, is_pull=False):
             acked_entries=acked_entries,
         )
 
+    if iforest_nan_configs:
+        for col, cfg in iforest_nan_configs.items():
+            final_algorithm.dataframe[col] = float("nan")
+            metrics_config[col] = cfg
+
     series = final_algorithm.setup_series()
 
     min_cp_index = None
@@ -376,6 +409,13 @@ def analyze(test, kwargs, is_pull=False):
     else:
         avg_values = final_algorithm.dataframe[metrics].mean()
 
+    confidence_by_metric = compute_confidence(
+        algorithm_name,
+        final_algorithm.original_dataframe,
+        change_points_by_metric,
+        raw_change_points_by_metric=final_algorithm.raw_change_points_by_metric,
+    )
+
     analysis_result = AnalysisResult(
         test_name=test["name"],
         test=test,
@@ -392,5 +432,6 @@ def analyze(test, kwargs, is_pull=False):
         version_field=test["version_field"],
         sippy_pr_search=kwargs.get("sippy_pr_search", False),
         github_repos=kwargs.get("github_repos", []),
+        confidence_by_metric=confidence_by_metric,
     )
     return analysis_result, viz_data
diff --git a/orion/tests/test_confidence.py b/orion/tests/test_confidence.py
new file mode 100644
index 00000000..5ea833dd
--- /dev/null
+++ b/orion/tests/test_confidence.py
@@ -0,0 +1,387 @@
+# pylint: disable=missing-class-docstring,missing-function-docstring
+"""Tests for confidence indicators module."""
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from orion.confidence import (
+    ConfidenceResult,
+    _map_label,
+    _get_segments,
+    _compute_stats,
+    compute_confidence,
+)
+import orion.constants as cnsts
+from orion.tests.conftest import make_change_point as _make_cp
+
+
+class TestMapLabel:
+    def test_large_shift(self):
+        assert _map_label(0.01, 1.0) == "Large shift (d=1.00, p=0.01)"
+
+    def test_moderate_shift(self):
+        assert _map_label(0.01, 0.6) == "Moderate shift (d=0.60, p=0.01)"
+
+    def test_small_shift(self):
+        assert _map_label(0.01, 0.3) == "Small shift (d=0.30, p=0.01)"
+
+    def test_negligible_shift(self):
+        assert _map_label(0.01, 0.1) == "Negligible shift (d=0.10, p=0.01)"
+
+    def test_large_d_high_p_still_large_shift(self):
+        label = _map_label(0.3, 1.5)
+        assert label == "Large shift (d=1.50, p=0.3)"
+
+    def test_boundary_p_value_at_005(self):
+        label = _map_label(0.05, 1.0)
+        assert "Large shift" in label
+        assert "p=0.05" in label
+
+    def test_boundary_cohens_d_at_08(self):
+        assert _map_label(0.01, 0.8) == "Large shift (d=0.80, p=0.01)"
+
+    def test_boundary_cohens_d_at_05(self):
+        assert _map_label(0.01, 0.5) == "Moderate shift (d=0.50, p=0.01)"
+
+    def test_boundary_cohens_d_at_02(self):
+        assert _map_label(0.01, 0.2) == "Small shift (d=0.20, p=0.01)"
+
+    def test_none_cohens_d_degenerate(self):
+        label = _map_label(0.001, None)
+        assert "Degenerate variance" in label
+
+    def test_p_value_scientific_notation(self):
+        label = _map_label(5e-10, 1.0)
+        assert "5e-10" in label
+
+    def test_p_value_not_rounded_to_zero(self):
+        label = _map_label(0.001, 1.0)
+        assert "p=0.001" in label
+
+
+class TestConfidenceResult:
+    def test_dataclass_fields(self):
+        result = ConfidenceResult(
+            p_value=0.01,
+            cohens_d=1.2,
+            confidence_label="Large shift (d=1.20, p=0.01)",
+            sufficient_data=True,
+            sample_size_before=10,
+            sample_size_after=5,
+        )
+        assert result.p_value == 0.01
+        assert result.cohens_d == 1.2
+        assert result.sufficient_data is True
+
+    def test_insufficient_data_result(self):
+        result = ConfidenceResult(
+            p_value=None,
+            cohens_d=None,
+            confidence_label="Insufficient data",
+            sufficient_data=False,
+            sample_size_before=5,
+            sample_size_after=1,
+        )
+        assert result.sufficient_data is False
+        assert result.p_value is None
+
+    def test_to_dict_always_includes_ci_95_null(self):
+        result = ConfidenceResult(
+            p_value=None,
+            cohens_d=None,
+            confidence_label="Insufficient data",
+            sufficient_data=False,
+            sample_size_before=0,
+            sample_size_after=0,
+        )
+        d = result.to_dict()
+        assert "ci_95" in d
+        assert d["ci_95"] is None
+
+    def test_to_dict_includes_ci_95_list(self):
+        result = ConfidenceResult(
+            p_value=0.01,
+            cohens_d=1.2,
+            confidence_label="Large shift",
+            sufficient_data=True,
+            sample_size_before=10,
+            sample_size_after=5,
+            ci_95=(10.0, 20.0),
+        )
+        d = result.to_dict()
+        assert d["ci_95"] == [10.0, 20.0]
+
+
+class TestGetSegments:
+    def test_edivisive_splits_at_index(self):
+        data = np.array([1.0, 2.0, 3.0, 10.0, 11.0])
+        before, after = _get_segments(cnsts.EDIVISIVE, data, 3)
+        np.testing.assert_array_equal(before, [1.0, 2.0, 3.0])
+        np.testing.assert_array_equal(after, [10.0, 11.0])
+
+    def test_edivisive_with_boundaries(self):
+        data = np.array([1.0, 2.0, 3.0, 10.0, 11.0, 5.0, 6.0])
+        before, after = _get_segments(cnsts.EDIVISIVE, data, 3, prev_boundary=0, next_boundary=5)
+        np.testing.assert_array_equal(before, [1.0, 2.0, 3.0])
+        np.testing.assert_array_equal(after, [10.0, 11.0])
+
+    def test_cmr_splits_all_previous_vs_last(self):
+        data = np.array([1.0, 2.0, 3.0, 10.0])
+        before, after = _get_segments(cnsts.CMR, data, 3)
+        np.testing.assert_array_equal(before, [1.0, 2.0, 3.0])
+        np.testing.assert_array_equal(after, [10.0])
+
+    def test_edivisive_index_at_start(self):
+        data = np.array([10.0, 1.0, 2.0])
+        before, after = _get_segments(cnsts.EDIVISIVE, data, 0)
+        assert len(before) == 0
+        np.testing.assert_array_equal(after, [10.0, 1.0, 2.0])
+
+    def test_edivisive_index_at_end(self):
+        data = np.array([1.0, 2.0, 10.0])
+        before, after = _get_segments(cnsts.EDIVISIVE, data, 2)
+        np.testing.assert_array_equal(before, [1.0, 2.0])
+        np.testing.assert_array_equal(after, [10.0])
+
+
+class TestComputeStats:
+    def test_clear_regression_produces_low_p_high_d(self):
+        before = np.array([100.0, 101.0, 99.0, 100.5, 100.2,
+                           99.8, 100.1, 99.9, 100.3, 99.7])
+        after = np.array([200.0, 201.0, 199.0, 200.5, 200.2,
+                          199.8, 200.1, 199.9, 200.3, 199.7])
+        result = _compute_stats(before, after)
+        assert result.sufficient_data is True
+        assert result.p_value < 0.05
+        assert result.cohens_d > 0.8
+        assert "Large shift" in result.confidence_label
+        assert result.mean_before == pytest.approx(np.mean(before))
+        assert result.mean_after == pytest.approx(np.mean(after))
+        assert result.std_before == pytest.approx(np.std(before, ddof=1))
+        assert result.std_after == pytest.approx(np.std(after, ddof=1))
+        assert result.ci_95 is not None
+        ci_low, ci_high = result.ci_95
+        assert ci_low > 0
+        assert ci_high > ci_low
+        assert ci_low < 100.0 < ci_high
+
+    def test_identical_data_produces_negligible(self):
+        before = np.array([100.0, 100.0, 100.0, 100.0, 100.0])
+        after = np.array([100.0, 100.0, 100.0, 100.0, 100.0])
+        result = _compute_stats(before, after)
+        assert result.sufficient_data is True
+        assert result.cohens_d == 0.0
+        assert "Negligible shift" in result.confidence_label
+
+    def test_insufficient_data_one_point_after(self):
+        before = np.array([100.0, 101.0, 99.0])
+        after = np.array([200.0])
+        result = _compute_stats(before, after)
+        assert result.sufficient_data is False
+        assert result.p_value is None
+        assert result.cohens_d is None
+        assert result.confidence_label == "Insufficient data"
+        assert result.mean_before == pytest.approx(100.0)
+        assert result.mean_after == pytest.approx(200.0)
+        assert result.std_before is not None
+        assert result.std_after is None
+        assert result.ci_95 is None
+
+    def test_insufficient_data_empty_before(self):
+        before = np.array([])
+        after = np.array([200.0, 201.0])
+        result = _compute_stats(before, after)
+        assert result.sufficient_data is False
+
+    def test_zero_std_different_means_degenerate(self):
+        before = np.array([100.0, 100.0, 100.0])
+        after = np.array([200.0, 200.0, 200.0])
+        result = _compute_stats(before, after)
+        assert result.sufficient_data is True
+        assert result.cohens_d is None
+        assert result.p_value is None
+        assert "Degenerate variance" in result.confidence_label
+
+    def test_zero_std_same_means(self):
+        before = np.array([100.0, 100.0, 100.0])
+        after = np.array([100.0, 100.0, 100.0])
+        result = _compute_stats(before, after)
+        assert result.cohens_d == 0.0
+
+    def test_sample_sizes_recorded(self):
+        before = np.array([1.0, 2.0, 3.0])
+        after = np.array([10.0, 11.0])
+        result = _compute_stats(before, after)
+        assert result.sample_size_before == 3
+        assert result.sample_size_after == 2
+
+
+class TestComputeConfidence:
+    def test_returns_dict_keyed_by_metric(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 9.8, 10.2, 10.1,
+                    20.0, 20.5, 19.8, 20.2, 20.1],
+        })
+        cps = {"cpu": [_make_cp("cpu", 5)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert "cpu" in result
+        assert len(result["cpu"]) == 1
+        assert isinstance(result["cpu"][0], ConfidenceResult)
+
+    def test_index_aligned_with_neighboring_segments(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 9.8, 20.0, 20.5,
+                    30.0, 30.5, 29.8, 30.2, 30.1],
+        })
+        cps = {"cpu": [_make_cp("cpu", 3), _make_cp("cpu", 5)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert len(result["cpu"]) == 2
+        sizes = [
+            (item.sample_size_before, item.sample_size_after)
+            for item in result["cpu"]
+        ]
+        assert sizes == [(3, 2), (2, 5)]
+
+    def test_reversal_detected_independently(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.0, 10.0, 10.0, 10.0,
+                    20.0, 20.0, 20.0, 20.0, 20.0,
+                    10.0, 10.0, 10.0, 10.0, 10.0],
+        })
+        cps = {"cpu": [_make_cp("cpu", 5), _make_cp("cpu", 10)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        first = result["cpu"][0]
+        second = result["cpu"][1]
+        assert first.mean_before == pytest.approx(10.0)
+        assert first.mean_after == pytest.approx(20.0)
+        assert second.mean_before == pytest.approx(20.0)
+        assert second.mean_after == pytest.approx(10.0)
+
+    def test_empty_changepoints_returns_empty(self):
+        df = pd.DataFrame({"cpu": [10.0, 11.0, 12.0]})
+        cps = {"cpu": []}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert result["cpu"] == []
+
+    def test_multiple_metrics(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 20.0, 20.5],
+            "mem": [50.0, 51.0, 100.0, 101.0],
+        })
+        cps = {
+            "cpu": [_make_cp("cpu", 2)],
+            "mem": [_make_cp("mem", 2)],
+        }
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert "cpu" in result
+        assert "mem" in result
+        assert len(result["cpu"]) == 1
+        assert len(result["mem"]) == 1
+
+    def test_same_index_different_metrics_get_different_confidence(self):
+        np.random.seed(99)
+        cpu_vals = np.concatenate([
+            np.random.normal(0.5, 0.05, 8),
+            np.random.normal(0.9, 0.05, 4),
+        ])
+        lat_vals = np.concatenate([
+            np.random.normal(40000, 4000, 8),
+            np.random.normal(70000, 4000, 4),
+        ])
+        df = pd.DataFrame({"cpu": cpu_vals, "latency": lat_vals})
+        cps = {
+            "cpu": [_make_cp("cpu", 8, mean_1=0.5, mean_2=0.9)],
+            "latency": [_make_cp("latency", 8,
+                                 mean_1=40000, mean_2=70000)],
+        }
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        cpu_conf = result["cpu"][0]
+        lat_conf = result["latency"][0]
+        assert cpu_conf.cohens_d != lat_conf.cohens_d
+        assert cpu_conf.p_value != lat_conf.p_value
+
+    def test_cmr_single_point_after_insufficient(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 9.8, 20.0],
+        })
+        cps = {"cpu": [_make_cp("cpu", 3)]}
+        result = compute_confidence(cnsts.CMR, df, cps)
+        assert result["cpu"][0].sufficient_data is False
+        assert result["cpu"][0].confidence_label == "Insufficient data"
+
+    def test_nan_before_changepoint_preserves_alignment(self):
+        df = pd.DataFrame({
+            "cpu": [1.0, np.nan, 2.0, 10.0, 11.0],
+        })
+        cps = {"cpu": [_make_cp("cpu", 3)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert result["cpu"][0].sufficient_data is True
+        assert result["cpu"][0].sample_size_before == 2
+        assert result["cpu"][0].sample_size_after == 2
+
+    def test_isolation_forest_skips_confidence(self):
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 9.8, 10.2, 10.1,
+                    20.0, 20.5, 19.8, 20.2, 20.1],
+        })
+        cps = {"cpu": [_make_cp("cpu", 5)]}
+        result = compute_confidence(cnsts.ISOLATION_FOREST, df, cps)
+        conf = result["cpu"][0]
+        assert conf.p_value is None
+        assert conf.cohens_d is None
+        assert conf.sufficient_data is False
+        assert "Anomaly detection" in conf.confidence_label
+
+    def test_missing_metric_column(self):
+        df = pd.DataFrame({"other": [1.0, 2.0, 3.0]})
+        cps = {"cpu": [_make_cp("cpu", 1)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, cps)
+        assert result["cpu"][0].sufficient_data is False
+        assert result["cpu"][0].confidence_label == "Insufficient data"
+
+    def test_filtered_recovery_uses_raw_boundaries(self):
+        """When a recovery CP is filtered out, raw boundaries still bound the window."""
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.0, 10.0, 10.0, 10.0,
+                    20.0, 20.0, 20.0, 20.0, 20.0,
+                    10.0, 10.0, 10.0, 10.0, 10.0],
+        })
+        filtered_cps = {"cpu": [_make_cp("cpu", 5)]}
+        raw_cps = {"cpu": [_make_cp("cpu", 5), _make_cp("cpu", 10)]}
+        result = compute_confidence(
+            cnsts.EDIVISIVE, df, filtered_cps,
+            raw_change_points_by_metric=raw_cps,
+        )
+        conf = result["cpu"][0]
+        assert conf.mean_before == pytest.approx(10.0)
+        assert conf.mean_after == pytest.approx(20.0)
+        assert conf.sample_size_before == 5
+        assert conf.sample_size_after == 5
+
+    def test_filtered_recovery_without_raw_boundaries_dilutes(self):
+        """Without raw boundaries, a filtered recovery dilutes the shift."""
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.0, 10.0, 10.0, 10.0,
+                    20.0, 20.0, 20.0, 20.0, 20.0,
+                    10.0, 10.0, 10.0, 10.0, 10.0],
+        })
+        filtered_cps = {"cpu": [_make_cp("cpu", 5)]}
+        result = compute_confidence(cnsts.EDIVISIVE, df, filtered_cps)
+        conf = result["cpu"][0]
+        assert conf.sample_size_after == 10
+        assert conf.mean_after == pytest.approx(15.0)
+
+    def test_cmr_original_dataframe_preserves_before_stats(self):
+        """CMR with original (uncollapsed) dataframe preserves before-segment stats."""
+        df = pd.DataFrame({
+            "cpu": [10.0, 10.5, 9.8, 20.0],
+        })
+        cps = {"cpu": [_make_cp("cpu", 3)]}
+        result = compute_confidence(cnsts.CMR, df, cps)
+        conf = result["cpu"][0]
+        assert conf.sample_size_before == 3
+        assert conf.sample_size_after == 1
+        assert conf.mean_before == pytest.approx(np.mean([10.0, 10.5, 9.8]))
+        assert conf.std_before is not None
+        assert conf.sufficient_data is False
diff --git a/orion/tests/test_formatters.py b/orion/tests/test_formatters.py
index a044f248..5878e680 100644
--- a/orion/tests/test_formatters.py
+++ b/orion/tests/test_formatters.py
@@ -4,10 +4,13 @@
 import json
 import xml.etree.ElementTree as ET
 
+import numpy as np
 import pandas as pd
 import pytest
 from otava.series import Series, Metric
 
+import orion.constants as cnsts
+from orion.confidence import ConfidenceResult, compute_confidence
 from orion.pipeline.analysis_result import AnalysisResult
 from orion.pipeline.formatters import FormatterFactory
 from orion.pipeline.formatters.base import BaseFormatter
@@ -17,6 +20,7 @@
     TextFormatter,
     _format_comparison_table,
 )
+from orion.reporting.summary import print_regression_summary
 from orion.tests.conftest import make_change_point
 
 
@@ -185,6 +189,20 @@ def test_format_non_changepoint_has_zero_percentage(self):
             for metric_data in record["metrics"].values():
                 assert metric_data["percentage_change"] == 0
 
+    def test_format_sets_per_metric_is_changepoint(self):
+        data = _make_analysis_result()
+        formatter = JsonFormatter()
+        result = formatter.format(data)
+        parsed = json.loads(result["test-workload"])
+
+        cp_record = [r for r in parsed if r["is_changepoint"]][0]
+        assert cp_record["metrics"]["cpu"]["is_changepoint"] is True
+
+        non_cp = [r for r in parsed if not r["is_changepoint"]]
+        for record in non_cp:
+            for metric_data in record["metrics"].values():
+                assert metric_data["is_changepoint"] is False
+
     def test_format_collapse_returns_context_only(self):
         data = _make_analysis_result()
         data.collapse = True
@@ -515,6 +533,307 @@ def test_junit_pr_output_skips_none_pulls(self, tmp_path):
         assert pull_elements[0].get("pr") == "1111"
 
 
+class TestSummaryConfidenceColumn:
+    def test_summary_table_shows_confidence_label(self, capsys):
+        regression_data = [{
+            "test_name": "test-workload",
+            "bad_ver": "4.20",
+            "prev_ver": "4.19",
+            "build_url": None,
+            "metrics_with_change": [{
+                "name": "cpu",
+                "value": 30.0,
+                "percentage_change": 100.0,
+                "labels": ["infra"],
+                "confidence": {
+                    "p_value": 0.01,
+                    "cohens_d": 1.2,
+                    "label": "Likely real [1.20] (large shift [0.00])",
+                    "sufficient_data": True,
+                },
+            }],
+            "prs": [],
+            "github_context": None,
+        }]
+        print_regression_summary(regression_data)
+        out = capsys.readouterr().out
+        assert "Confidence" in out
+        assert "Likely real [1.20]" in out
+
+    def test_summary_table_no_confidence_shows_empty(self, capsys):
+        regression_data = [{
+            "test_name": "test-workload",
+            "bad_ver": "4.20",
+            "prev_ver": "4.19",
+            "build_url": None,
+            "metrics_with_change": [{
+                "name": "cpu",
+                "value": 30.0,
+                "percentage_change": 100.0,
+                "labels": ["infra"],
+            }],
+            "prs": [],
+            "github_context": None,
+        }]
+        print_regression_summary(regression_data)
+        out = capsys.readouterr().out
+        assert "Confidence" in out
+        assert "cpu" in out
+
+    def test_summary_table_insufficient_data(self, capsys):
+        regression_data = [{
+            "test_name": "test-workload",
+            "bad_ver": "4.20",
+            "prev_ver": "4.19",
+            "build_url": None,
+            "metrics_with_change": [{
+                "name": "cpu",
+                "value": 30.0,
+                "percentage_change": 100.0,
+                "labels": [],
+                "confidence": {
+                    "p_value": None,
+                    "cohens_d": None,
+                    "label": "Insufficient data",
+                    "sufficient_data": False,
+                },
+            }],
+            "prs": [],
+            "github_context": None,
+        }]
+        print_regression_summary(regression_data)
+        out = capsys.readouterr().out
+        assert "Insufficient data" in out
+
+
+class TestJsonConfidence:
+    def test_changepoint_has_confidence_object(self):
+        data = _make_analysis_result()
+        data.confidence_by_metric = {
+            "cpu": [ConfidenceResult(
+                p_value=0.003, cohens_d=1.2,
+                confidence_label="Likely real [1.20] (large shift [0.00])",
+                sufficient_data=True,
+                sample_size_before=10, sample_size_after=5,
+                mean_before=100.0, mean_after=200.0,
+                std_before=10.0, std_after=12.0,
+                ci_95=(80.0, 120.0),
+            )],
+        }
+        formatter = JsonFormatter()
+        result = formatter.format(data)
+        parsed = json.loads(result["test-workload"])
+        cp_record = [r for r in parsed if r["is_changepoint"]][0]
+        conf = cp_record["metrics"]["cpu"]["confidence"]
+        assert conf["p_value"] == pytest.approx(0.003)
+        assert conf["cohens_d"] == pytest.approx(1.2)
+        assert conf["label"] == "Likely real [1.20] (large shift [0.00])"
+        assert conf["sufficient_data"] is True
+        assert conf["sample_size_before"] == 10
+        assert conf["sample_size_after"] == 5
+        assert conf["mean_before"] == pytest.approx(100.0)
+        assert conf["mean_after"] == pytest.approx(200.0)
+        assert conf["std_before"] == pytest.approx(10.0)
+        assert conf["std_after"] == pytest.approx(12.0)
+        assert conf["ci_95"] == [pytest.approx(80.0), pytest.approx(120.0)]
+
+    def test_non_changepoint_has_no_confidence(self):
+        data = _make_analysis_result()
+        data.confidence_by_metric = {
+            "cpu": [ConfidenceResult(
+                p_value=0.003, cohens_d=1.2,
+                confidence_label="Likely real [1.20] (large shift [0.00])",
+                sufficient_data=True,
+                sample_size_before=10, sample_size_after=5,
+            )],
+        }
+        formatter = JsonFormatter()
+        result = formatter.format(data)
+        parsed = json.loads(result["test-workload"])
+        non_cp = [r for r in parsed if not r["is_changepoint"]]
+        for record in non_cp:
+            assert "confidence" not in record["metrics"]["cpu"]
+
+    def test_no_confidence_data_no_key(self):
+        data = _make_analysis_result()
+        # No confidence_by_metric set (default {})
+        formatter = JsonFormatter()
+        result = formatter.format(data)
+        parsed = json.loads(result["test-workload"])
+        cp_record = [r for r in parsed if r["is_changepoint"]][0]
+        assert "confidence" not in cp_record["metrics"]["cpu"]
+
+    def test_insufficient_data_in_json(self):
+        data = _make_analysis_result()
+        data.confidence_by_metric = {
+            "cpu": [ConfidenceResult(
+                p_value=None, cohens_d=None,
+                confidence_label="Insufficient data",
+                sufficient_data=False,
+                sample_size_before=5, sample_size_after=1,
+            )],
+        }
+        formatter = JsonFormatter()
+        result = formatter.format(data)
+        parsed = json.loads(result["test-workload"])
+        cp_record = [r for r in parsed if r["is_changepoint"]][0]
+        conf = cp_record["metrics"]["cpu"]["confidence"]
+        assert conf["p_value"] is None
+        assert conf["cohens_d"] is None
+        assert conf["sufficient_data"] is False
+
+
+class TestRegressionDataConfidence:
+    def test_regression_data_includes_confidence(self):
+        data = _make_analysis_result()
+        data.confidence_by_metric = {
+            "cpu": [ConfidenceResult(
+                p_value=0.003, cohens_d=1.2,
+                confidence_label="Likely real [1.20] (large shift [0.00])",
+                sufficient_data=True,
+                sample_size_before=10, sample_size_after=5,
+            )],
+        }
+
+        class ConcreteFormatter(BaseFormatter):
+            def format(self, data):
+                return {}
+            def format_average(self, data):
+                return ""
+            def save(self, test_name, formatted, save_output_path):
+                pass
+            def print_output(self, test_name, formatted, data,
+                             pr=0, is_pull=False):
+                pass
+            def print_and_save_pr(self, periodic, pulls, save_output_path):
+                pass
+
+        formatter = ConcreteFormatter()
+        regressions = formatter.extract_regression_data(data)
+        metric_entry = regressions[0]["metrics_with_change"][0]
+        assert "confidence" in metric_entry
+        assert metric_entry["confidence"]["p_value"] == pytest.approx(0.003)
+        assert metric_entry["confidence"]["label"] == "Likely real [1.20] (large shift [0.00])"
+        assert metric_entry["confidence"]["sample_size_before"] == 10
+        assert metric_entry["confidence"]["sample_size_after"] == 5
+
+    def test_same_index_different_metrics_get_own_confidence(self):
+        np.random.seed(42) # pylint: disable=duplicate-code
+        df = pd.DataFrame({
+            "uuid": [f"uuid-{i}" for i in range(10)],
+            "ocpVersion": [f"4.{18+i//5}" for i in range(10)],
+            "timestamp": [1700000000 + i * 100000 for i in range(10)],
+            "buildUrl": [f"http://b{i}" for i in range(10)],
+            "prs": [None] * 10,
+            "ovsCPU": np.concatenate([
+                np.random.normal(0.12, 0.01, 7),
+                np.random.normal(0.155, 0.01, 3),
+            ]),
+            "podLatency": np.concatenate([
+                np.random.normal(35000, 5000, 7),
+                np.random.normal(57000, 5000, 3),
+            ]),
+        })
+        series = Series(
+            test_name="test",
+            branch=None,
+            time=list(df["timestamp"]),
+            metrics={
+                "ovsCPU": Metric(1, 1.0),
+                "podLatency": Metric(1, 1.0),
+            },
+            data={
+                "ovsCPU": df["ovsCPU"],
+                "podLatency": df["podLatency"],
+            },
+            attributes={
+                "uuid": df["uuid"],
+                "ocpVersion": df["ocpVersion"],
+            },
+        )
+        cps_by_metric = {
+            "ovsCPU": [make_change_point("ovsCPU", 7,
+                                         mean_1=0.12, mean_2=0.155)],
+            "podLatency": [make_change_point("podLatency", 7,
+                                             mean_1=35000, mean_2=57000)],
+        }
+        confidence = compute_confidence(
+            cnsts.EDIVISIVE, df, cps_by_metric
+        )
+        data = AnalysisResult(
+            test_name="test-workload",
+            test={"name": "test-workload", "uuid_field": "uuid",
+                  "version_field": "ocpVersion",
+                  "metadata": {"benchmark.keyword": "test"}},
+            dataframe=df,
+            metrics_config={
+                "ovsCPU": {"direction": 1, "labels": [],
+                           "threshold": 0, "correlation": "",
+                           "context": None},
+                "podLatency": {"direction": 1, "labels": [],
+                               "threshold": 0, "correlation": "",
+                               "context": None},
+            },
+            change_points_by_metric=cps_by_metric,
+            series=series,
+            regression_flag=True,
+            avg_values=pd.Series({"ovsCPU": 0.12, "podLatency": 35000}),
+            collapse=False,
+            display_fields=[],
+            column_group_size=5,
+            uuid_field="uuid",
+            version_field="ocpVersion",
+            sippy_pr_search=False,
+            github_repos=[],
+            confidence_by_metric=confidence,
+        )
+
+        class ConcreteFormatter(BaseFormatter):
+            def format(self, data):
+                return {}
+            def format_average(self, data):
+                return ""
+            def save(self, test_name, formatted, save_output_path):
+                pass
+            def print_output(self, test_name, formatted, data,
+                             pr=0, is_pull=False):
+                pass
+            def print_and_save_pr(self, periodic, pulls, save_output_path):
+                pass
+
+        formatter = ConcreteFormatter()
+        regressions = formatter.extract_regression_data(data)
+        assert len(regressions) == 1
+        metrics = regressions[0]["metrics_with_change"]
+        assert len(metrics) == 2
+        cpu_conf = metrics[0]["confidence"]
+        lat_conf = metrics[1]["confidence"]
+        assert cpu_conf["cohens_d"] != lat_conf["cohens_d"]
+        assert cpu_conf["p_value"] != lat_conf["p_value"]
+
+    def test_regression_data_no_confidence_when_empty(self):
+        data = _make_analysis_result()
+        # default confidence_by_metric = {}
+
+        class ConcreteFormatter(BaseFormatter):
+            def format(self, data):
+                return {}
+            def format_average(self, data):
+                return ""
+            def save(self, test_name, formatted, save_output_path):
+                pass
+            def print_output(self, test_name, formatted, data,
+                             pr=0, is_pull=False):
+                pass
+            def print_and_save_pr(self, periodic, pulls, save_output_path):
+                pass
+
+        formatter = ConcreteFormatter()
+        regressions = formatter.extract_regression_data(data)
+        metric_entry = regressions[0]["metrics_with_change"][0]
+        assert "confidence" not in metric_entry
+
+
 class TestFormatterFactory:
     def test_get_json_formatter(self):
         formatter = FormatterFactory.get_formatter("json")
diff --git a/orion/tests/test_matcher_batch.py b/orion/tests/test_matcher_batch.py
index e2494e73..d241ea74 100644
--- a/orion/tests/test_matcher_batch.py
+++ b/orion/tests/test_matcher_batch.py
@@ -644,7 +644,7 @@ def test_nested_field_not_filter_excludes(self, matcher_instance, monkeypatch):
         assert result["podReadyLatency"][0]["P99"] == 4500
 
 
-class TestGetNested:
+class TestGetNested: # pylint: disable=protected-access
     """Unit tests for Matcher._get_nested."""
 
     @pytest.mark.parametrize("doc,key,expected", [
diff --git a/orion/utils.py b/orion/utils.py
index b3ded061..aa25b04a 100644
--- a/orion/utils.py
+++ b/orion/utils.py
@@ -847,9 +847,11 @@ def create_record(record):
                 "%Y-%m-%dT%H:%M:%SZ"
             ),
             metric_name: record["metrics"][metric_name]["value"],
-            "is_changepoint": bool(record["metrics"][metric_name]["percentage_change"]),
-            "percentage_change": record["metrics"][metric_name]["percentage_change"],
+            "is_changepoint": record["metrics"][metric_name].get("is_changepoint", False),
+            "percentage_change": f"{record['metrics'][metric_name]['percentage_change']:.2f}%",
         }
+        conf = record["metrics"][metric_name].get("confidence")
+        base_record["Confidence"] = conf.get("label", "") if conf else ""
         # Add metadata field if it exists in the record
         for display_field in display_fields:
             if display_field and display_field in record:
@@ -865,7 +867,7 @@ def create_record(record):
     if lines:
         highlighted_lines += lines[0:3]
     for i, line in enumerate(lines[3:-1]):
-        if df["percentage_change"][
+        if df["is_changepoint"][
             i
         ]:  # Offset by 3 to account for header and separator
             highlighted_line = f"{lines[i+3]} -- changepoint"
diff --git a/pyproject.toml b/pyproject.toml
index 0f53c4d8..3728601b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -13,6 +13,7 @@ dependencies = [
     "PyYAML==6.0.1",
     "pyshorteners==1.0.1",
     "numpy>=2.2.0,<2.4",
+    "scipy>=1.14.0,<2.0",
     "scikit-learn==1.5.0",
     "pandas==2.3.3",
     "tabulate==0.9.0",
diff --git a/requirements.txt b/requirements.txt
index 698e7597..80d9de5e 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -8,6 +8,7 @@ opensearch-py==3.0.0
 Jinja2==3.1.3
 PyYAML==6.0.1
 numpy>=2.2.0,<2.4
+scipy>=1.14.0,<2.0
 scikit-learn==1.5.0
 pandas==2.3.3
 tabulate==0.9.0
diff --git a/setup.py b/setup.py
index 5eedb5bf..7cd01f4e 100644
--- a/setup.py
+++ b/setup.py
@@ -24,6 +24,7 @@
         'Jinja2==3.1.3',
         'PyYAML==6.0.1',
         "numpy==2.2.0; python_version=='3.14'",
+        'scipy>=1.14.0,<2.0',
         'scikit-learn==1.5.0',
         "pandas==2.3.3; python_version=='3.14'",
         'tabulate==0.9.0',
diff --git a/test-local.bats b/test-local.bats
index e2d8c8cc..d2dd25b5 100755
--- a/test-local.bats
+++ b/test-local.bats
@@ -138,9 +138,14 @@ setup() {
     exit 1
   fi
 
-  changepoint=$(echo '404.549 | https://prow.ci/2013174937652563968 | -- changepoint')
-  if ! grep -qF -- "$changepoint" ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$changepoint' not found in results_olm-integration-test.xml"
+  if ! grep -qF "404.55%" ./outputs/results_olm-integration-test.xml; then
+    echo "Expected percentage '404.55%' not found in results_olm-integration-test.xml"
+    cat ./outputs/results_olm-integration-test.xml
+    exit 1
+  fi
+
+  if ! grep -qF "Large shift" ./outputs/results_olm-integration-test.xml; then
+    echo "Expected confidence label 'Large shift' not found in results_olm-integration-test.xml"
     cat ./outputs/results_olm-integration-test.xml
     exit 1
   fi
@@ -235,9 +240,14 @@ setup() {
     exit 1
   fi
 
-  changepoint=$(echo '404.549')
-  if ! grep -qF -- "$changepoint" ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$changepoint' not found in results_olm-integration-test.xml"
+  if ! grep -qF "404.55%" ./outputs/results_olm-integration-test.xml; then
+    echo "Expected percentage '404.55%' not found in results_olm-integration-test.xml"
+    cat ./outputs/results_olm-integration-test.xml
+    exit 1
+  fi
+
+  if ! grep -qF "Large shift" ./outputs/results_olm-integration-test.xml; then
+    echo "Expected confidence label 'Large shift' not found in results_olm-integration-test.xml"
     cat ./outputs/results_olm-integration-test.xml
     exit 1
   fi
@@ -361,7 +371,7 @@ setup() {
     exit 1
   fi
 
-  CHANGEPOINTS=$(grep -c '"is_changepoint": true' ./outputs/results-anomaly_olm-integration-test.json)
+  CHANGEPOINTS=$(grep -c '"is_changepoint": true$' ./outputs/results-anomaly_olm-integration-test.json)
   if [ "$CHANGEPOINTS" -ne 3 ]; then
     echo "Expected 3 changepoints, found $CHANGEPOINTS in ./outputs/results-anomaly_olm-integration-test.json"
     exit 1
@@ -380,18 +390,23 @@ setup() {
     exit 1
   fi
 
-  if ! grep -q "155.648" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '155.648' not found in ./outputs/results-anomaly_olm-integration-test.xml"
+  if ! grep -qF "155.65%" ./outputs/results-anomaly_olm-integration-test.xml; then
+    echo "Expected percentage '155.65%' not found in ./outputs/results-anomaly_olm-integration-test.xml"
+    exit 1
+  fi
+
+  if ! grep -qF "56.72%" ./outputs/results-anomaly_olm-integration-test.xml; then
+    echo "Expected percentage '56.72%' not found in ./outputs/results-anomaly_olm-integration-test.xml"
     exit 1
   fi
 
-  if ! grep -q "56.7208" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '56.7208' not found in ./outputs/results-anomaly_olm-integration-test.xml"
+  if ! grep -qF "38.89%" ./outputs/results-anomaly_olm-integration-test.xml; then
+    echo "Expected percentage '38.89%' not found in ./outputs/results-anomaly_olm-integration-test.xml"
     exit 1
   fi
 
-  if ! grep -q "38.8858" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '38.8858' not found in ./outputs/results-anomaly_olm-integration-test.xml"
+  if ! grep -qF "Anomaly detection" ./outputs/results-anomaly_olm-integration-test.xml; then
+    echo "Expected confidence label 'Anomaly detection' not found in ./outputs/results-anomaly_olm-integration-test.xml"
     exit 1
   fi
 
@@ -449,8 +464,13 @@ setup() {
     exit 1
   fi
 
-  if ! grep -q "True             |             160.879" ./outputs/results-cmr_olm-integration-test.xml; then
-    echo "Expected string 'True             |             160.879' not found in results-cmr_olm-integration-test.xml"
+  if ! grep -qF "160.88%" ./outputs/results-cmr_olm-integration-test.xml; then
+    echo "Expected percentage '160.88%' not found in results-cmr_olm-integration-test.xml"
+    exit 1
+  fi
+
+  if ! grep -qF "Insufficient data" ./outputs/results-cmr_olm-integration-test.xml; then
+    echo "Expected confidence label 'Insufficient data' not found in results-cmr_olm-integration-test.xml"
     exit 1
   fi
 
diff --git a/test.bats b/test.bats
index e05f1322..f4e8f0f3 100644
--- a/test.bats
+++ b/test.bats
@@ -274,20 +274,6 @@ setup() {
   VERSION=$before_version
 }
 
-@test "orion auto-loads ack/all_ack.yaml when present" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' 2>&1 | tee ./outputs/results-ack-auto.txt
-  EXIT_CODE=$?
-  set -e
-  if [ ! -f ack/all_ack.yaml ]; then
-    skip "ack/all_ack.yaml not present, skipping auto-load test"
-  fi
-  if ! grep -q "all_ack.yaml" ./outputs/results-ack-auto.txt; then
-    echo "Expected orion to mention all_ack.yaml when auto-loading ACK"
-    exit 1
-  fi
-}
-
 @test "orion with quay config " {
   export quay_image_push_pull_index="quay-push-pull*"
   export quay_load_test_index="quay-vegeta-results*"
@@ -301,606 +287,3 @@ setup() {
   run_cmd orion --node-count false --config "examples/quay-load-test-stable-stage.yaml" --hunter-analyze --es-server=${QUAY_QE_ES_SERVER} --output-format junit --save-output-path=./outputs/junit.xml --collapse --input-vars='{"quay_version": "quayio-stage", "ocp_version": "4.18"}'
 }
 
-@test "orion version check" {
-  set +e
-  version=$(orion --version)
-  echo $version
-  expected_tag=$(git tag -l | sort -V | tail -1)
-  expected_tag=${expected_tag#v}
-  if [[ -z $expected_tag ]]; then
-    expected_tag=0.0
-  fi
-
-  expected_version="orion ${expected_tag}"
-
-  last_commit=$(git rev-parse --short=7 HEAD)
-  describe=$(git describe --tags --dirty --always)
-
-  if [[ "$describe" == *"$last_commit"* ]]; then
-    echo "Is ahead of Tag adding '.post1.dev'"
-    expected_version+=".post1.dev"
-  fi
-
-  if [[ "$describe" == *"dirty"* ]]; then
-    if [[ ! "$version" == *"+dirty"* ]]; then
-      echo "Failed checking for dirty append"
-      exit 1
-    fi
-  fi
-
-  echo $expected_version
-
-  if [[ ! "$version" == *"$expected_version"* ]]; then
-    exit 1
-  fi
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage string exists in the output file
-  if ! grep -q "+404.5%" ./outputs/results.txt; then
-    echo "Expected string '+404.5%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Previous Version string exists in the output file
-  if ! grep -q "Previous version:    4.20.0-0.nightly-2026-01-14-195655" ./outputs/results.txt; then
-    echo "Expected string 'Previous version:    4.20.0-0.nightly-2026-01-14-195655' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20.0-0.nightly-2026-01-15-195655" ./outputs/results.txt; then
-    echo "Expected string 'Changepoint at:      4.20.0-0.nightly-2026-01-15-195655' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint json" {
-  set +e
-
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format json --save-output-path=./outputs/results.json
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  bad_version=$(jq -r '.[] | select(.is_changepoint == true) | .ocpVersion' ./outputs/results_olm-integration-test.json)
-
-  if [ "$bad_version" != "4.20.0-0.nightly-2026-01-15-195655" ]; then
-    echo "Version did not match. Expected '4.20.0-0.nightly-2026-01-15-195655', got '$bad_version'"
-    exit 1
-  fi
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint junit" {
-  set +e
-
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format junit --save-output-path=./outputs/results.xml
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  failure=$(echo 'failures="1"')
-  # Check if the failures string exists in the output file
-  if ! grep -q $failure ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$failure' not found in results_olm-integration-test.xml"
-    cat ./outputs/results_olm-integration-test.xml
-    exit 1
-  fi
-
-  changepoint=$(echo '404.549 | https://prow.ci/2013174937652563968 | -- changepoint')
-  # Check if the changepoint string exists in the output file
-  if ! grep -q $changepoint ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$changepoint' not found in results_olm-integration-test.xml"
-    cat ./outputs/results_olm-integration-test.xml
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint with custom display" {
-  set +e
-
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --display upstreamJob > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage string exists in the output file
-  if ! grep -q "+404.5%" ./outputs/results.txt; then
-    echo "Expected string '+404.5%' not found in results.txt"
-    cat ./outputs/results.txt
-    exit 1
-  fi
-
-  # Check if the Previous Version string exists in the output file
-  if ! grep -q "Previous version:    4.20.0-0.nightly-2026-01-14-195655" ./outputs/results.txt; then
-    echo "Expected string 'Previous version:    4.20.0-0.nightly-2026-01-14-195655' not found in results.txt"
-    cat ./outputs/results.txt
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20.0-0.nightly-2026-01-15-195655" ./outputs/results.txt; then
-    echo "Expected string 'Changepoint at:      4.20.0-0.nightly-2026-01-15-195655' not found in results.txt"
-    cat ./outputs/results.txt
-    exit 1
-  fi
-
-  customDisplay="upstreamJob"
-  # Check if the customDisplay string exists in the output file
-  if ! grep -q $customDisplay ./outputs/results.txt; then
-    echo "Expected string '$customDisplay' not found in results.txt"
-    cat ./outputs/results.txt
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint json with custom display" {
-  set +e
-
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format json --display upstreamJob --save-output-path=./outputs/results.json
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  bad_version=$(jq -r '.[] | select(.is_changepoint == true) | .ocpVersion' ./outputs/results_olm-integration-test.json)
-  if [ "$bad_version" != "4.20.0-0.nightly-2026-01-15-195655" ]; then
-    echo "Version did not match. Expected '4.20.0-0.nightly-2026-01-15-195655', got '$bad_version'"
-    exit 1
-  fi
-
-  upstreamJob=$(jq -r '.[] | select(.is_changepoint == true) | .upstreamJob' ./outputs/results_olm-integration-test.json)
-  if [ "$upstreamJob" != "periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-gcp-4.20-nightly-x86-olmv1-benchmark-test" ]; then
-    echo "upstreamJob did not match. Expected 'periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-gcp-4.20-nightly-x86-olmv1-benchmark-test', got '$upstreamJob'"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint junit with custom display" {
-  set +e
-
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format junit --display upstreamJob --save-output-path=./outputs/results.xml
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  failure=$(echo 'failures="1"')
-  # Check if the failures string exists in the output file
-  if ! grep -q $failure ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$failure' not found in results_olm-integration-test.xml"
-    cat ./outputs/results_olm-integration-test.xml
-    exit 1
-  fi
-
-  changepoint=$(echo '404.549')
-  # Check if the changepoint string exists in the output file
-  if ! grep -q $changepoint ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$changepoint' not found in results_olm-integration-test.xml"
-    cat ./outputs/results_olm-integration-test.xml
-    exit 1
-  fi
-
-  customDisplay="upstreamJob"
-  # Check if the customDisplay string exists in the output file
-  if ! grep -q $customDisplay ./outputs/results_olm-integration-test.xml; then
-    echo "Expected string '$customDisplay' not found in results_olm-integration-test.xml"
-    cat ./outputs/results_olm-integration-test.xml
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion with regression should contain inline changepoint no metadata index" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-metrics-only.yaml --metadata-index "orion-integration-test-metrics*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage string exists in the output file
-  if ! grep -q "+404.5%" ./outputs/results.txt; then
-    echo "Expected string '+404.5%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Previous Version string exists in the output file
-  if ! grep -q "Previous version:    4.20" ./outputs/results.txt; then
-    echo "Expected string 'Previous version:    4.20' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20" ./outputs/results.txt; then
-    echo "Expected string 'Changepoint at:      4.20' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion early-changepoint metric - changepoint in first 5 is skipped when expansion finds no extra data" {
-  # Early-cp metric has changepoint at 5th point. Orion expands the window to re-validate;
-  # with only 10 runs there is no additional data, so it skips the early changepoint and does not report regression.
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-early-cp.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results-early-cp.txt
-  EXIT_CODE=$?
-
-  if [ $EXIT_CODE -eq 2 ]; then
-    echo "Regression was reported but should be skipped (no extra data after expansion)"
-    cat ./outputs/results-early-cp.txt
-    exit 1
-  fi
-
-  # Output should not show Bad/Previous version (early changepoint was cleared)
-  if grep -q "Bad Version:         4.20.0-0.nightly-2026-01-14-195655" ./outputs/results-early-cp.txt; then
-    echo "Expected early changepoint to be skipped (no Bad Version in output)"
-    cat ./outputs/results-early-cp.txt
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion --anomaly-detection with regression should contain inline changepoint" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --anomaly-detection --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results-anomaly.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage #1 string exists in the output file
-  if ! grep -q "+155.6%" ./outputs/results-anomaly.txt; then
-    echo "Expected string '+155.6%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the percentage #2 string exists in the output file
-  if ! grep -q "+56.7%" ./outputs/results-anomaly.txt; then
-    echo "Expected string '+56.7%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the percentage #3 string exists in the output file
-  if ! grep -q "+38.9%" ./outputs/results-anomaly.txt; then
-    echo "Expected string '+38.9%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20.0-0.nightly-2026-01-15-195655" ./outputs/results-anomaly.txt; then
-    echo "Expected string 'Changepoint at:      4.20.0-0.nightly-2026-01-15-195655' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20.0-0.nightly-2026-01-17-195655" ./outputs/results-anomaly.txt; then
-    echo "Expected string 'Changepoint at:      4.20.0-0.nightly-2026-01-17-195655' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-
-@test "orion --anomaly-detection with regression should contain inline changepoint json" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --anomaly-detection --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format json --save-output-path=./outputs/results-anomaly.json
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if changepoints string exists in the output file
-  CHANGEPOINTS=$(grep -c '"is_changepoint": true' ./outputs/results-anomaly_olm-integration-test.json)
-  if [ "$CHANGEPOINTS" -ne 3 ]; then
-    echo "Expected 3 changepoints, found $CHANGEPOINTS in ./outputs/results-anomaly_olm-integration-test.json"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion --anomaly-detection with regression should contain inline changepoint junit" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --anomaly-detection --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format junit --save-output-path=./outputs/results-anomaly.xml
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage #1 string exists in the output file
-  if ! grep -q "155.648" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '155.648' not found in ./outputs/results-anomaly_olm-integration-test.xml"
-    exit 1
-  fi
-
-  # Check if the percentage #2 string exists in the output file
-  if ! grep -q "56.7208" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '56.7208' not found in ./outputs/results-anomaly_olm-integration-test.xml"
-    exit 1
-  fi
-
-  # Check if the percentage #3 string exists in the output file
-  if ! grep -q "38.8858" ./outputs/results-anomaly_olm-integration-test.xml; then
-    echo "Expected string '38.8858' not found in ./outputs/results-anomaly_olm-integration-test.xml"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion --cmr with regression should contain inline changepoint" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --cmr --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results-cmr.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage #1 string exists in the output file
-  if ! grep -q "+160.9%" ./outputs/results-cmr.txt; then
-    echo "Expected string '+160.9%' not found in results-cmr.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion --cmr with regression should contain inline changepoint json" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --cmr --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format json --save-output-path=./outputs/results-cmr.json
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  bad_version=$(jq -r '.[] | select(.is_changepoint == true) | .ocpVersion' ./outputs/results-cmr_olm-integration-test.json)
-  if [ "$bad_version" != "4.20.0-0.nightly-2026-01-18-195655" ]; then
-    echo "Version did not match. Expected '4.20.0-0.nightly-2026-01-18-195655', got '$bad_version'"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion --cmr with regression should contain inline changepoint junit" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --cmr --config hack/ci-tests/configurations/ci-tests.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' --output-format junit --save-output-path=./outputs/results-cmr.xml
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check if the percentage string exists in the output file
-  if ! grep -q "True             |             160.879" ./outputs/results-cmr_olm-integration-test.xml; then
-    echo "Expected string 'True             |             160.879' not found in results-cmr_olm-integration-test.xml"
-    exit 1
-  fi
-
-  set -e
-}
-
-
-@test "orion inheriting config" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-inherits.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check for gloabl metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_sum_sum" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_sum_sum' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the percentage string exists in the output file
-  if ! grep -q "+404.5%" ./outputs/results.txt; then
-    echo "Expected string '+404.5%' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Previous version string exists in the output file
-  if ! grep -q "Previous version:    4.20.0-0.nightly-2026-01-14-195655" ./outputs/results.txt; then
-    echo "Expected string 'Previous version:    4.20.0-0.nightly-2026-01-14-195655' not found in results.txt"
-    exit 1
-  fi
-
-  # Check if the Changepoint at string exists in the output file
-  if ! grep -q "Changepoint at:      4.20.0-0.nightly-2026-01-15-195655" ./outputs/results.txt; then
-    echo "Expected string 'Changepoint at:      4.20.0-0.nightly-2026-01-15-195655' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion inheriting config ignore global" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-inherits-ignore-global.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check for global metric
-  if grep -q "OCP-84094_catalogdCPU_GCP_sum_sum" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_sum_sum' found in results.txt, should not be present"
-    exit 1
-  fi
-
-  # Check for metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_avg" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_avg' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion inheriting config local metadata" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-inherits-local-metadata.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 3 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-}
-
-@test "orion inheriting config local metrics with global ignore" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-inherits-local-metrics-with-ignore.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check for local metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_max_max" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_max_max' not found in results.txt"
-    exit 1
-  fi
-
-  # Check for global metric
-  if grep -q "OCP-84094_catalogdCPU_GCP_sum_sum" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_sum_sum' found in results.txt, should not be present"
-    exit 1
-  fi
-
-  # Check for metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_avg" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_avg' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion inheriting config local metrics" {
-  set +e
-  orion --lookback 15d --since 2026-01-20 --hunter-analyze --config hack/ci-tests/configurations/ci-tests-inherits-local-metrics.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --es-server=${QE_ES_SERVER} --node-count true --input-vars='{"version": "4.20"}' > ./outputs/results.txt
-  EXIT_CODE=$?
-
-  if [ ! $EXIT_CODE -eq 2 ]; then
-    echo "no regression found"
-    exit 1
-  fi
-
-  # Check for local metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_max_max" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_max_max' not found in results.txt"
-    exit 1
-  fi
-
-  # Check for global metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_sum_sum" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_sum_sum' not found in results.txt"
-    exit 1
-  fi
-
-  # Check for metric
-  if ! grep -q "OCP-84094_catalogdCPU_GCP_avg" ./outputs/results.txt; then
-    echo "Expected string 'OCP-84094_catalogdCPU_GCP_avg' not found in results.txt"
-    exit 1
-  fi
-
-  set -e
-}
-
-@test "orion browbeat config should contain keystone metrics text" {
-  set +e
-  orion --lookback 15d --hunter-analyze --config hack/ci-tests/configurations/ci-tests-browbeat.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --since 2026-02-23 --display='' --input-vars='{"version": "4.18"}' --es-server=${QE_ES_SERVER} > ./outputs/results-browbeat.txt
-  set -e
-
-  for metric in keystone_v3_list_users_avg_avg keystone_v3_list_users_count_count keystone_v3_list_users_P99_percentiles keystone_v3_list_users_P95_percentiles keystone_v3_list_users_P90_percentiles keystone_v3_list_users_max_max keystone_v3_list_users_min_min keystone_v3_list_users_sum_sum; do
-    if ! grep -q "$metric" ./outputs/results-browbeat.txt; then
-      echo "Expected metric '$metric' not found in results-browbeat.txt"
-      exit 1
-    fi
-  done
-}
-
-@test "orion browbeat config should contain keystone metrics json" {
-  set +e
-  orion --lookback 15d --hunter-analyze --config hack/ci-tests/configurations/ci-tests-browbeat.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --since 2026-02-23 --display='' --input-vars='{"version": "4.18"}' --es-server=${QE_ES_SERVER} --output-format json > ./outputs/results-browbeat.json
-  set -e
-
-  for metric in keystone_v3_list_users_avg_avg keystone_v3_list_users_count_count keystone_v3_list_users_P99_percentiles keystone_v3_list_users_P95_percentiles keystone_v3_list_users_P90_percentiles keystone_v3_list_users_max_max keystone_v3_list_users_min_min keystone_v3_list_users_sum_sum; do
-    if ! grep -q "$metric" ./outputs/results-browbeat.json; then
-      echo "Expected metric '$metric' not found in results-browbeat.json"
-      exit 1
-    fi
-  done
-}
-
-@test "orion browbeat config should contain keystone metrics junit" {
-  set +e
-  orion --lookback 15d --hunter-analyze --config hack/ci-tests/configurations/ci-tests-browbeat.yaml --metadata-index "orion-integration-test-data*" --benchmark-index "orion-integration-test-metrics*" --since 2026-02-23 --display='' --input-vars='{"version": "4.18"}' --es-server=${QE_ES_SERVER} --output-format junit > ./outputs/results-browbeat.xml
-  set -e
-
-  for metric in keystone_v3_list_users_avg_avg keystone_v3_list_users_count_count keystone_v3_list_users_P99_percentiles keystone_v3_list_users_P95_percentiles keystone_v3_list_users_P90_percentiles keystone_v3_list_users_max_max keystone_v3_list_users_min_min keystone_v3_list_users_sum_sum; do
-    if ! grep -q "$metric" ./outputs/results-browbeat.xml; then
-      echo "Expected metric '$metric' not found in results-browbeat.xml"
-      exit 1
-    fi
-  done
-}