Skip to content

Add backfill script for historical time-series data in scripts/ #49

Description

@christabel888

Context: The new time-series tables (#45-#47) start empty -- dashboards will look broken/sparse until historical data is backfilled from existing ingested ledger/event data.

Where:

  • scripts/ -- new backfill script, following the existing pattern in src/jobs/backfill.rs and scripts/

What to do:

Suggested approach:

  1. Follow src/jobs/backfill.rs's existing pattern for a one-off/re-runnable backfill script.
  2. Backfill Design and migrate time-series schema for daily network metrics #45-Design and migrate time-series schema for wallet balance history #47's tables from whatever historical ledger/contract-event data already exists -- this is bounded by how far back ingestion already goes, it can't manufacture history that was never ingested.
  3. Make it idempotent/resumable (safe to re-run, safe to interrupt) since backfilling real historical volume can take a while.

Watch out for:

  • Running this against production data for the first time is a good candidate for a dry-run mode first (compute and log, don't write) before actually writing rows.

Definition of done:

  • Running the script populates historical data for the new dashboards without requiring a wait period

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions