Skip to content

feat(data): org-level retention enforcement job with TTL, audit + dry-run (#255)#307

Merged
Baskarayelu merged 2 commits intoCredenceOrg:mainfrom
Itodo-S:feature/data-retention-fresh
Apr 26, 2026
Merged

feat(data): org-level retention enforcement job with TTL, audit + dry-run (#255)#307
Baskarayelu merged 2 commits intoCredenceOrg:mainfrom
Itodo-S:feature/data-retention-fresh

Conversation

@Itodo-S
Copy link
Copy Markdown
Contributor

@Itodo-S Itodo-S commented Apr 25, 2026

Summary

Closes #255 — implements a configurable org-level data retention enforcement job that prunes expired records with full audit output and a safe dry-run mode.

New files

File Purpose
src/config/retention.ts RetentionConfig type + loadRetentionConfig() that reads per-entity TTLs and the dry-run flag from environment variables with sensible defaults
src/repositories/retentionRepository.ts RetentionRepository — count-then-delete helpers for score_history, audit_logs, slash_events, event_outbox; all mutations are no-ops when dryRun=true
src/jobs/dataRetentionJob.ts DataRetentionJob — orchestrates all entities in parallel, respects batchLimit to avoid table-lock pressure, returns a structured DataRetentionResult per run
src/jobs/dataRetentionJob.test.ts 10 vitest unit tests covering: zero-expiry, batch deletes, TTL=0 skip, dry-run (no DELETE issued), mixed TTL config, batchLimit propagation, logging

Configuration (env vars)

Variable Default Description
RETENTION_DRY_RUN false Log-only mode — no rows deleted
RETENTION_BATCH_LIMIT 5000 Max rows pruned per entity per run
RETENTION_TTL_SCORE_HISTORY_DAYS 90
RETENTION_TTL_AUDIT_LOGS_DAYS 365
RETENTION_TTL_SLASH_EVENTS_DAYS 0 Keep forever by default
RETENTION_TTL_OUTBOX_EVENTS_DAYS 30

Design notes

  • Safe and reversible: dry-run mode is the recommended first deployment step; flip RETENTION_DRY_RUN=false only after reviewing the counts.
  • No schema changes: queries target tables that already exist (score_history, audit_logs, slash_events, event_outbox).
  • Batch CTE deletes (WITH rows AS (SELECT … LIMIT $n) DELETE …) bound the lock duration on each run.

Test plan

  • npx vitest run src/jobs/dataRetentionJob.test.ts — all 10 tests pass
  • Verify dry-run mode: set RETENTION_DRY_RUN=true, run job, confirm zero rows deleted and correct expired counts logged
  • Verify TTL=0 entities are skipped (no SQL issued)
  • Confirm batchLimit is respected in DELETE queries

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented Apr 25, 2026

@Itodo-S Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Baskarayelu Baskarayelu merged commit a1e2a54 into CredenceOrg:main Apr 26, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Fresh 2026-04][Backend] Data: org-level retention enforcement job (TTL + audit + dry-run)

2 participants