Skip to content

feat: unify zmail workflow targets#3

Open
mannie-exe wants to merge 5 commits into
devfrom
feat/unified-workflow-targets
Open

feat: unify zmail workflow targets#3
mannie-exe wants to merge 5 commits into
devfrom
feat/unified-workflow-targets

Conversation

@mannie-exe
Copy link
Copy Markdown
Contributor

Summary

  • add shared mutation envelopes and keyed node refresh across zmail workflows
  • canonicalize finance text imports and add durable ledger overrides
  • add decision decks, DataInspector, and shared UI primitives

Test plan

  • mise run test:unit -- platform
  • mise run test:unit -- domain
  • mise run test:unit -- finance
  • mise run test:unit -- runtime
  • mise run test:integration
  • mise run test:e2e
  • mise run test
  • mise run test:coverage

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 22, 2026

Greptile Summary

This PR unifies zmail workflow targets by adding durable finance ledger overrides, date-precision tracking, a shared client mutation envelope contract, and new UI primitives (DataInspector, decision decks). It also canonicalizes the "statement" source kind to "text" across all tables via migration 009 and wires a finance drilldown feature into loadFinanceData.

  • Finance ledger overrides (lib/finance-ledger-overrides.ts): new apply/revert/reapply pipeline; overrides are stored as JSON patches and replayed into the in-memory entries map during rebuildFinanceKnowledge.
  • Date precision (lib/finance-ledger-dates.ts, migration 011): classifies occurred_at/posted_at/cleared_at precision and recovers a usable beancount date when occurredAt is only month/year-precise by falling back to message.received_at.
  • Shared mutation envelope (lib/client-contract.ts, src/lib/client/mutation.ts): single postMutation helper handles toasts, SvelteKit invalidation keys, and redirects across all form actions.

Confidence Score: 5/5

Safe to merge; the two findings are non-blocking quality notes with no impact on correctness.

The override apply/revert/reapply logic is well-structured, the statement-to-text migration is complete across all affected tables, and the readiness stats remain correctly year-scoped via canonical-key filtering. The unbounded ledger table scan is a future performance concern but does not produce wrong results today.

db/migrations/010_finance_ledger_override_indexes.sql (redundant index) and the rawLedgerRows query in server/actions.ts (unbounded selectAll).

Important Files Changed

Filename Overview
lib/finance-ledger-overrides.ts New file implementing durable ledger override apply/revert. Supersession pattern is clean; the double-metadata-merge in reapplyActiveFinanceLedgerOverrides (noted in prior reviews) is the main concern here.
lib/finance-ledger-dates.ts New date classification and beancount-date resolution utilities; well-structured with regex guards and UTC-safe date validation.
server/actions.ts Adds finance drilldown parsing, rawLedgerRows full-table query, and readiness summary delegation to summarizeExportReadiness; readiness is correctly year-scoped via filteredCanonicalKeys but the selectAll() query is unbounded.
lib/finance-knowledge.ts Adds date precision fields, date recovery for month/year-precision occurredAt, and reapplies active overrides after knowledge rebuild; sourcePriority renamed statement to text is safe because migration 009 backfills existing rows.
db/migrations/009_unified_workflows_text_source_overrides.sql Creates finance_ledger_entry_overrides table, both indexes, and backfills statement to text across all affected tables; migration is self-contained and idempotent.
db/migrations/010_finance_ledger_override_indexes.sql Redundant: creates the same partial index already present in migration 009; harmless due to IF NOT EXISTS but signals a copy-paste oversight.
lib/finance-imports.ts Fixes artifact SHA-256 backward-compat by storing normalized artifact JSON (with artifactSha256 filled in) via withFinalArtifactSha256; adds classifyImportTransactionDates helper.
src/lib/client/mutation.ts New shared mutation envelope handler: posts JSON, parses envelope, fires toasts, SvelteKit invalidation, and redirects in a single utility; clean implementation.
lib/client-contract.ts New shared mutation envelope contract with Zod schemas for invalidation keys, toasts, jobs, and events; well-typed and strict.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[rebuildFinanceKnowledge] --> B[Build LedgerEntryDraft map]
    B --> C[email transactions loop]
    B --> D[import rows loop]
    C --> E[classifyLedgerDate / dateRecovery]
    D --> E
    E --> F[addOrMerge into entries map]
    F --> G[reapplyActiveFinanceLedgerOverrides]
    G --> H{Active overrides in DB?}
    H -- yes --> I[dbPatch + mergeMetadata]
    I --> J[Object.assign entry]
    H -- no --> K[skip]
    J --> L[bulk upsert finance_ledger_entries]
    K --> L
    M[applyFinanceLedgerOverride] --> N[dbPatch + mergeMetadata]
    N --> O[supersede old override]
    O --> P[insert new override]
    P --> Q[update ledger entry]
    Q --> R{affectsRollups?}
    R -- yes --> S[queue rebuild_finance_rollups]
    R -- no --> T[done]
    S --> T
    U[revertFinanceLedgerOverride] --> V[mark override reverted]
    V --> W[queue rebuild_finance_knowledge + rollups]
Loading
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
db/migrations/010_finance_ledger_override_indexes.sql:1-3
**Duplicate index already created by migration 009**

Migration 009 already contains an identical `CREATE INDEX IF NOT EXISTS finance_ledger_entry_overrides_status_updated_idx` statement on the same table with the same partial filter. Migration 010 is therefore a complete no-op after 009 runs. If 010 was intended to add the index retroactively to databases that ran an earlier version of 009 (before the index was part of it), that scenario can't occur since both migrations ship in the same PR.

### Issue 2 of 2
server/actions.ts:1535
**Unbounded full-table scan loads all columns including large JSON blobs on every finance page view**

`db.selectFrom("finance_ledger_entries").selectAll().execute()` has no `WHERE`, no `LIMIT`, and fetches every column — including `raw_payload_json`, `field_confidence_json`, and `ledger_metadata_json` — for every row in the table. As the ledger grows (multiple years × thousands of transactions), this will load potentially tens of megabytes of JSON on every `loadFinanceData` call. Consider selecting only the columns that `summarizeExportReadiness` and `buildFinanceDrilldown` actually consume, or scoping the query to the rows already in `filteredCanonicalKeys`.

Reviews (4): Last reviewed commit: "feat: migrate operator UI to SvelteKit" | Re-trigger Greptile

Comment thread lib/finance-imports.ts
Comment thread lib/finance-upload.ts Outdated
Comment thread server/actions.ts
Replace the Hono JSX/browser partial stack with SvelteKit page rendering and Hono-owned API/auth/WebSocket routes. Add the client mutation invalidation contract, WebSocket realtime protocol, SvelteKit route skeleton, and updated docs/tests for the clean-break target.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant