Skip to content

Add sync error logging and --rebuild-cache flag#227

Closed
robelkin wants to merge 5 commits intowesm:mainfrom
robelkin:fix/sync-error-logging-and-cache-rebuild
Closed

Add sync error logging and --rebuild-cache flag#227
robelkin wants to merge 5 commits intowesm:mainfrom
robelkin:fix/sync-error-logging-and-cache-rebuild

Conversation

@robelkin
Copy link
Copy Markdown
Contributor

Summary

  • Log message IDs at WARN level when GetMessagesRawBatch returns nil (message fetch failed), in both incremental and full sync paths. Previously the error count incremented silently with no indication of which messages were affected.
  • Add --rebuild-cache flag to sync and sync-full commands to optionally rebuild the Parquet analytics cache after sync. The serve command already does this automatically, but standalone CLI invocations (e.g. via launchd) left the cache stale.

Usage: msgvault sync --rebuild-cache or msgvault sync-full --rebuild-cache

🤖 Generated with Claude Code

@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Mar 26, 2026

roborev: Combined Review (17efaac)

The PR safely introduces cache rebuilding and logging improvements, but contains a logic mismatch regarding the new rebuild flag's behavior.

Medium

  • Location: cmd/msgvault/cmd/sync.go:177, cmd/msgvault/cmd/ syncfull.go:169
  • Problem: The new --rebuild-cache flag does not actually force a rebuild; it only runs buildCache when cacheNeedsBuild(...).NeedsBuild is true. If a user invokes the flag to recover from a bad-but-
    not-detected cache state, the command will silently skip the rebuild even though the flag/help text says it will rebuild the analytics cache.
  • Fix: Make --rebuild-cache bypass the staleness check and rebuild unconditionally, or rename/document it as a conditional refresh instead of a rebuild.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Mar 26, 2026

roborev: Combined Review (2a9aa80)

The PR introduces a high-severity compilation error and medium-severity error handling
issues that must be addressed before merging.

High

  • Location: cmd/msgvault/cmd/sync.go (around line 180), cmd/msgvault/cmd/syncfull.go (around line 172)
  • Problem: The code attempts to assign
    the result of cacheNeedsBuild(dbPath, analyticsDir) to a single variable staleness and access a FullRebuild field. However, cacheNeedsBuild returns two values (bool, string), not a struct. This results in a compilation error: "assignment mismatch:
    1 variable but cacheNeedsBuild returns 2 values" and "staleness.FullRebuild undefined".
  • Fix: Remove the invalid staleness check. To force an incremental cache rebuild (which exports newly synced messages), simplify the logic to directly call buildCache(dbPath, analyticsDir, false) .

Medium

  • Location: cmd/msgvault/cmd/sync.go:177, cmd/msgvault/cmd/syncfull.go:169
  • Problem: --rebuild-cache failures are downgraded to warnings, so the command can exit
    successfully even when the explicitly requested cache rebuild did not happen. That makes automation and user feedback inaccurate for this flag.
  • Fix: Propagate buildCache errors as command failures when --rebuild-cache is set, or at minimum return a non-zero exit after sync completes.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

When GetMessagesRawBatch returns nil for a message (typically a 404
because the message was deleted between listing and fetching), the
error count was incremented but no message ID was logged. This made
it impossible to diagnose which messages failed to sync.

Add Warn-level logging with the message ID in both incremental and
full sync paths so failed fetches are visible in logs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The serve command rebuilds the Parquet analytics cache after each
sync, but the standalone CLI commands did not. This caused stale
search results when syncs were triggered via launchd or manual CLI.

Rather than always rebuilding (which couples sync and caching), add
an opt-in --rebuild-cache flag that users can enable in their
launchd config or scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the flag is set, always run buildCache rather than gating on
cacheNeedsBuild. The staleness check is still consulted to decide
whether a full rebuild is needed, but the cache build itself runs
unconditionally so users can recover from bad-but-undetected cache
states.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the user explicitly requests cache rebuild via --rebuild-cache,
failures should cause a non-zero exit so automation can detect them.
Previously these were silently downgraded to log warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wesm wesm force-pushed the fix/sync-error-logging-and-cache-rebuild branch from 2a9aa80 to b4d510e Compare April 2, 2026 13:27
@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Apr 2, 2026

roborev: Combined Review (b4d510e)

Verdict: Changes are not ready to merge due to one blocking compile error and one medium-severity behavior mismatch in the new cache rebuild flow.

High

  • Compile error at cmd/msgvault/cmd/sync.go:180 and cmd/msgvault/cmd/syncfull.go:172: cacheNeedsBuild returns two values (bool, string), but the code assigns it to a single variable staleness and then accesses staleness.FullRebuild. This will not compile. Use separate return values such as needsBuild, reason := cacheNeedsBuild(...), or change cacheNeedsBuild to return the struct the call site expects.

Medium

  • --rebuild-cache does not actually bypass the staleness check as described at cmd/msgvault/cmd/sync.go:180 and cmd/msgvault/cmd/syncfull.go:172: the new path still calls cacheNeedsBuild, and if fullRebuild is false then buildCache can still skip work via its own staleness logic. If the intended behavior is an unconditional rebuild, pass true to buildCache; if the intent is only to skip the outer check, call buildCache directly without depending on cacheNeedsBuild.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Report cache failures independently rather than mixing them into the
"account(s) failed to sync" message. Sync failures and cache failures
are now distinct error paths with accurate messages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Apr 2, 2026

roborev: Combined Review (07f61c2)

Verdict: Changes are mostly sound, but --rebuild-cache does not reliably honor an explicit forced rebuild request.

Medium

  • cmd/msgvault/cmd/sync.go:180, cmd/msgvault/cmd/syncfull.go:172
    --rebuild-cache still goes through normal cache staleness checks, so an explicit rebuild request can become a no-op when the cache is considered current. That behavior does not match the flag’s meaning and prevents users from forcing a rebuild after cache logic or schema changes.
    Suggested fix: add an explicit force path so --rebuild-cache rebuilds even when staleness logic says the cache is up to date.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@wesm
Copy link
Copy Markdown
Owner

wesm commented Apr 2, 2026

closing in favor of #241

@wesm wesm closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants