Skip to content

feat(scan): add application data-boundary diagnostic mode#621

Open
fraware wants to merge 17 commits into
open-jarvis:mainfrom
fraware:main
Open

feat(scan): add application data-boundary diagnostic mode#621
fraware wants to merge 17 commits into
open-jarvis:mainfrom
fraware:main

Conversation

@fraware

@fraware fraware commented Jul 1, 2026

Copy link
Copy Markdown

Summary

This PR extends jarvis scan with a new application data-boundary mode:

jarvis scan --data-boundaries
jarvis scan --data-boundaries --json
jarvis scan --data-boundaries --strict

The new mode reports configured OpenJarvis data surfaces such as cloud-capable
inference settings, memory-to-cloud composition, traces, learning/training,
web search, MCP, browser/local tools, server/A2A exposure, messaging channels,
connector credential files, local runtime stores, and API-key environment
variables.

The scanner inspects configuration values, environment-variable presence, and file existence.
It does not read connector-token contents, memory databases, trace databases, prompts,
logs, OAuth scopes, browser localStorage, Tauri storage, or provider policies.

Why

OpenJarvis is local-first while also supporting optional cloud engines,
connectors, tools, channels, traces, memory, learning, and optimization. Users
need a simple way to verify whether a configuration is local-only, cloud-capable,
or a mixed setup where local memory may be injected into cloud-bound prompts.

The existing jarvis scan command audits host/environment posture. This PR adds
an application-level counterpart without creating a competing top-level command.

What changed

  • Added openjarvis.security.data_boundary_audit with structured findings and
    redacted JSON output.
  • Added jarvis scan --data-boundaries with table and JSON output.
  • Added --show-paths for local debugging while keeping paths redacted by
    default.
  • Added --strict to fail on either fail or warn findings.
  • Honored OPENJARVIS_CONFIG so the scan audits the same explicit config file
    used by the runtime.
  • Reported OpenJarvis home/config-root resolution errors as findings rather than
    crashing.
  • Skipped the global update check only for jarvis scan --data-boundaries, so
    this diagnostic mode does not trigger the normal update-check thread.
  • Separated channel secrets from channel endpoints/identifiers and added local
    messaging credential-directory checks.
  • Added local-store permission warnings for symlinked or group/other-readable
    stores on POSIX systems.
  • Added tests for core report construction, redaction, cloud/memory composition,
    tools, MCP, channels, env credentials, strict mode, config/root-load failures, and
    normal jarvis scan regression coverage.
  • Added user documentation.

Non-goals

Review follow-up validation

Addressed the five items from Elliot's review in 12fa4e79:

  • Ruff formatting applied to all changed Python files.
  • channel.default_channel is presence-only and redacted in all output modes.
  • WhatsApp Baileys default auth path updated to whatsapp_baileys_bridge/auth.
  • Registered browser and local-access tool names added with regression coverage.
  • GEMINI_API_KEY detection added with redaction coverage.
  • Targeted pytest: 88 passed, 2 skipped (98f4df1f).
  • ruff check: passed.
  • ruff format --check: passed.
  • compileall: passed.
  • mkdocs build --strict: not passed locally (18 pre-existing doc warnings under strict mode; left to upstream CI).

fraware and others added 11 commits July 1, 2026 01:28
feat(scan): add application data-boundary diagnostic mode
Drop the one-off apply patch helper, dedupe security user-guide copy, and add a CLI integration test ensuring scan --data-boundaries skips update checks.
Add integration test for scan --data-boundaries update-check suppression and replace duplicated three-layer section with a short link.
chore: data-boundary scan PR cleanup
feat(scan): add application data-boundary diagnostic mode
@fraware

fraware commented Jul 1, 2026

Copy link
Copy Markdown
Author

Pushed a small test-formatting cleanup and re-ran the targeted local checks.

@fraware

fraware commented Jul 1, 2026

Copy link
Copy Markdown
Author

Follow-up: hardened _low_noise_config()\ in both data-boundary test modules so JarvisConfig defaults (absolute store paths under the real OPENJARVIS_HOME, MCP enabled, etc.) no longer leak warn findings into info-only scenarios. Targeted suite re-run: 54 passed, 2 skipped. Pushed to \origin/main\ as \37a4f692.

@fraware

fraware commented Jul 1, 2026

Copy link
Copy Markdown
Author

The latest push includes the test cleanup and the _low_noise_config()\ hardening noted above. The targeted local suite is green: 54 passed, 2 skipped.

@fraware

fraware commented Jul 1, 2026

Copy link
Copy Markdown
Author

Follow-up: preserved configured store paths for POSIX symlink detection while keeping resolved-path deduplication. Linux-targeted local-store permission tests pass, including test_symlink_local_store_permission_warns (POSIX symlink/group-permission cases skip locally on Windows; fix validated via full security/CLI pytest suite here). Targeted Ruff, pytest, and compileall are green.

@ElliotSlusky

Copy link
Copy Markdown
Collaborator

Hey @fraware, thank you for the PR! I have a few fixes for you:

  1. Blocking: formatter check fails
    ruff format --check would reformat:
    scan_cmd.py, data_boundary_audit.py, test_scan_data_boundaries.py.
    The repo checklist includes format checking, so this should be fixed before merge.

  2. High: redacted output can leak channel.default_channel
    data_boundary_audit.py appends the raw default_channel value into evidence. That value can contain user identifiers like phone-number-style or chat-channel targets. For a “paste-safe” audit report, this should be presence-only, e.g. “default_channel is set; value redacted”.

  3. High: WhatsApp Baileys auth directory check looks at the wrong default path
    data_boundary_audit.py checks whatsapp_auth, but the actual Baileys default is whatsapp_baileys_bridge/auth from whatsapp_baileys.py and whatsapp_baileys.py.
    This creates a false negative for the real default credential directory.

  4. High: tool detection misses real registered tool names
    data_boundary_audit.py only checks names like browser_open, but the app registers tools like browser_navigate, browser_click, browser_type, browser_screenshot, and browser_extract.
    Same problem at data_boundary_audit.py: local-access detection misses real sensitive tools like file_write, apply_patch, docker_shell_exec, db_query, knowledge_sql, memory_manage, and repl. This weakens the audit’s main purpose.

  5. Medium: cloud key scan misses GEMINI_API_KEY
    data_boundary_audit.py includes GOOGLE_API_KEY but not GEMINI_API_KEY, even though the runtime supports both. That is another false negative.

Verification

Passed:

OPENJARVIS_HOME=/tmp/openjarvis-test-home-621 PYTHONPATH=src python3 -m pytest tests/security/test_data_boundary_audit.py tests/cli/test_scan_data_boundaries.py -q
# 56 passed

python3 -m ruff check src/openjarvis/security/data_boundary_audit.py src/openjarvis/cli/scan_cmd.py src/openjarvis/cli/__init__.py tests/security/test_data_boundary_audit.py tests/cli/test_scan_data_boundaries.py

python3 -m compileall -q src/openjarvis/security/data_boundary_audit.py src/openjarvis/cli/scan_cmd.py src/openjarvis/cli/__init__.py

Failed:

python3 -m ruff format --check ...
# 3 files would be reformatted

@fraware

fraware commented Jul 15, 2026

Copy link
Copy Markdown
Author

Thank you, Elliot — this was very helpful. I addressed all five items:

  • Applied Ruff formatting and confirmed ruff format --check passes.
  • Changed channel.default_channel reporting to presence-only redaction in all output modes.
  • Updated the WhatsApp Baileys default credential directory to whatsapp_baileys_bridge/auth.
  • Expanded browser and local-access detection to cover the registered tool names you identified.
  • Added GEMINI_API_KEY detection and regression coverage.

I also added focused tests for each case and re-ran the same validation commands from your review:

  • Targeted pytest: 75 passed
  • ruff check: passed
  • ruff format --check: passed
  • compileall: passed
  • mkdocs build --strict: skipped locally (gen-files plugin not installed)

The fixes are in commit 12fa4e7. Thank you again for the detailed review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants