refactor: modularize CLI and adopt argparse subparsers

[flitzrrr]

Stack position

This is PR 2 of 2 in the fork-native stack.

• Base PR (merge first): feat: authenticated crawling + SPA support (stack base) #2
• This PR (merge second): refactor: modularize CLI and adopt argparse subparsers #1

Summary

Implements the architecture follow-up from reviews/pr2-review.md.

Changes

• Split crawler/cli.py into focused modules:
  • crawler/cli_auth.py
  • crawler/cli_output.py
  • crawler/cli_config.py
  • crawler/cli_parsers.py
• Switched crawl command routing to argparse subparsers (capture-auth).
• Preserved backward-compatible CLI UX (crawl <url> still works).
• Preserved compatibility wrappers in crawler/cli.py for existing tests/internals.

Validation

• uv run --extra dev --with pytest-cov python -m pytest --cov=crawler --cov-report=term-missing
• Result: 271 passed, TOTAL 100% coverage.

[flitzrrr]

Final readiness check completed.

What was rechecked:

• Full test suite with coverage (uv run --extra dev --with pytest-cov python -m pytest --cov=crawler --cov-report=term-missing)
• CLI parser behavior for default sys.argv handling
• Stacked PR scope isolation (only architecture refactor + related tests)

One issue found during review check and fixed in b5e0478:

• parse_crawl_args(None) was not reading sys.argv after refactor.
• Fix: _normalize_crawl_argv now defaults to sys.argv[1:].
• Regression tests added in tests/test_cli.py.

Current status:

• 271 passed
• TOTAL 100% coverage
• Branch is clean and ready for review/merge.