Git Insights Hub

⚠️ SECURITY: this application ships with NO built-in authentication. Every API and dashboard route, including the write/execute endpoints (POST /api/run/*), is open to anyone who can reach the port. Never expose it directly to an untrusted network. Run it behind an authenticating reverse proxy / SSO gateway / VPN. It also holds a GitHub token and (optionally) an LLM API key and connects to a database with your organization's repository contents; treat the host and DB accordingly. See Deployment and Limitations.

Organization-wide GitHub analytics and portfolio intelligence platform. It scans every repository in your GitHub organization, extracts contributor and pull-request metrics, runs AI-powered code quality and security assessments, and serves the results as an interactive dashboard, plus a deep portfolio-analysis pipeline that produces executive-grade modernization briefs for a selected set of repositories.

Built by the Government of Alberta and designed to be deployed by any organization: everything org-specific (organization name, GitHub orgs, divisions, staff-login conventions, compliance frameworks) lives in a JSON profile you supply, nothing is hardcoded.

What it does

Capability	How
Repository discovery	GitHub GraphQL/REST scan of all repos: metadata, languages, PRs, reviews, branches, milestones
Contributor intelligence	Per-contributor commit/PR metrics, effectiveness scoring, staff-vs-external classification, AI behavioral analysis
Code assessment	Local git clone + AI review of key files; scores for quality, security, architecture, maintainability
Vulnerability pre-pass	46-pattern static grep across all repo files (PHP, JavaScript/Node, Java, .NET, Python) feeding the AI review
Capability enrichment	Auth patterns, database tech, legacy markers, integration points, TIME classification
PII exposure scan	AI-assisted detection of personally identifiable information committed to repositories
Portfolio pipeline	UI-driven deep analysis of a selected repo set: inventory, consolidation candidates, cost model, modernization plan, executive brief
Dashboard	Express + Vue single-page apps: org overview, workforce, exposure, code health, portfolio runs

Requirements

Core (required):

• Node.js 22+
• PostgreSQL 14+, any instance; schema is created by npm run migrate
• git on the PATH, required for the clone-based scans (deep-scan, capability-scan, portfolio, robust security scan)
• GitHub Personal Access Token, classic scopes repo, read:org, read:user (or the fine-grained equivalents) on the organization you scan
• LLM provider for AI features, either the Anthropic API directly (ANTHROPIC_API_KEY) or Claude on Google Vertex AI (service account with the Vertex AI User role). Choose the provider that matches your data classification — see Choosing an LLM provider below

Optional (enable extra capabilities in the robust security scan / reports):

• Semgrep (pipx install semgrep), deterministic static-analysis detector. Without it, that layer is skipped.
• A Chrome/Chromium binary, for PDF report rendering. Without it, reports are produced as HTML only.

Before deploying for another organization, see Adapting for your organization below, a few defaults (organization profile, the bundled design system, and the Chrome discovery path) assume the original deployment and should be replaced.

Quick start

git clone <this-repo> && cd git-insights-prep
npm install

cp .env.example .env        # fill in GITHUB_PAT, DB_CONNECTION_STRING, LLM keys
npm run migrate             # create/update all database tables (idempotent)
node test-llm.js all        # optional: verify your LLM provider works

npm run scan                # phase 1a: GitHub API discovery (repos, PRs, contributors)
npm run deep-scan           # phase 1b: clone + AI code assessment
node capability-scan.js     # phase 2: capability enrichment (optional)

npm start                   # dashboard at http://localhost:3000

All scans are resumable, re-running the same command picks up where an interrupted scan left off. Use the :force variants (npm run scan:force, npm run deep-scan:force) to start over, or :no-ai to collect data without LLM calls.

See SCANS.md for the full scan reference (what each scan collects, durations, concurrency, troubleshooting).

Documentation map

Doc	What you find there
docs/RUNNING.md	Operator runbook: the full flag and environment-variable reference for every scan, the robust pipeline, reports, and the fleet campaign
docs/FLEET-SCAN.md	Whole-of-org fleet campaign: enqueue, run detached workers, resume, and harvest results across the entire organization
docs/DATAPACK.md	Unclassified whole-of-gov export: the de-identified capability/vintage/portfolio DataPack, its reproducible close-out, and the de-identification guarantees
SCANS.md	The core collection scans (org / deep / capability / extract): what each collects, how to run them, and what they unlock in the dashboard
docs/CONFIGURATION.md	Every organization-profile field and every environment variable, documented

Configuring your organization

Select a profile with the ORG_PROFILE environment variable:

ORG_PROFILE=default npm start          # profiles/default.json
ORG_PROFILE=myorg npm start            # profiles/myorg.json
ORG_PROFILE=/abs/path/profile.json npm start

A profile is a single JSON file describing your organization, its name, GitHub org(s), internal email domains, division/ministry structure, staff-login conventions, compliance frameworks, and classification hints for the AI. Missing keys inherit from profiles/default.json, so a minimal profile only needs the fields you care about:

{
  "id": "myorg",
  "organization_name": "State of Examplia",
  "organization_short": "SoE",
  "github_orgs": ["examplia-gov"],
  "internal_email_domains": ["examplia.gov"],
  "internal_login_suffixes": ["_soe"],
  "division_label": "Department",
  "divisions": [{ "short_name": "DOT", "name": "Department of Transportation" }]
}

Every profile field is documented in docs/CONFIGURATION.md, and profiles/example.json is a complete, generic worked example you can copy.

Keep your operational profile out of git. Your real profiles/<org>.json carries your GitHub org, repo prefixes, and program-code maps, so add profiles/<org>.json to .gitignore and keep it local. Nothing org-specific is hardcoded anywhere in the codebase, so the committed tree stays fully generic: one version serves every organization, with no real system or repository names in it. The shipped profiles/example.json and profiles/default.json are the only profiles in the repo.

Choosing an LLM provider (data classification)

The AI features send repository content (source code, file paths, commit metadata) to a Claude model. Pick the provider whose data-handling terms match the classification of the repositories you are scanning, per your own security threat and risk assessment (STRA) and enterprise agreements:

Provider	LLM_PROVIDER	Intended for	Basis
Anthropic API (direct)	anthropic	Unclassified workloads	Standard commercial API terms
Google Vertex AI (Claude on the Google Enterprise Agent Platform)	vertex	Protected A and Protected B workloads	Data stays within your GCP enterprise tenancy/region under your enterprise agreement

This is the mapping the Government of Alberta operates under its current STRA: Anthropic direct for unclassified data, and Claude on Google Vertex AI (Enterprise Agent Platform) under the enterprise agreement for Protected A/B data. Other jurisdictions should substitute their own classification scheme and the terms their agreements actually cover, this is deployer configuration, not a property of the code. The provider is selected entirely by LLM_PROVIDER (+ the matching credentials) in .env; the model, prompts, and outputs are identical across providers.

Note on the opt-in Agent SDK paths. Two optional features, the agentic verifier (AGENT_VERIFIER=1) and the Agent-SDK BCM deep-dive (bcm-scan --max / BCM_DEEPDIVE_AGENT=1), run through @anthropic-ai/claude-agent-sdk, which talks to the Anthropic API directly regardless of LLM_PROVIDER. Both default off. On a Vertex/Protected-A-or-B deployment, leave them off (the default one-shot verifier and the default parallel BCM deep-dive both run on your selected Vertex provider) unless your STRA also covers the direct Anthropic path.

The dashboard

Page	Purpose
/org.html	Organization dashboard, repos, contributors, code health, exposure, workforce intelligence
/portfolio.html	Portfolio analysis runs, pick repos in the UI, launch the pipeline, browse artifacts live
/index.html	Single-repository deep dive (contributor analytics for one repo)
/status.html	Scan job status and live logs

The UI ships with a working default theme under public/ds/ (stylesheet, design tokens, and Tailwind/chart config), so a fresh clone renders cleanly with no extra setup. Fonts come from Google Fonts via CDN; no proprietary binaries are bundled. To rebrand, replace the files in public/ds/, the app references them by fixed paths and needs no code changes. The only asset not redistributed is the masthead logo image (the masthead hides it gracefully when absent). See public/ds/README.md.

The portfolio pipeline

The portfolio pipeline (lib/portfolio/) is a 15-phase deep analysis for a chosen set of repositories: full clones, per-repo inventory, application framework, consolidation candidates, cost modeling, target architecture, migration plan, and an executive brief written for the audience defined in your profile (executive_audience).

Everything is driven from /portfolio.html: select repositories (search/filter, up to 200 per run), launch, watch phase progress and live logs, and browse the generated artifacts. Each run is an isolated workspace under data/portfolio/<runId>/, runs never interfere with each other and can be deleted from the UI.

Robust security scanning and reports

Beyond the AI code assessment built into deep-scan.js, the project includes a multi-detector security pipeline that combines deterministic ground truth with AI verification to keep false positives and negatives low. The robust security scan itself (and a portfolio-wide vulnerable-package rollup) is surfaced in the web UI: launch it from the Scan control panel (/status.html → "Robust security scan") and browse results in the Robust security scan tab on /org.html. The downstream rebuild-decision artifacts (Chief Architect, business case, bottom-up estimate, disposition, the Business Capability Map, the portfolio meta-report, and the fleet campaign) are currently report/PDF and CLI only; dedicated UI pages are being added.

npm run robust-scan -- --repo owner/name        # full multi-detector scan of one repo
npm run portfolio-scan -- --concurrency 5        # scan the whole active portfolio (resumable)
npm run report -- --last 20                      # generate per-repo HTML + PDF reports
npm run report -- --repo owner/name --chief-architect   # + rebuild proposal, estimate, business case
npm run portfolio-report -- --repos owner/a,owner/b --title "My System"   # multi-repo ministry meta-report

Full flag + env reference: docs/RUNNING.md.

Three different "portfolio" things, do not confuse them: (a) the legacy 15-phase UI modernization pipeline (lib/portfolio/run.js, driven from /portfolio.html); (b) the resumable security batch that runs the robust scan across every active repo (scripts/portfolio-scan.js, npm run portfolio-scan); and (c) the ministry/system meta-report that rolls a chosen set of already-analyzed repos into one document (scripts/portfolio-report.js, npm run portfolio-report).

The pipeline layers:

• Semgrep static analysis (language- and cloud-specific rule packs), optional, recommended
• OSV.dev dependency-vulnerability correlation (no API key needed)
• Secret detection (regex + entropy, in-tree, no dependency)
• Framework reachability, parses framework wiring to identify externally reachable entry points
• Import-graph tree walking, upstream callers / downstream dependencies of each flagged file
• AI verification, a large language model re-evaluates each candidate finding against the actual code, reachability, and a framework knowledge base, then a consensus step assigns the final severity
• Scan integrity & coverage gate, a final one-pass self-audit of the scan itself (see below)

Scan integrity & coverage gate

The last stage answers a question the detectors never ask: did this scan actually analyze what it thinks it analyzed, or did something silently degrade it? A scan can finish "successfully" while a crashed static analyzer, unfetched git submodules, an unsupported dependency manager, or an unrecognized framework quietly cause it to miss large parts of the codebase, producing a confident-but-shallow "all clear" that is worse than no scan.

The gate (lib/security/integrity-gate.js) runs two layers:

1. Deterministic checks (always run, no LLM, no cost) read the scan's own telemetry and flag known degradation modes: a static analyzer that ran but found nothing on a large codebase, git submodules that were declared but never populated, "no dependency manifest found" being reported as a clean dependency score, and thin deep-analysis coverage.
2. One Opus meta-review pass (skippable via --skip-integrity) reads the same telemetry and writes a plain-English confidence assessment, identifies the probable errors that limited the analysis with concrete remediation steps, and lists any output that should not be trusted given the gaps.

Every report opens with a Scan integrity & confidence section (confidence high/medium/low, completeness, the concerns found, what not to trust, and how to close the gaps), and the assessment is persisted to scan_metrics (integrity_confidence, integrity_completeness, integrity_degraded, integrity_assessment). This makes a degraded scan loud, the honest coverage caveat is produced automatically, whether or not a human reviews the run.

Note: the standalone pipeline does not yet auto-correct the gaps it finds (e.g. re-cloning with submodules). It surfaces them with remediation guidance; acting on them is a manual re-run. Cloning does not currently recurse git submodules, for a repo with submodules, clone manually with git clone --recurse-submodules and scan via --repo-dir.

Application architecture diagram

Each scan also infers an architecture diagram, a Mermaid model of the application's internal components and the external systems it depends on (other APIs, databases, cloud services, message queues, identity providers), reconstructed from detected frameworks, entry-point routes, dependency manifests, and integration signals grepped from the code (HTTP clients, DB drivers, cloud SDKs, hostnames, env vars). External dependencies are tagged observed (a direct signal was found) or inferred (deduced from dependencies/config). The diagram is pre-rendered to an inline SVG (self-contained, no CDN, no client JS, prints to PDF) and appears as the Application architecture section of the report, alongside an external-dependency table. Rendering reuses the PDF Chrome binary plus a vendored mermaid.min.js (lib/reporting/vendor/), so there is no npm dependency. Disable with --skip-architecture.

Full file-tree inventory (coverage feelers)

Every scan first builds a full file-tree inventory (lib/scan/file-inventory.js): every file is walked, categorized (source / config / service-def [.svc/.wsdl] / manifest / infra / test / doc / data / binary), language-tagged, and run through deterministic security "feelers" (WCF clientCredentialType="None", ValidateInput(false), raw-SQL concatenation, unsafe deserialization, connection strings, weak crypto, …). This is persisted per scan (scan_file_inventory, migration 25) as the honest coverage denominator and a checkback artifact, and it drives file selection: signal-bearing files, service definitions, and key configs are force-included in the deep read (the layer a keyword selector misses), and one LLM pass over the compressed tree pulls in anything still missed. Combined with C# NuGet (packages.config/.csproj) dependency parsing, this closed a real gap where a legacy .NET repo scanned as "clean" while its unauthenticated WCF services and vulnerable dependencies went unseen.

Business Capability Map (npm run bcm-scan)

A separate, on-demand, heavier augmenting scan (scripts/bcm-scan.js) that maps what an application does. It walks every business-relevant file, groups them into coherent file-sets, maps each set to a business capability (open-minded, it proposes new capabilities, not limited to a fixed list), takes deep dives on workflow/screen/key-logic nodes to extract functions and workflows, then reconciles discoveries into a canonical, organically-growing whole-of-government capability map (deduped by name/alias; migration 26: business_capabilities, file_capability_map, v_bcm_capabilities). Runnable per repo or across the org; each run augments the universal map. The report renders it as an inline treemap (domain → capability by file count) plus a capability/coverage table. Deep dives default to a fast provider pass; set BCM_DEEPDIVE_AGENT=1 for the autonomous Agent-SDK explorer where available.

npm run bcm-scan -- --repo owner/name        # map one repo into the growing BCM
npm run bcm-register -- --out register.html  # whole-of-gov register report (cross-repo, consolidation candidates)
npm run bcm-register -- --merge "A" "B"      # governance: merge two capabilities
npm run reset-bcm -- --yes                    # rebuild the register from a clean baseline

Full flag + env reference: docs/RUNNING.md.

Deep dives extract rebuild-grade detail per capability, screens, workflows, data entities, business rules (as testable Given/When/Then), API contracts, and acceptance criteria, the functional-spec material an AI-Factory rebuild needs. Every scan commits its observations into the canonical, deduplicated, whole-of-government register (business_capabilities), and the register report highlights capabilities implemented by multiple repositories (consolidation candidates).

Reports inform the rebuild decision (AI Garage vs AI Factory)

Beyond findings, each report now surfaces the full decision context: a Disposition recommendation (Remediate / Rebuild / Retire-Consolidate / Maintain, with rationale + signals), a Maintenance & activity block (commit velocity, recency, bus factor), a Technology & modernization profile (DB/auth/integrations/TIME/effort), the AI-drafted README, and the business-capability tree with per-capability security risk and deep-dive detail. Together they inform how and whether an application should be remediated in place or rebuilt.

When the disposition is Rebuild (or you pass --chief-architect), the report adds three decision-grade artifacts, all driven by editable org companion docs (see below):

• Chief Architect, given your opinionated target-stack standards (profiles/<org>.standards.md), a phased migration plan, a future-architecture proposal mapped onto your stack, a rendered target-architecture diagram, a legacy→target component mapping, and a 12-item rebuild-readiness checklist.
• Bottom-up estimate, the rebuild cost/effort computed from the app's actual extracted unit counts (services, screens, workflows, APIs, data entities) × the per-unit rates in your estimation heuristics (profiles/<org>.estimation.md), shown as an auditable line-item table. The rates assume an AI-assisted team (no double-discount).
• Business case, an executive, Deputy-Minister-ready case following the government Five Case Model (profiles/<org>.business-case.md): recommendation-and-ask, the opportunity, critical business capabilities (what each does, who relies on it, what's lost), a four-option short-list (status quo / remediate / rebuild / procure-RFP), cost of doing nothing, benefits + measures, and total cost of ownership. Written in business terms; its rebuild figures are locked to the estimate.

Companion docs (all markdown, org-editable; default.* is the generic template, alberta.* a worked example): profiles/<org>.standards.md, .estimation.md, .business-case.md. Each is resolved by --flag > ENV > profiles/<ORG_PROFILE>.<suffix> > profiles/default.<suffix>. Report flags: --chief-architect / --no-chief-architect / --business-case / --no-business-case / --refresh-architect / --standards <path> / --estimation <path> / --business-case-doc <path>.

Ministry / portfolio meta-analysis (npm run portfolio-report)

Roll a set of already-analyzed applications (2-200) into one ministry / system document, built from the persisted per-repo analysis (no rescan) plus one LLM meta-pass. It ties the applications together and drops the granular cyber detail (that stays in the per-app reports):

npm run portfolio-report -- --repos owner/a,owner/b,owner/c --title "My System"
npm run portfolio-report -- --scan-ids <id1>,<id2> --title "My System"   # by scan id instead
npm run portfolio-report -- --repos ... --refresh   # recompute the meta-analysis (else cached)

The report covers: an executive summary, how the applications relate (system interplay), a portfolio-at-a-glance table, consolidated capabilities (with cross-repo consolidation candidates), portfolio health, rolled-up costing (summed modernization investment), a suite-level meta business case, a future-state vision + target architecture, and a phased modernization roadmap. Flags: --repos / --scan-ids / --title / --out <dir> / --refresh / --no-pdf. The meta-analysis is cached in portfolio_analyses (keyed by the repo set).

Whole-of-org fleet scan (scripts/fleet-scan.js)

For a full campaign across an entire organization (hundreds to thousands of repos), the fleet scan is a control plane that queues every repo, runs the robust pipeline + report generation on each through detached workers, and lets you resume and harvest at any point. It is the batch you reach for when portfolio-scan is not enough and you want a durable, inspectable run record.

node scripts/fleet-scan.js --enqueue                        # create a fleet run and queue its repos
node scripts/fleet-scan.js --start --run <id> --workers N   # launch N detached workers for that run
node scripts/fleet-status.js --run <id>                     # live progress, per-repo status, failures

Each run and each per-repo job is persisted (fleet_runs, fleet_jobs), so the campaign is fully resumable: workers claim jobs atomically, killed workers can be requeued (--requeue --run <id>), and every failure is logged with its error for follow-up. Harvest the whole campaign from the v_fleet_results view (one row per repo with health, findings, disposition, and estimate). Full flag + env reference: docs/FLEET-SCAN.md and docs/RUNNING.md.

Unclassified whole-of-gov export (DataPack)

After a fleet scan, npm run datapack -- --run <id> produces a de-identified, shareable DataPack: a set of JSON files describing the estate's shape (business capabilities, domains, technology vintage, health bands, best-fit government-function portfolio) with every identifying detail stripped. Application names become opaque UUIDs; there are no CVEs, package names, hosts, file paths, secrets, or organization/system names in the shared files. It is safe to hand to another jurisdiction or a vendor to answer "what does this government build, and where would you help?" A holistic overview PDF (DATAPACK-OVERVIEW.pdf) and an interactive explorer (DATAPACK-EXPLORER.html, a drill-in capability treemap) ship alongside it.

ORG_PROFILE=<org> npm run datapack -- --run <fleetRunId>            # incremental close-out (default)
ORG_PROFILE=<org> npm run datapack -- --run <id> --rebuild --refresh-domains  # re-cluster + re-derive

The close-out runs one correct, reproducible sequence: abstract new capabilities to generic labels → fold them into a frozen, monotonic canonical taxonomy (adding repos only grows it, never re-compresses) → assign domains derived bottom-up from the scan data (not a legacy seed list) → export the de-identified JSONs → regenerate the PDF + explorer. Identical inputs produce a byte-identical export. The taxonomy, domain vocabulary, tech-vintage table, and ministry→portfolio map are all org-editable profile files (profiles/<org>.*). The withheld crossref.json and AUDIT.json (marked _DO_NOT_DISTRIBUTE) hold the UUID↔repo mapping and the real ministry, and are never shared. Full detail: docs/DATAPACK.md.

Reproducibility & consensus

The pipeline is deterministic and reproducible by design. Decision-grade numbers are computed in code, never by the LLM (the rebuild estimate, health and risk scores, coverage percentages), so identical inputs produce an identical result. Inputs handed to the model are sorted and stable, LLM calls are truncation-safe and retry transient failures (LLM_MAX_RETRIES, default 5), and setting CONSENSUS_SAMPLES=3 draws the enumerated LLM outputs several times and keeps only the agreed findings, tightening run-to-run consistency.

Depth & coverage controls

How much code the AI reads is tunable (see .env.example). A small repo whose files fit under the caps is read in full; larger repos are prioritized reachable-code-first, the AI selector draws from the entry-point-rooted import-graph clusters, then unions in high-priority (auth/crypto/config) files. Key knobs: SCAN_MAX_KEY_FILES (default 600), SCAN_MAX_TOTAL_MB (8), and CLUSTER_LEFTOVER_MULTIPLIER (8, how much non-entry-reachable code to scan). Raise for deeper coverage at higher cost.

Results land in the database (code_issues, static_findings, scan_metrics, cross-repo views) and the report generator produces standalone HTML + PDF security reports suitable for handing to a development team. See docs/REPORTS.md and, for the API surface, docs/openapi.yaml.

Cost note: the AI stages call an LLM per file cluster, per candidate finding, and per report pass. The most comprehensive scan (robust security scan + BCM + report artifacts) on the current default models, Opus 4.8 (deep) and Sonnet 5 (fast), the most expensive models available, costs roughly $10 CAD per repository; a full portfolio of a few thousand repositories therefore runs into the low tens of thousands of dollars. This is tunable down sharply:

• Cheaper hosted models: set VERIFIER_MODEL_TIER=fast, CHIEF_ARCHITECT_MODEL_TIER=fast, BUSINESS_CASE_MODEL_TIER=fast, etc., and/or point the tier model names (VERTEX_CLAUDE_*_MODEL / ANTHROPIC_*_MODEL) at smaller/older models. Turn off the priciest stages with AGENT_VERIFIER=0 and skip flags (--skip-ai, --skip-integrity, --no-chief-architect, --no-business-case).
• Your own open-source models (no per-token cost): the provider layer is pluggable (lib/llm-provider.js exposes a small callWithTools / callText interface). Add a provider that targets a self-hosted model server (e.g. vLLM / llama.cpp / Ollama running Llama, Qwen, or Mistral) and select it via LLM_PROVIDER to run the whole pipeline at zero marginal cost, trading some analysis quality for price. This is a small code addition, not a built-in endpoint toggle: the bundled anthropic provider calls the hosted Anthropic API and the vertex provider calls Vertex; neither currently takes a custom base URL.

Adapting for your organization

This codebase originated at the Government of Alberta. A handful of things assume that origin and should be reviewed before you deploy it for a different organization:

• Organization profile, set ORG_PROFILE to your own profile under profiles/. The fallback is the generic profiles/default.json; profiles/alberta.json ships only as a worked example. Nothing silently adopts it.
• Companion profile docs drive the LLM/report passes and are all optional (each falls back to a generic profiles/default.* template): standards.md (rebuild target stack), estimation.md (rebuild rates), business-case.md (Five Case Model), and, for the DataPack, tech-vintage.md (technology→year table), ministries.md (ministry→generic-portfolio map), and deny-tokens.txt (de-identification proper-noun list). Copy the default.* or alberta.* versions and edit. profiles/README.md is the full step-by-step guide to building your org's profile set. Two files are not committed for your org: deny-tokens.txt (it names internal systems, gitignored — only the generic default ships) and the pipeline-generated derived-domains.json.
• Design system / branding, public/ds/ ships a working default theme (stylesheet, tokens, and Tailwind/chart config) so the dashboards render cleanly immediately. To rebrand, replace those files with your own, no code changes needed. Only the masthead logo image (assets/logo/masthead-logo.png) is not bundled; drop yours in to add it. See public/ds/README.md.
• Chrome discovery for PDFs, set PDF_CHROME_BIN to your Chrome/Chromium binary rather than relying on auto-discovery paths.

Testing

npm test        # node:test suite, profile loaders, SQL builders, scoring,
                # vulnerability patterns, manifest parsing, API smoke tests

API smoke tests exercise a live server if one is running on port 3000 and skip gracefully otherwise.

Security and privacy

• Secrets live only in .env (gitignored). .env.example documents every variable. Nothing in the codebase logs or returns credential material; the PAT is displayed only as •••• + last 4 characters.
• Scan data (data/, clones/, exports/) is gitignored, it contains your organization's repository contents and contributor data. Treat the PostgreSQL database with the same sensitivity.
• Repository clones for scanning are temporary and cleaned up after extraction; portfolio-run clones live inside the run workspace.
• Profile values are operator configuration, but they are still SQL-escaped before being used in queries.

Deployment

The app is a single Node process serving the API and static UI on $PORT (default 3000). It needs outbound HTTPS to github.com and your LLM provider, plus a PostgreSQL connection. See docs/DEPLOYMENT.md for container and platform notes.

Repository layout

server.js               Express API + static UI server
org-scan.js             Phase 1a: GitHub API discovery scan
deep-scan.js            Phase 1b: clone + AI code assessment
capability-scan.js      Phase 2: capability enrichment
contributor-scan.js     Workforce / contributor-risk intelligence
pii-scan.js             PII exposure scan
extract.js              Single-repo extraction engine
scripts/                Robust pipeline + report + campaign CLIs:
                          robust-scan.js       full multi-detector scan of one repo
                          portfolio-scan.js    resumable security batch across the portfolio
                          generate-report.js   standalone HTML + PDF report generator
                          bcm-scan.js          Business Capability Map scan
                          portfolio-report.js  ministry / system meta-report (no rescan)
                          fleet-scan.js        whole-of-org fleet campaign control plane
                          fleet-worker.js      detached per-repo fleet worker
                          fleet-status.js      fleet run progress / status
                          datapack.js          de-identified whole-of-gov DataPack close-out
                                               (abstract → consolidate → derive-domains →
                                                export → overview PDF → explorer)
db/                     PostgreSQL pool + idempotent migrations
lib/                    Scan engines, LLM provider, profile loader, exporters
lib/portfolio/          15-phase portfolio pipeline (spawned per run)
profiles/               Organization profiles (default.json + yours)
public/                 Dashboard SPAs + design system
tests/                  node:test suite

Limitations

• No built-in authentication. The app serves its API and dashboard without auth. Run it behind an authenticating reverse proxy / SSO gateway; never expose it directly to an untrusted network. See docs/DEPLOYMENT.md.
• Security findings are candidates, not confirmed exploits. They require engineering review. Scan coverage is partial and reported honestly per repo; absence of a finding is not proof of safety.
• The robust security scan is available via CLI, the HTTP API, and the web UI, but the rebuild-decision artifacts are not yet in the UI. Trigger the scan from the Scan control panel (/status.html → "Robust security scan") and view results (including the portfolio-wide vulnerable-package rollup) in the Robust security scan tab on /org.html. Underlying endpoints: POST /api/run/robust-scan, /api/run/portfolio-scan, /api/run/generate-report; reads GET /api/org/robust-scan-summary, /api/org/security-findings, /api/org/vuln-packages. The Chief Architect, business case, bottom-up estimate, disposition, Business Capability Map, portfolio meta-report, and fleet campaign are currently report/PDF and CLI only (UI pages are being added).
• AI features require an LLM provider and incur cost (see the cost note above). The tool can run without AI using the --skip-ai / --no-ai variants, with reduced output.
• Semgrep and a Chrome binary are optional but recommended. Without them the static-analysis layer is skipped and reports are HTML-only.
• API quota / rate limits. Large organizations consume significant GitHub API quota and LLM tokens; scans are resumable and support delta (--since) modes to manage this.

License

This project is released under the MIT License. See the LICENSE file for the full text.