Skip to content

devpost.md

Latest commit

 

History

History
51 lines (33 loc) · 6.36 KB

devpost.md

File metadata and controls

51 lines (33 loc) · 6.36 KB

npm Supply Chain Risk Analyzer

Inspiration

The npm ecosystem ships over 2 million packages, and a single compromised dependency can cascade through thousands of downstream projects. We saw this play out in real incidents — event-stream (2018), ua-parser-js (2021), colors.js (2022), node-ipc (2022) — where one package brought down entire ecosystems. Existing tools like npm audit tell you what's vulnerable, but not how bad it would be if that vulnerability were exploited. We wanted to answer the question: if this package gets compromised right now, how many people get hurt?

What it does

The Supply Chain Risk Analyzer is a 3D interactive visualization of the npm ecosystem's single points of failure. It has four modes:

  • Risk Heatmap — 1,087 npm packages visualized as a 3D force-directed graph, color-coded by blast radius (dependents × downloads). The leaderboard ranks the ecosystem's most dangerous chokepoints.
  • Attack Replays — Curated replays of four real supply chain attacks (event-stream, ua-parser-js, colors.js, node-ipc). Historical dependency edges are injected into the graph so you can watch the actual cascade as it happened.
  • Package Scanner — Paste any package.json (or try Express, PM2, Fastify presets) and cross-reference your dependencies against 217,000+ vulnerability records from OSV.dev. Vulnerable deps glow red, safe ones glow green.
  • Multi-Agent Investigation — Click "Investigate with Agents" on any package and five autonomous Jac walkers deploy simultaneously. Each uses by llm(tools=[...]) with Claude Haiku to call graph-querying tool functions, reason about findings, and report back. A synthesis agent combines all four specialist findings into a strategic threat assessment.

How we built it

Backend (Jac): The entire backend is written in Jac using Object-Spatial Programming. Package nodes and DependsOn edges model the dependency graph. Walkers traverse the graph for blast radius computation, infection cascading, and risk scoring. The investigation agents use Jac's by llm(tools=[...]) syntax — each agent is a def:pub endpoint backed by an LLM function that gets Jac tool functions (like lookup_vulns, compute_blast_radius, get_dependents) passed as callable tools. Claude Haiku decides which tools to call, interprets results, and produces structured findings. The sem directive provides semantic prompts for each agent.

Data Pipeline (Python): Scripts fetch real dependency graphs from the deps.dev REST API (200+ seed packages, 1,087 discovered), download 217K+ npm vulnerability records from OSV.dev, compute risk scores (blast_radius = dependent_count × log10(downloads)), and export an enriched JSON file.

Frontend: A single HTML file using Three.js and 3d-force-graph. No React, no build system. The 3D graph renders 1,087 nodes with risk-based coloring, cascade animations, and a multi-agent investigation panel showing agents working in parallel with a typewriter effect.

Deployment: Jac backend on Railway, static frontend on Vercel, auto-deploys from GitHub.

Challenges we ran into

  • The historical data problem. The current npm dependency graph doesn't show connections that existed during past attacks — the ecosystem already moved on. We solved this by injecting historical dependency edges at replay time, so the cascade actually flows through the graph.
  • Jac cache persistence. The Jac runtime caches compiled .jir files aggressively. During development, stale caches caused the server to serve old endpoints even after code changes. We had to learn the cache invalidation behavior and nuke it properly between restarts.
  • Graph loading performance. The build_graph function with 1,087 packages took 29 seconds due to O(n²) node lookups. We solved this by making the backend sync non-blocking — the frontend renders the graph client-side immediately and syncs to the backend in the background.
  • LLM tool calling in Jac. Getting by llm(tools=[...]) to work required understanding how Jac's byllm module constructs prompts, passes function schemas, and handles the ReAct loop. The tool functions need to return strings (not structured data) for the LLM to interpret.

Accomplishments that we're proud of

  • Five autonomous agents running in parallel — each with its own tools, reasoning, and findings. The synthesis agent takes all four reports and produces a strategic assessment. This isn't a chatbot — it's real agentic AI with tool use.
  • 217,000 real vulnerability records cross-referenced against 1,087 packages. When you scan Express and see 9 vulnerable dependencies with 23 CVEs, those are real CVEs from OSV.dev.
  • The cascade animation — watching an infection propagate through the 3D graph in real-time, depth by depth, with red lines spreading across the ecosystem. It makes the abstract concept of supply chain risk viscerally tangible.
  • Historical attack replays that actually work — injecting edges that no longer exist in the current graph so you can see what colors.js looked like when it took down AWS CDK.

What we learned

  • Jac's Object-Spatial Programming maps naturally to dependency graph analysis. Nodes, edges, and walkers are first-class concepts — we didn't have to build a graph abstraction on top of a language that doesn't have one.
  • by llm(tools=[...]) is genuinely powerful. Defining a Jac function as a tool and letting the LLM autonomously decide when to call it feels like the right abstraction for agentic systems — cleaner than manually orchestrating API calls.
  • Supply chain security is scarier than we thought. When you see that compromising debug (a logging utility) would cascade to 93 packages representing 3.2 billion weekly downloads, the scale of the problem becomes real.

What's next for npm Supply Chain Risk Analyzer

  • BigQuery integration — Replace the REST API fallback with a full deps.dev BigQuery export (5,000+ packages with real dependent counts instead of 1,087).
  • Real-time CVE monitoring — Poll OSV for new vulnerabilities and automatically update risk scores.
  • Reachability analysis — Use the LLM agents to check whether a dependent actually calls the vulnerable function, not just depends on the package. This would cut false positives dramatically.
  • GitHub App — Scan repositories automatically on PR and comment with the blast radius of any vulnerable dependency changes.