Skip to content

PROJECT.md

Latest commit

 

History

History
42 lines (36 loc) · 3.13 KB

PROJECT.md

File metadata and controls

42 lines (36 loc) · 3.13 KB
Raw

Project: GitAtlas v0.1

Architecture

GitAtlas is a deterministic temporal knowledge graph engine for Git that converts repositories into a graph stored in Apache HugeGraph. The pipeline follows: Trigger -> Sync Engine -> Git Provider -> Git Events -> Graph Transformer -> Graph Events -> Validator -> Storage Adapter -> HugeGraph

It is structured into three main layers:

  1. Git Layer: Local command execution, repo scanning, and sync engine tracking.
  2. Graph Layer: Semantic translation and in-memory validation of invariants.
  3. Storage Layer: Idempotent mutations, Gremlin query builders, batch execution, and Docker Compose environment.

Milestones

# Name Scope Dependencies Status
E2E E2E Testing Track Design E2E framework, implement Tiers 1-4 tests, publish TEST_READY.md None IN_PROGRESS (Conv: 679d1c21-8314-4227-9a01-43398ca259a0)
M1 Graph Layer Correctness Fix validator (orphan parents, duplicate vertices), transformer (Repository/Branch nodes, remove DELETED, deterministic property sorting), Gremlin upserts None IN_PROGRESS (Conv: 9412b943-a245-4260-ab10-8e9437da161b)
M2 Git Scanner & Sync Engine Fix Git log cursor range query bug, scanner branches/repo, integrate scan pipeline in SyncEngine None IN_PROGRESS (Conv: e78ba057-7a1a-4d6d-8a56-9ac497c914ca)
M3 Batch Ingestion & Telemetry Implement batch mutation (500-1000 events), add telemetry, flags, and structured logging in CLI, update docker-compose.yml to 1.7.0 M1, M2 PLANNED
M4 Final Integration & E2E Pass 100% of E2E tests, run adversarial coverage hardening (Tier 5) E2E, M3 PLANNED

Interface Contracts

Git Layer ↔ Graph Layer

  • The Git Layer scans the history and emits GitEvent objects representing Git entities (Commits, Diffs, Branches).
  • The Graph Layer (Transformer) takes GitEvent structures and maps them to GraphEvent nodes and edges according to the MVP schema.
  • The SyncEngine manages the cursor (atlas_cursor) to sync only new commits.

Graph Layer ↔ Storage Layer

  • Validator validates in-memory sequence of GraphEvent structures to guarantee graph integrity before insertion.
  • StorageAdapter accepts batch mutations of validated GraphEvent objects and executes them via sorted Gremlin queries in Apache HugeGraph.

Code Layout

  • cmd/gitatlas-cli/main.go - Command-line interface entry point.
  • internal/git/ - Native git execution, branch scanning, repository scanning, and commit/diff parsing.
  • internal/sync/ - Orchestrates the ingestion sync, tracks cursor.
  • internal/events/ - Definitions of GitEvent and GraphEvent.
  • internal/schema/ - Vertex and Edge labels/properties definitions.
  • internal/transformer/ - Transformer logic mapping Git events to graph events.
  • internal/validator/ - Graph integrity validation (detecting orphans/duplicates).
  • internal/storage/ - Storage adapter interface.
  • internal/storage/hugegraph/ - Storage implementation with Gremlin builder and batch runner.
  • docker-compose.yml - Docker compose file for HugeGraph Server and Hubble.
  • tests/ - Directory for E2E tests.