Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
2c11dc7
feat: add SQLite-backed VexFS filesystem
vibeinging Jul 20, 2026
230a911
Add VexFS filesystem integration
vibeinging Jul 23, 2026
4f53577
Add chunked VexFS storage and archive v2
vibeinging Jul 23, 2026
8eb5a6c
Document preview 20 release verification
vibeinging Jul 23, 2026
9c684c5
Fix unpublished VexFS history isolation
vibeinging Jul 23, 2026
8a8b0c1
Filter deleted metadata from VexFS archives
vibeinging Jul 23, 2026
7520f24
Document preview 22 clean Mac verification
vibeinging Jul 23, 2026
3e6d72d
Add PostgreSQL VexFS runtime and validation
vibeinging Jul 24, 2026
d90a8fc
Document macOS NFS prototype findings
vibeinging Jul 24, 2026
3a97841
Add macOS NFS mount backend for VexFS
vibeinging Jul 24, 2026
f9d305b
Expand macOS NFS workspace eval coverage
vibeinging Jul 24, 2026
fd2419b
Support macOS 13 for the NFS mount path
vibeinging Jul 24, 2026
50329a9
Optimize SQLite VexFS staging overlays
vibeinging Jul 26, 2026
9bfe48a
Optimize PostgreSQL VexFS staged publishing
vibeinging Jul 27, 2026
bdc7d15
Complete PostgreSQL VexFS crash-safe publishing
vibeinging Jul 27, 2026
90b29ec
Fix VexFS PostgreSQL recovery boundaries
vibeinging Jul 27, 2026
776615a
Add VexFS find and safe workspace restore
vibeinging Jul 28, 2026
ad27b11
Optimize VexFS PostgreSQL batch history
vibeinging Jul 28, 2026
ab1ea42
Optimize PostgreSQL VexFS ACL history
vibeinging Jul 28, 2026
ee8a35c
Add unified VexFS workspace log
vibeinging Jul 28, 2026
2f4d525
Add VexFS snapshot retention policy
vibeinging Jul 28, 2026
a46064c
Optimize VexFS snapshot checkpoints and recovery
vibeinging Jul 28, 2026
0ab7cb0
Complete VexFS agent checkpoints and commit recovery
vibeinging Jul 28, 2026
4bc1293
Make macOS packages reproducible from clean source
vibeinging Jul 28, 2026
51079ee
Include SQLite shell in verified vendor sources
vibeinging Jul 28, 2026
037874a
Record preview 38 notarization gate
vibeinging Jul 28, 2026
be21801
Refresh VexFS release gate status
vibeinging Jul 28, 2026
21e15bf
Fix PostgreSQL mount idle and grep publishing
vibeinging Jul 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ m4/lt~obsolete.m4
build/
build-release/
build-release-vex/
/.cache/
duckdb_benchmark_data/
#*.m4
*.o
Expand Down Expand Up @@ -380,6 +381,9 @@ __pycache__/
*.egg-info/
*.whl

# Rust build artifacts
agent_files/mount/macos/nfs_gateway/target/

# DuckDB runtime / built extensions
*.duckdb
*.duckdb.wal
Expand All @@ -406,8 +410,17 @@ vexdb_duckdb/test/sql/vex/
scripts/env.local.sh
scripts/*.local.sh

# 内部文档目录(本地保留,不入仓库)
docs/
# 本地分析和生成报告不入仓库;当前 VexFS 产品基线必须随代码版本管理。
docs/*
!docs/specs/
docs/specs/*
!docs/specs/2026-07-16_vexfs-product-spec.md
!docs/plans/
docs/plans/*
!docs/plans/2026-07-21_vexfs-final-goal-and-roadmap.md
!docs/design/
docs/design/*
!docs/design/2026-07-16_agent-files-universal-filesystem.md
CHANGELOG.md

# SIFT benchmark data (large, generated from hdf5; not in repo)
Expand Down
28 changes: 27 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**English** | **[中文](README.zh.md)**

`VexDB-Lite` is a vector similarity search engine for PostgreSQL (`vexdb_lite` extension) and DuckDB (`vexdb_lite` extension). Both backends share the same graph index algorithm, SIMD distance dispatch, and quantization kernel.
`VexDB-Lite` extends PostgreSQL, DuckDB, and SQLite with a shared graph index, SIMD distance dispatch, and quantization kernel. The SQLite edition also includes VexFS, so one database can hold regular tables, vector indexes, and database-managed files.

> See [vexdb_duckdb/README.md](vexdb_duckdb/README.md) for the DuckDB extension docs.
> This root README is a project-level overview and build guide.
Expand Down Expand Up @@ -46,6 +46,32 @@ Current functionality:
- Vector buffer cache and parallel index build
- Runtime settings: `vexdb_ef_search`, `vexdb_brute_force_threshold`, `vexdb_pq_search_mode`, `vexdb_pq_refine_k_factor`

### 1.3 SQLite: unified VexDB-Lite

The unified macOS package installs:

- `vexdb`, with SQLite 3.45.3 and automatic extension registration;
- `GRAPH_INDEX` and vector distance functions;
- VexFS files, directories, history, restore, SHA-256, and read-only integrity checks;
- `vexdb fs ...`, including database-batched `grep` and an optional FTS5 trigram index, with `vexfs ...` kept as a compatible shortcut;
- a VexFS FSKit extension for native `ls`, `cat`, `grep`, `cp`, and `mv` after enablement.

```bash
vexdb agent.db
vexdb agent.db "SELECT vexdb_l2_distance('[1,2]', '[4,6]');"
vexdb fs --db agent.db write /notes/hello.txt
vexdb fs --db agent.db grep -n hello /notes
vexdb fs --db agent.db check
vexdb fs --db agent.db index enable
vexdb fs --db agent.db mount ~/VexDB
```

The text index is disabled by default, so ordinary writes have no index overhead. Without the
index, or when the SQLite host has no FTS5 support, `grep` falls back to a single-connection
database scan.

Build the unified macOS preview with `bash agent_files/macos/package_preview.sh`.

---

## 2. Capability Matrix
Expand Down
27 changes: 26 additions & 1 deletion README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**[English](README.md)** | **中文**

`VexDB-Lite` 是一个高性能向量检索系统,提供 PostgreSQL(`vexdb_lite` 扩展)和 DuckDB(`vexdb_lite` 扩展)两种适配形式,共享同一套 graph_index 图索引算法、SIMD 距离分发和量化器内核。
`VexDB-Lite` 是一个数据库能力扩展,提供 PostgreSQL、DuckDB 和 SQLite 三种适配形式,共享同一套 graph_index 图索引算法、SIMD 距离分发和量化器内核。SQLite 版还内置 VexFS 文件能力,让同一个数据库同时保存普通表、向量索引和文件

> DuckDB 扩展详见 [vexdb_duckdb/README.md](vexdb_duckdb/README.md)。
> 本根 README 只做项目级综述与构建总览。
Expand Down Expand Up @@ -45,6 +45,31 @@
- 向量缓存、并行建索引
- 运行参数:`vexdb_ef_search`、`vexdb_brute_force_threshold`、`vexdb_pq_search_mode`、`vexdb_pq_refine_k_factor`

### 1.3 SQLite:统一的 VexDB-Lite

macOS 统一包一次安装即可获得:

- `vexdb`:自带 SQLite 3.45.3,无需手动 `.load`;
- `GRAPH_INDEX` 和向量距离函数;
- VexFS 文件、目录、历史版本、恢复、SHA-256 和只读完整性检查;
- `vexdb fs ...` 文件命令,含数据库批量 `grep` 和可选 FTS5 trigram 索引,兼容入口为 `vexfs ...`;
- VexFS FSKit extension,启用后可用原生 `ls`、`cat`、`grep`、`cp`、`mv`。

```bash
vexdb agent.db
vexdb agent.db "SELECT vexdb_l2_distance('[1,2]', '[4,6]');"
vexdb fs --db agent.db write /notes/hello.txt
vexdb fs --db agent.db grep -n hello /notes
vexdb fs --db agent.db check
vexdb fs --db agent.db index enable
vexdb fs --db agent.db mount ~/VexDB
```

文本索引默认关闭,不增加普通写入开销;未开启索引或宿主没有 FTS5 时,`grep` 自动使用
单连接数据库扫描。

统一 macOS 技术预览包通过 `bash agent_files/macos/package_preview.sh` 构建。

---

## 2. 产品能力矩阵
Expand Down
59 changes: 59 additions & 0 deletions agent_files/cli/test/vexdb_unified_smoke.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/bash
set -euo pipefail

VEXDB_INPUT="${1:?usage: vexdb_unified_smoke.sh /path/to/vexdb}"
VEXDB="$(cd "$(dirname "$VEXDB_INPUT")" && pwd)/$(basename "$VEXDB_INPUT")"
TMP_DIR="$(mktemp -d -t vexdb-unified-smoke)"
DB="$TMP_DIR/agent.sqlite3"
COPY="$TMP_DIR/agent-copy.sqlite3"
BACKUP_LINK_TARGET="$TMP_DIR/backup-link-target"
BACKUP_LINK="$TMP_DIR/backup-link.sqlite3"
VEXFS_EXE="$TMP_DIR/vexfs.exe"
cleanup() {
rm -f "$DB" "$DB-wal" "$DB-shm" "$COPY" "$COPY-wal" "$COPY-shm" \
"$BACKUP_LINK" "$BACKUP_LINK_TARGET" "$VEXFS_EXE"
rmdir "$TMP_DIR" 2>/dev/null || true
}
trap cleanup EXIT

"$VEXDB" --version | grep -q 'vexdb-lite'
# Keep the executable next to its package-relative runtime libraries. The real
# installation also exposes vexfs as a symlink to vexdb; copying only the Mach-O
# binary would deliberately break @executable_path dependency resolution.
ln -s "$VEXDB" "$VEXFS_EXE"
"$VEXFS_EXE" --help | grep -q 'Commands:'

RESULT="$("$VEXDB" "$DB" \
"SELECT vexfs_contract_version(), vexdb_l2_distance('[1,2]','[4,6]');")"
[ "$RESULT" = "0.9.0|5.0" ]

"$VEXDB" "$DB" \
"CREATE VIRTUAL TABLE idx USING GRAPH_INDEX(embedding FLOAT[2], metric=l2); \
INSERT INTO idx(rowid, embedding) VALUES (1, '[1,1]'), (2, '[9,9]');" >/dev/null
[ "$("$VEXDB" "$DB" \
"SELECT rowid FROM idx WHERE embedding MATCH '[1,1]' AND k=1;")" = "1" ]

"$VEXDB" fs --db "$DB" --workspace smoke setup >/dev/null
printf 'one product' | "$VEXDB" fs --db "$DB" --workspace smoke write /note.txt >/dev/null
[ "$("$VEXDB" fs --db "$DB" --workspace smoke cat /note.txt)" = "one product" ]

"$VEXDB" backup "$DB" "$COPY" >/dev/null
[ "$("$VEXDB" "$COPY" "SELECT count(*) FROM idx;")" = "2" ]
[ "$("$VEXDB" fs --db "$COPY" --workspace smoke cat /note.txt)" = "one product" ]
COPY_MODE="$(stat -f '%Lp' "$COPY" 2>/dev/null || stat -c '%a' "$COPY")"
[ "$COPY_MODE" = "600" ]

if [ "$(uname -s)" != Windows_NT ]; then
printf 'must-not-change' > "$BACKUP_LINK_TARGET"
chmod 0644 "$BACKUP_LINK_TARGET"
ln -s "$BACKUP_LINK_TARGET" "$BACKUP_LINK"
if "$VEXDB" backup "$DB" "$BACKUP_LINK" >/dev/null 2>&1; then
echo "backup unexpectedly followed a destination symlink" >&2
exit 1
fi
[ "$(cat "$BACKUP_LINK_TARGET")" = "must-not-change" ]
TARGET_MODE="$(stat -f '%Lp' "$BACKUP_LINK_TARGET" 2>/dev/null || stat -c '%a' "$BACKUP_LINK_TARGET")"
[ "$TARGET_MODE" = "644" ]
fi

echo "VEXDB UNIFIED CLI SMOKE: PASS"
Loading