Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
95c0761
docs: standardize local data root naming
BorisGuo6 Aug 14, 2026
3af03f4
docs: record canonical data root migration
BorisGuo6 Aug 14, 2026
a41b6af
腔底角色拆分:包含检查与生成防楔入各用各的参数(新机全链路 fail=0)
huyuxinn Aug 14, 2026
057a1da
fix(web): 检索梯队规模去重 + objaverse LVIS 规模展示
wczmiracle Aug 14, 2026
0df4fb4
asset_ledger.v3 定稿:四路辩证 + 实验裁决驱动的契约升级(79 账本迁移零失败)
huyuxinn Aug 14, 2026
fb0e014
docs: v3 契约决策记录(四路辩证+E1/E2 实验裁决全文)
huyuxinn Aug 14, 2026
7af1b87
docs: v3 定稿成果页(三 tab 单文件,45 字段卡+44 对照行内嵌)
huyuxinn Aug 14, 2026
fd46b04
TV→沙发案四道防线 + 蓝衣架多材质着色崩溃修复
huyuxinn Aug 15, 2026
20ac887
docs: 账本字段手册(面向团队:逐字段含义+真实案例+各流程对齐指南)
huyuxinn Aug 15, 2026
c58d299
docs: 字段手册改版(各管线环节对齐指南)+ 过校验的 v3 标准 JSON 参考
huyuxinn Aug 15, 2026
37d47e2
尺寸方案 C:类目典型真实尺寸 + 桌面视图三档裁决(解除"万物 25cm")
huyuxinn Aug 15, 2026
8789f33
模型级颜色选型落生产:CatalogModel.colors 全链贯通
huyuxinn Aug 15, 2026
4212191
settle 修复(3/4 复活) + 许可回填(23->0 unknown)
huyuxinn Aug 15, 2026
c54fec7
Isaac 沉降 harness 首跑双 pass + 验证写回通道落地
huyuxinn Aug 15, 2026
8f79f14
docs: 六项授权队列批次执行报告(2026-08-15 夜批)
huyuxinn Aug 15, 2026
87f8253
Merge remote-tracking branch 'origin/main' into hyx
wczmiracle Aug 16, 2026
9b94596
路径卫生:账本可移植 uri 契约(public 仓不留个人路径,异机可解析)
huyuxinn Aug 16, 2026
7b926d3
Merge origin/main into hyx: 引入 USG quality reproduction study 四提交
wczmiracle Aug 18, 2026
233cccf
资产库落位改造 阶段0:路径读取收口到 iter_assets/asset_dir(平铺与分层双兼容)
wczmiracle Aug 18, 2026
57512bf
docs(acquire): 类目清单模板 + 字段说明(searched/pinned/local 三形态接入指引)
wczmiracle Aug 18, 2026
6657000
资产库落位改造 阶段1:66 个资产按来源分夹(nvidia 41 / objaverse 21 / github 4)
wczmiracle Aug 18, 2026
c579d4c
资产库落位改造 阶段3+5:新资产自动落来源夹 + 库 README
wczmiracle Aug 18, 2026
f680748
docs(asset_library): 再分片指引改为自包含(不再指向 work/oneoff 的临时脚本)
wczmiracle Aug 18, 2026
7706968
修两处既有故障:s12 验收门复活(5/5)+ 账本悬空 uri 归位(65/65 clean)
wczmiracle Aug 18, 2026
bbc09d8
refactor(configs): 按职责重组 —— 活配置/模板/夹具/留档/台账各归其位
wczmiracle Aug 18, 2026
b89ec9a
docs(acquire): 资产搜索使用指南(网页+清单四形态+结果解读+排障)
wczmiracle Aug 18, 2026
215d00e
docs(acquire): 模块定位表述更新 —— 检索引进为主叙事,生态互通为底层支撑
wczmiracle Aug 18, 2026
4961c78
docs(acquire): 使用指南补输出逻辑 —— evidence 结构/四种结局/过程产物/副作用链/退出码
wczmiracle Aug 18, 2026
106ae11
上游 RoboTwin 131 个资产入库:asset_library/robotwin/(owner 决策)
wczmiracle Aug 18, 2026
ff9c8e5
feat(acquire): 元数据对齐 —— materials直通账本 + VLM观察回填(attribute_basis) + 清单…
wczmiracle Aug 18, 2026
50e91a4
feat(acquire): 三档匹配输出 exact/similar/none —— 属性化复用+similar兜底+资产本体返回
wczmiracle Aug 19, 2026
292a107
docs(acquire): 使用指南与字段表补三档匹配说明(allow_similar)
wczmiracle Aug 19, 2026
ea620df
data(asset-pipeline): import measured upstream ledgers
wczmiracle Aug 20, 2026
ce7d37c
fix(ledger): 上游账本个人路径归零 —— 写入方回portable契约 + 90份数据修复
wczmiracle Aug 20, 2026
a0d45f1
feat(acquire): profile 感知的资产位分配 —— 跨来源同类另开新位(owner 决策 A)
wczmiracle Aug 20, 2026
995a778
docs(acquire): 输出逻辑段退出语义与三档实现对齐
wczmiracle Aug 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
19 changes: 19 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,22 @@ self_improving/asset_pipeline/active/data/scene_gen_ext/*
self_improving/alchedata/runs/*
!self_improving/alchedata/runs/probe_static_apple_plate_action_repair/
!self_improving/alchedata/runs/probe_static_apple_plate_action_repair/final_placement.json
# 账本是唯一权威入库记录,必须有 git 历史(spec §3.9;2026-08-14 事故中
# 17 份在 git 的账本是資產池唯一幸存的部分,例外规则随迁移带到本仓库)
!self_improving/asset_pipeline/active/data/asset_library/
self_improving/asset_pipeline/active/data/asset_library/*
!self_improving/asset_pipeline/active/data/asset_library/README.md
!self_improving/asset_pipeline/active/data/asset_library/*/
self_improving/asset_pipeline/active/data/asset_library/*/*
!self_improving/asset_pipeline/active/data/asset_library/*/ledger.json
# 库按来源分层后账本深了一级(<provider>/<asset>/ledger.json)。上面那组
# 规则只放行到 */ledger.json;不补下面三行,65 份账本会在搬迁的瞬间静默
# 掉出版本控制——而账本正是 2026-08-14 事故里唯一幸存的东西。
!self_improving/asset_pipeline/active/data/asset_library/*/*/
self_improving/asset_pipeline/active/data/asset_library/*/*/*
!self_improving/asset_pipeline/active/data/asset_library/*/*/ledger.json
!self_improving/asset_pipeline/active/data/upstream_ledgers/
self_improving/asset_pipeline/active/data/upstream_ledgers/*
!self_improving/asset_pipeline/active/data/upstream_ledgers/*/
self_improving/asset_pipeline/active/data/upstream_ledgers/*/*
!self_improving/asset_pipeline/active/data/upstream_ledgers/*/ledger.json
6 changes: 6 additions & 0 deletions scene_gen/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ class CatalogModel(StrictModel):
support_surface_z_offset_m: float | None = Field(default=None, ge=0.0)
support_margin_m: float = Field(default=0.005, ge=0.0)
support_spawn_clearance_m: float = Field(default=0.003, ge=0.0, le=0.02)
# Measured per-model colours (asset-pipeline appearance channel). Colour
# is a property of the MODEL, not the asset: 110_basket ships a yellow, a
# red and a white basket under one asset_id, and asset-level colours can
# only stay silent about all three. Empty = unmeasured (never a claim).
colors: tuple[str, ...] = ()
stable_pose_id: str | None = None
stable_orientation_wxyz: tuple[float, float, float, float] | None = None
z_policy: Literal["origin_on_table", "center_on_table"] = "origin_on_table"
Expand Down Expand Up @@ -480,6 +485,7 @@ def _scan_model(
),
support_margin_m=float(override.get("support_margin_m", 0.005)),
support_spawn_clearance_m=float(override.get("support_spawn_clearance_m", 0.003)),
colors=_string_tuple(override.get("colors")),
stable_pose_id=str(stable_pose_id) if stable_pose_id else None,
stable_orientation_wxyz=stable_orientation_wxyz if stable_pose_id else None,
z_policy=override.get("z_policy", "origin_on_table"),
Expand Down
10 changes: 9 additions & 1 deletion scene_gen/envs/generated_scene.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,15 @@ def _apply_color_override(actor: Any, color: str) -> int:
components = getattr(entity, "components", ())
for component in components:
for shape in getattr(component, "render_shapes", ()):
candidates = [getattr(shape, "material", None)]
# A multi-material triangle mesh RAISES on the aggregate
# .material getter (SAPIEN C++ side throws RuntimeError, so
# getattr's default never applies) -- a hanger with mixed
# materials killed the whole render on a colour-tint request
# (2026-08-15). Its materials are reachable via .parts below.
try:
candidates = [shape.material]
except (RuntimeError, AttributeError):
candidates = []
candidates.extend(
getattr(part, "material", None)
for part in getattr(shape, "parts", ())
Expand Down
21 changes: 21 additions & 0 deletions scene_gen/grounding.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,27 @@ def ground_object(query: SceneObjectSpec, catalog: AssetCatalog, *, seed: int) -
)
)
continue
if query.color:
if query.color in model.colors:
# a measured true-colour model outranks unknown-colour
# candidates in EVERY entry, so "a red cup" finds the
# actually-red cup instead of tinting the first cup
# (measured 2026-08-15: true hits 9/44 -> 23/44)
score += 4.0
reasons.append(f"model color matches {query.color}")
elif model.colors:
rejected.append(
RejectedCandidate(
asset_id=entry.asset_id,
model_id=model.model_id,
score=score,
reasons=(
f"model colors {list(model.colors)} lack "
f"{query.color}",
),
)
)
continue
if model.collision_path or model.urdf_path:
score += 2.0
reasons.append("collision representation available")
Expand Down
17 changes: 9 additions & 8 deletions self_improving/asset_pipeline/active/1_asset_reuse/OVERVIEW.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# OVERVIEW — 资产复用管线(1_asset_reuse)

> 一句话定位:让 RoboTwin 和 Isaac/USD 生态两边的资产互通,并且都能被 env-gen 的
> 「文字 → 场景 → 物理验证」全流程复用的一条资产管线。
> 一句话定位:面向不同仿真生态的资产检索与引进管线——文字需求/类目清单 →
> 多源信任梯度检索 → 验货质检 → 入库注册 → 被 env-gen「文字 → 场景 → 物理验证」
> 全流程真实选中。

## 1. 这是什么

Expand Down Expand Up @@ -30,7 +31,7 @@
输入(只读) 输出
├─ RoboTwin 资产库(134 物体,jingxiang 处) ├─ AssetBundle 账本 JSON(每资产一份,双后端表示+哈希+结构化未知)
├─ NVIDIA Isaac 资产服务器(HTTP,逐 prop 拉) ├─ (线A 产物 bottle.usd / cabinet.usd 已随线 A 归档,见 archive/)
├─ GitHub 公开仓库(tier2 检索源,逐仓声明许可) ├─ ../data/asset_library/(外部资产池,RoboTwin 布局
├─ GitHub 公开仓库(tier2 检索源,逐仓声明许可) ├─ ../data/asset_library/(外部资产池,按来源分夹
├─ openxsim IR 库(AssetBundle 数据结构) ├─ ../data/robotwin_shadow/(影子根)+ ../data/scene_gen_ext/(扩展 overrides+catalog)
└─ env-gen 上游(catalog/场景编译/验证器) ├─ 检索证据 selection_evidence / coverage_report / asset_gap_blocker
└─ 双后端验证报告 / e2e 场景包 / 回放视频截图
Expand Down Expand Up @@ -113,7 +114,7 @@ coacd 碰撞体)。`run_smoke.sh` 已改走批量管线。`s7_probe_reverse`

- **目标**:证明 USD 生态资产能反向进 RoboTwin 布局并过 SAPIEN 验证(单件打样)。
- **输入 → 输出**:NVIDIA 服务器 YCB 025_mug →
`../data/asset_library/301_cup/{visual,collision,model_data0.json}` + 账本。
`../data/asset_library/nvidia/301_cup/{visual,collision,model_data0.json}` + 账本。

| 文件 | 功能 |
|---|---|
Expand All @@ -129,14 +130,14 @@ coacd 碰撞体)。`run_smoke.sh` 已改走批量管线。`s7_probe_reverse`

- **目标**:一份 manifest 批量引入外部资产——一次 Kit 会话转全部,逐模型进程隔离验证,
未过门的带原因淘汰、物理隔离出资产池。
- **输入 → 输出**:`configs/external_manifest.json` → 资产池多模型目录 + 逐模型 bundle +
- **输入 → 输出**:`archive/external_manifest.json` → 资产池多模型目录 + 逐模型 bundle +
`import_matrix.json` + `external_overrides_fragment.yml`。

| 文件 | 功能 |
|---|---|
| `import_fetch_convert.py` | 阶段 1(isaac-smoke):镜像各组服务器目录(除 .thumbs,逐文件哈希)、读源 USD upAxis、单次 SimulationApp 会话全量转 GLB,写 `staging_manifest.json` |
| `import_materialize.py` | 阶段 2(env-gen-yuxin):GLB 规范化→物化→SAPIEN settle 硬门(位移<2mm、z>-2mm、倾角<15°/平躺物 45°)→账本+矩阵+fragment;惯例继承与尺寸策略在此落账 |
| `configs/external_manifest.json` | 人工清单:组(服务器 prefix)+ 逐条 usd/asset/model/category/aliases/colors/footprint,可选 `collision: coacd`(离线精分解)、`size_policy` |
| `archive/external_manifest.json` | 人工清单:组(服务器 prefix)+ 逐条 usd/asset/model/category/aliases/colors/footprint,可选 `collision: coacd`(离线精分解)、`size_policy` |

**参数**:

Expand Down Expand Up @@ -204,7 +205,7 @@ s10/s12 无参数(路径写死在脚本头部变量)。**一键运行**:`r
- **目标**:类目清单或场景 prompt 驱动,按信任梯度自动找资产、过门禁、复用批量管线
引进,全程留决策证据;查缺补漏而非静默失败。
- **输入 → 输出**:`acquire_categories.json` 或 prompt → 入库资产 +
`configs/acquired_manifest.json`(自动生成清单)+ `selection_evidence.json` /
`data/acquired_manifest.json`(自动生成清单)+ `selection_evidence.json` /
`coverage_report.json` / `asset_gap_blocker.json`。

| 文件 | 功能 |
Expand All @@ -217,7 +218,7 @@ s10/s12 无参数(路径写死在脚本头部变量)。**一键运行**:`r
| `scripts/1_search/scene_acquire.py` | 场景驱动自适应:prompt→覆盖检查→缺口自动引进→重查→生成场景;仍未覆盖的写 `asset_gap_blocker.json`(生成式兜底的结构化输入) |
| `configs/providers.json` | provider 开关/层级/源配置 + 全局 `top_k`/`max_fallback`/`max_size_bytes`/`license_gate` |
| `configs/acquire_categories.json` | 类目需求清单(category + aliases) |
| `configs/acquired_manifest.json` | acquire_batch 自动生成的已引进清单(与手写 external_manifest 同构) |
| `data/acquired_manifest.json` | acquire_batch 自动生成的已引进清单(与手写 external_manifest 同构) |
| `tests/`(7 文件 + fixtures) | 检索层单测:providers / selection / webfetch / coverage / acquire_batch / scene_acquire / 分层搜索(`fixtures/mini_catalog.json`) |

**参数**:acquire_batch `--categories --providers --dev-root --out`,`--refresh-index`
Expand Down
22 changes: 14 additions & 8 deletions self_improving/asset_pipeline/active/1_asset_reuse/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,38 @@
# 1_asset_reuse — 资产复用(阶段 1)

让 RoboTwin 和 Isaac/USD 生态两边的资产互通、并都能被 env-gen 全流程复用。
三条线:**A** RoboTwin→Isaac USD;**B** 外部 USD→RoboTwin 布局;**C** 注册进扩展
catalog 被文字场景真实选中。全景与逐文件说明见本目录 `OVERVIEW.md`。
**面向不同仿真生态的资产检索与引进管线**:把「要什么物体」(文字 prompt 或类目清单)
变成「可直接用于场景生成的资产」——按信任梯度检索 本地库 → NVIDIA → GitHub → Objaverse,
AI 看图验货 + 物理质检合格后收进 RoboTwin 布局资产库、注册进 env-gen 扩展 catalog,
被文字场景真实选中。底层依托 RoboTwin(SAPIEN) 与 Isaac/USD 两生态的资产互通能力
(线 B 反向引入 · 线 C 全流程复用 · 线 D 检索选定;线 A 正向转换已归档)。
全景与逐文件说明见本目录 `OVERVIEW.md`,操作指南见 `docs/asset_search_usage.md`。

## 结构

| 路径 | 作用 |
|---|---|
| `scripts/` | 步骤脚本按流程顺序编号分夹:1_search/ 2_convert/ 3_materialize/ 4_validate/ 5_catalog/ + 横切 ledger/ probe/(入口 run_smoke.sh 在根,地图见 scripts/README.md) |
| `archive/` | 已归档不再维护的代码(1_forward_convert/ = 线 A 正向转换,2026-08-10 归档) |
| `configs/external_manifest.json` | 外部资产批量清单(来源 URL、资产归组、类别/别名/footprint) |
| `../data/asset_library/` | 外部资产池(RoboTwin 布局;`_source/` 存源镜像+哈希清单;不入 git) |
| `archive/external_manifest.json` | 外部资产批量清单(来源 URL、资产归组、类别/别名/footprint) |
| `../data/asset_library/` | 外部资产池(RoboTwin 布局,**按来源分一级**:`nvidia/ objaverse/ github/`;`_source/` 存源镜像+哈希清单,不按来源分。只有 `ledger.json` 入 git,见该目录 README) |
| `../data/robotwin_shadow/` | 影子根(symlink 真 RoboTwin + 注入外部资产;不入 git) |
| `../data/scene_gen_ext/` | 扩展 overrides + catalog(上游扫描器产出;不入 git) |


## 用法

> 资产搜索/引进的完整操作指南(网页 + 命令行清单 + 模板 + 结果解读)→ `docs/asset_search_usage.md`

```bash
export OMNI_KIT_ACCEPT_EULA=YES
export SAPIEN_PYTHON=/path/to/env-gen-python
export ISAAC_PYTHON=/path/to/isaac-sim-python
export ROBOTWIN_ROOT=/path/to/RoboTwin
# 打样回归(B+C 全链路,4 步)—— 全程隔离,产物全落 results/_test/,生产 data/ 只读不写
bash scripts/run_smoke.sh
# 批量导入外部资产(按 configs/external_manifest.json)
# 批量导入外部资产(按 archive/external_manifest.json)
"$ISAAC_PYTHON" scripts/2_convert/import_fetch_convert.py \
--manifest configs/external_manifest.json \
--manifest archive/external_manifest.json \
--source-root ../data/asset_library/_source --staging <staging目录>
"$SAPIEN_PYTHON" scripts/3_materialize/import_materialize.py \
--staging <staging目录> --library-dir ../data/asset_library \
Expand All @@ -49,7 +55,7 @@ bash scripts/5_catalog/s10_e2e_scene.sh

### 账本契约(asset_ledger.v1)

- **权威位置**:`../data/asset_library/<asset>/ledger.json`(每资产一份,含
- **权威位置**:`../data/asset_library/<来源>/<asset>/ledger.json`(每资产一份,含
`models[]`;随 git 提交,是唯一权威入库记录——`results/**/bundles/` 只是它的摊平快照产物)。
- **现存资产升级 v1**(一次性、幂等;已是 v1 的资产自动跳过):
```bash
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,40 @@
# configs/ — 配置与清单
# configs/ — 活配置(代码运行时真读的)

| 文件 | 谁产 | 谁消费 |
| 文件 | 谁写 | 谁读 | 手改 |
|---|---|---|---|
| `providers.json` | 人 | `acquire_batch`/`scene_acquire`/web(检索源 tier0–4 + top_k/门禁开关) | ✅ 这是主要调参入口 |
| `category_sizes.yml` | 人 | a4 需求解析(size_decision)、物化缩放、rescale_backfill、s9 | ✅ 新类别补一行典型尺寸 |
| `templates/` | 人(照抄起点) | 不被代码读 | ✅ 复制出去改 |

## templates/ — 引进请求清单模板

要资产时照着填的表(喂给 `acquire_batch --categories`;管线内部称 categories 清单,
run 目录里机器写的 `acquire_categories.json` 即同一 schema 的运行痕迹):

- `acquire_request.template.json` — 全形态参考:searched(自动检索)/ pinned(钉死
NVIDIA 文件)/ local(验收本地文件)四条带注释
- `acquire_request.example.json` — 最简可跑:2 条 searched,复制改类别即可用

字段说明见本文件末表;执行方式见 `../README.md`。

## 已迁出(2026-08-18 重组)

| 原住户 | 现址 | 原因 |
|---|---|---|
| `acquired_manifest.json` | `../../data/` | 机器写的引进台账(编号+查重依据),须与 asset_library 同区同进退,勿手改 |
| `smoke_manifest.json` | `../tests/fixtures/` | run_smoke.sh 常驻回归的输入夹具 |
| `external_manifest.json` | `../archive/` | 首批 NVIDIA 批量导入的历史记录,只留档 |

## 清单字段速查

| 字段 | 必填 | 作用 |
|---|---|---|
| `external_manifest.json` | 人工手写 | `scripts/2_convert/import_fetch_convert.py`:批量导入清单——组(服务器 prefix)+ 逐条 usd/asset/model/category/aliases/colors/footprint,可选 `collision: coacd`、`size_policy` |
| `providers.json` | 人工手写 | `scripts/1_search/` + `lib/a1`:检索 provider 开关/层级/白名单 + 全局门禁(`top_k`/`max_fallback`/`max_size_bytes`/`license_gate`) |
| `smoke_manifest.json` | 人工手写 | `scripts/run_smoke.sh`:冒烟回归的输入——单件 YCB 025_mug → 301_cup(带 coacd),与 `external_manifest.json` 同名条目保持一致 |
| `acquire_categories.json` | 人工手写 | `scripts/1_search/acquire_batch.py`:类目需求清单(category + aliases) |
| `acquired_manifest.json` | **acquire_batch 自动追加** | 已引进资产台账,与 `external_manifest` 同构;**勿手改** |
| `category` | ✅ | 要什么类别(库里已有同类会复用不重复引进) |
| `aliases` | | 同义词:扩大检索面 + VLM 名称核对 |
| `colors` / `materials` | | 进检索词 + VLM 看图验货(不符淘汰候选) |
| `size_policy` | | `match_category`(默认) / `absolute:<米>` / `none` |
| `collision` / `reorient` / `flat` | | 物化旋钮:凸分解 / 静置定姿 / 放宽直立门 |
| `pinned: {prefix, usd}` | | 钉死 NVIDIA 服务器具体文件,跳过检索 |
| `local: {path}` | | 验收本地文件入库,跳过检索下载 |
| `allow_similar` | | 默认 `true`:exact 落空时允许返回/引进相似资产(similar 档);`false`=严格模式,宁可 none |
| `comment` | | 仅注释,代码忽略 |
Loading