[FEATURE] Decouple visual from physics geometry for FEM entities.#2904
Conversation
Introduce a `render_meshes` + `sim_vert_maps` model on FEMEntity so the visualizer can draw the user's original input surface (multi-sub-mesh GLBs supported) while the FEM solver retains its own tet-boundary topology for physics. Changes: - FEMEntity.sample() builds `_render_meshes` (list[gs.Mesh]) and `_sim_vert_maps` (list[np.ndarray]) via `gs.Mesh.from_morph_surface` and a new `_merge_elements` helper that dedupes vertices across sub-meshes and remaps element indices. - FEM solver drops the `surface_render_f` (triangle-indices) and `surface_render_uvs` (per-vertex UV) fields; rendering topology / UVs now live on FEMEntity's per-sub-mesh `Mesh` objects. - `get_state_render(f)` returns sim vertex positions only. - Rasterizer + raytracer iterate per-sub-mesh and key static nodes by `(env_idx, entity.uid, sub_idx)`. - `mesh_to_elements` signature change: accepts a trimesh directly, returns (verts, elems) only (UVs come from the input render mesh). - `gs.Mesh.from_morph_surface` returns a list for primitive morphs too — uniform return type makes consumers simpler. - `test_interior_tetrahedralized_vertex` updated to assert the new invariant: viz vertices = sim_verts[sim_vert_maps[0]].
…-world into pr/fem-render-meshes
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 063d6bc580
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- FEM `morph.pos` now applied as float64 in `instantiate`; gs.np_float (float32) quantizes 0.4/-0.1/0.1 etc. and perturbs downstream coords enough to push the macOS `linear_corotated` falling-sphere rest-state assertion over its tolerance. - `_merge_elements` no longer downcasts merged verts to gs.np_float; tetgen is sensitive to vertex coord precision and the downcast-then-upcast loses ULPs vs the upstream float64 path. - `split_all_surface_tets` is now applied only to file-loaded `Mesh` morphs (matching upstream); calling it for `Sphere`/`Box`/`Cylinder` primitives changed tet topology vs upstream. - Viewer's normal-pass `CustomShaderCache` strips the `DOUBLE_SIDED` define before compiling. Without the matching geometry shader, `mesh_normal.vert` emits `v_frag_*` while `mesh_normal.frag` expects `frag_*`, breaking shader linking on GUI normal captures of double-sided primitives.
The previous fix (commit 14e6d7c) only changed the float32→float64 dtype of `p` in `instantiate`, but left tetgen seeing the centered (un-translated) mesh. Tetgen's Steiner-point insertion is sensitive to absolute coords: - Centered sphere → 960 verts / 3726 elems - Sphere translated to (0.4, -0.1, 0.1) → 968 verts / 3784 elems ← upstream The mesh-count delta cascaded through the implicit FEM integrator on macOS and pushed `test_implicit_falling_sphere_box[SAP-linear_corotated-64]`'s 0.025 rest-state tolerance over by ~12% (90/2973 vertices violating). Apply pos to the merged surface verts in `sample` (before tetgen runs), and remove the duplicate translation from `instantiate`. The tet mesh now matches upstream's `sphere_to_elements` / `box_to_elements` output byte-for-byte.
# Conflicts: # genesis/engine/entities/fem_entity.py # genesis/vis/rasterizer_context.py # tests/rendering/test_offscreen.py # tests/utils.py
duburcqa
changed the title
|
Superseded by #3079. Successive review and CI rounds (per-visual-geom API aligned with the rigid entity, vertex welding pipeline, solver-side render state, normal-pass shader cache, snapshot and test rework) rewrote more than half of the original diff - less than half of the final added lines remain from this branch - so the consolidated version moves to a new PR with authorship reflecting that split. The original commits are preserved on this branch and the new commit credits you via Co-authored-by. |
|
Reopening: reassessed against the design rather than the diff line count. The consolidated version keeps this PR's architecture, welding algorithm, shaders and validation approach unchanged; the rewrites are repo conventions and API surface. Authorship stays here, and #3079 is closed. |
|
@codex review |
b733601 to
ea076ed
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b733601c24
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
duburcqa
changed the title
…d pixel matching.
dc730bf to
45dfcc1
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
duburcqa
changed the title
duburcqa
changed the title
duburcqa
changed the title
2 participants
Description
FEM entities now expose their morph sub-meshes as visual geoms (
fem_entity.vgeoms, mirroring the rigid entity API): eachFEMVisGeompairs a render mesh carrying its own surface and UVs with the map from its vertices to the simulated vertices. The rasterizer and raytracer draw the authored input surface per visual geom and update its vertices from the simulation state through that map, while the FEM solver drops its render-only topology and UV fields. Sub-meshes are welded into a single simulation mesh before tetrahedralization, which now runs before the morph position is applied so the tetrahedralization and its on-disk cache are shared across placements of the same geometry. The normal render pass supports double-sided primitives in both offscreen and interactive viewer renders through a shared shader cache keyed on the primitive defines.Motivation and Context
Multi-material assets lost their appearance when simulated as FEM entities: rendering used the tetrahedral boundary with a single entity-wide surface and zero-padded UVs. Decoupling render geometry from simulation geometry preserves the authored look (per-sub-mesh materials, textures, UVs) and enables sub-mesh level segmentation for FEM entities.
How Has This Been / Can This Be Tested?
tests/deformable/test_fem.py: the newtest_multi_submesh_render_decouplingasserts exact per-vertex tracking between visual geoms and simulated vertices on a two-material GLB;test_interior_tetrahedralized_vertexchecks the same invariant on a tetrahedralized box. The full file passes, including the implicit falling sphere/box rest-state checks.tests/rendering/test_offscreen.py: the newtest_multi_submesh_fem_rendersnapshot test covers rgb/depth/segmentation/normal on the rasterizer and rgb on the raytracer;test_segmentation_mapcovers the per-sub-mesh segmentation keys at "geom" level.Genesis-Intelligence/snapshotsupdated to revision11a81c48367a18aedf0a3bed460b69c7ba7ccd4e(5 new PNGs), pinned intests/utils.py.Screenshots (if appropriate):
Pair 1 - Ducks (Left Rigid, Right FEM)
Pair 2 - Multi-sub-mesh FEM rendering (trashbag_rope.glb)
Checklist:
Submitting Code Changessection of CONTRIBUTING document.