Roadmap for production-ready robotics simulation assets and IsaacSim/MuJoCo demos?

[Suibian-YY-pro]

Hi, thanks for releasing this project and the recent integration work around the single-image -> 3D asset pipeline.

I have been testing the latest fork at Yuchi-Zhang-00/sam3d_gs as of commit 927017e (provide a docker image), and also compared it with the upstream repository. The updated Docker image and checkpoint bootstrap are helpful for reproducibility.

However, I am trying to understand the intended robotics use case and current limitations. From my local tests, the pipeline can produce:

• SAM3-based object masks and background inpainting
• AnySplat background Gaussian reconstruction
• SAM-3D-Objects per-object .obj / .ply assets
• optional mesh -> MuJoCo MJCF conversion for individual objects

But I could not find an official example that is directly usable as a robotics simulation/training/deployment environment. In particular, I did not find:

• an IsaacSim / IsaacLab scene or task
• a MuJoCo task scene with robot, contacts, reset/step loop, observations, rewards, etc.
• a robot controller / IK / grasping or placement demo
• validation criteria for asset quality, scale, collision geometry, and object poses
• a full real-to-sim workflow from real images/video to a physically usable manipulation scene

I also tried converting one reconstructed scene into an IsaacSim USD scene locally. The result was useful as a visualization experiment, but not suitable for robot training: background Gaussians were noisy/fragmented, object meshes had unreliable geometry, contact relationships were not physically meaningful, and collision/articulation setup required substantial manual repair.

Could you clarify the project roadmap and intended scope? Specifically:

1. Is this repository intended only as a visual asset reconstruction pipeline, or is robotics simulation/training a planned target?
2. Are there plans to release an official IsaacSim, IsaacLab, MuJoCo, ManiSkill, or robosuite demo that can be used for manipulation training?
3. Is there a recommended workflow for turning the generated .obj / .ply / 3DGS outputs into physically valid robotics assets, including scale calibration, pose cleanup, collision decomposition, and material/texture handling?
4. Do you have examples where a real image or video is converted into a scene that supports robot grasping or placement with reliable contacts?
5. If the current release is only a preview/prototype, what components are expected to be open-sourced later for full real-to-sim use?

This clarification would be very helpful for users who want to evaluate whether the project can be used as a practical real-to-sim tool for embodied AI or robot manipulation, rather than only as a reconstruction demo.

Thanks again for the work!