Summary
Add a new non-LLM plugin to let Synthetic Heart interact with the scene camera, manage webcam state, and capture images from the virtual scene. This includes support for selfies, screenshots, photo sending, and a selfie-oriented animation pose state.
Motivation
Synth already has a digital body and animation system. Enabling real webcam/camera interactions closes the gap between virtual persona and scene-aware media actions, allowing:
- turning the webcam on/off
- capturing photos or screenshots
- sending image attachments
- triggering selfie poses before capture
Proposed Changes
-
New webcam/camera plugin
- Create
plugins/webcam_plugin.py (or similar)
- Implement
get_supported_actions() for actions like:
enable_webcam
disable_webcam
take_screenshot
take_selfie
send_photo
set_camera_pose
-
Session metadata persistence
- Reuse session_meta.py for camera state storage
- Persist webcam on/off state, current camera rect, and scene focus per
interface_path
- Ensure archive endpoints include camera metadata on save
-
Animation support
- Extend animation_handler.py to support selfie/camera pose states
- Register a new logical animation state or alias for selfie pose
- Keep state-driven animation selection compatible with the existing handler
-
Action validation
- Ensure the new actions are discovered by the auto-registration process in component_auto_registration.py
- Add required field definitions for validation
-
WebUI / interface integration
- Wire new camera metadata into WebUI session save/load flows
- Optionally expose camera commands in the UI once backend support exists
Acceptance Criteria
Notes
- This PR should remain focused on backend/plugin integration and metadata plumbing.
- Full WebUI controls for webcam toggling and photo capture can be scoped to follow-up work.
- Selfie pose support should use logical animation states, not hardcoded file paths.
Relevant files
- session_meta.py
- webui.py
- animation_handler.py
- component_auto_registration.py
- plugin_base.py
- plugins (new module)
Summary
Add a new non-LLM plugin to let Synthetic Heart interact with the scene camera, manage webcam state, and capture images from the virtual scene. This includes support for selfies, screenshots, photo sending, and a selfie-oriented animation pose state.
Motivation
Synth already has a digital body and animation system. Enabling real webcam/camera interactions closes the gap between virtual persona and scene-aware media actions, allowing:
Proposed Changes
New webcam/camera plugin
plugins/webcam_plugin.py(or similar)get_supported_actions()for actions like:enable_webcamdisable_webcamtake_screenshottake_selfiesend_photoset_camera_poseSession metadata persistence
interface_pathAnimation support
Action validation
WebUI / interface integration
Acceptance Criteria
get_supported_actions()Notes
Relevant files