Having to type takes you out of the flow, especially if you're using gamepad input. Optional feature: use voice input to dictate what your edit should be. The VLM should be able to clean up voice ramblings.
Unfortunately, Electron does not ship with speech recognition, so we'd have to use our own solution. There are quite a few models for these nowadays, but it is another thing we'd have to load. We'd also have to stream the voice input to the server, and then back to the client, so there's a fair bit of work involved here in making this work well.
Open questions:
- How do you indicate you want a voice input? Is it always on when the feature is on?
- What's the best way to indicate that the edit is "done" and ready to go? We could use voice activity detection, or an explicit input (e.g. the same button as was used to trigger Scene Edit, but this would lock us into always-input)
Having to type takes you out of the flow, especially if you're using gamepad input. Optional feature: use voice input to dictate what your edit should be. The VLM should be able to clean up voice ramblings.
Unfortunately, Electron does not ship with speech recognition, so we'd have to use our own solution. There are quite a few models for these nowadays, but it is another thing we'd have to load. We'd also have to stream the voice input to the server, and then back to the client, so there's a fair bit of work involved here in making this work well.
Open questions: