Input System Touch Controls

[Dhruv-0-Arora]

Description

Creating touch controls joysticks for mobile devices. To enable the touch controls, you must be on a device with touch capabilities. When this happens, the Enable touch controls button will appear in the MainHUD.

After enabling the touch controls the controls themselves will appear on screen.

To use them, you must configure a inputscheme or use the preset "Brandon"

Notes

• In the future we will create a queue of the different actions on the top of the screen that will appear as they are available to execute. For example, intake when there is an object in the intakesensor, eject when needed, and place assembly. These will be based off the design of the place assembly button.
• When configuring the inputscheme, the useGamepad and useTouchControls checkboxes can be selected at the same time. This is a bug. To fix this, we will need to change the implementation of the Checkbox to utilize the checked={} prop instead of default value.
• When creating a new configuration and enabling touch controls, the different joint configurations appear (which causes a bug of joints moving as the robot moves). To fix this, we should create a "add joint configuration" button when creating a new configuration.
• In the future, on the top left of the screen there will be sliders for joints similar to the queue that will appear only if there is a configuration for a joint.

Testing

To test this on a mobile device:

1. Enable SSL by going to vite.config.js and enabling the boolean to be true
2. run bun run host or npm run host to expose it to the network
3. Open the URL (not the localhost url) on your mobile device that is connected to the same network.

Note: Ensure you are not on a wifi network with a firewall that blocks external access (such as the Autodesk network). Alternatively, you can use a hotspot.

JIRA Issue

[LucaHaverty]

What if instead of getValue() taking in useTouchControls and useGamepad, the input scheme stored an enum InputType, replacing the boolean it currently has called useGamepad? That would avoid different input type conditions originating from different places and potentially overriding each other.

export enum InputType {
    KEYBOARD,
    GAMEPAD,
    TOUCH
 }

export type InputScheme = {
    schemeName: string
    descriptiveName: string
    customized: boolean
    inputType: InputType
    inputs: Input[]
}

[LucaHaverty]

Every input should be between -1 and 1 for consistency

[LucaHaverty]

This is looking good, but I have a couple requests and recommendations:

1. If you aren't on a device that supports touch controls, the touch control scheme should not be selectable because there's not way to control it. If we keep a similar button, consider making it more opaque and blurring the background.
   

2. Instead of a button to open touch controls, they should automatically appear if there's a robot spawned that uses them.

3. The place assembly button text is really hard to read with some backgrounds. It also looks a little out of place. I think this is because the colors and style are so different than the others buttons. Maybe it could appear on the panel that that loading bar is on once it finishes loading?

4. When I connect to the network version, I get the error Uncaught TypeError: can't access property "getDirectory", navigator.storage is undefined MirabufLoader.ts:29. Is this just me?

[AlexD717]

There are a lot of scaling issues on a phone, but I am assuming those either don't happen on a tablet/iPad or will be fixed later

[Dhruv-0-Arora]

There are a lot of scaling issues on a phone, but I am assuming those either don't happen on a tablet/iPad or will be fixed later

The scaling issues with be fixed with the UI refactor

[AlexD717]

I get an error when I try to spawn a robot after spawning a field. The error message is: Place assembly before spawning another (on desktop).

[Dhruv-0-Arora]

I get an error when I try to spawn a robot after spawning a field. The error message is: Place assembly before spawning another (on desktop).

Must be a merging error; I'll look into that

[Dhruv-0-Arora]

Note: with the addition of the usesTouchControls, prettier will autoformat the axisInputs over multiple lines rather than condensing them.

[BrandonPacewic]

Now why does prettier want to do this? Who knows 🤷

[ariesninjadev]

The code looks good.

As far as functionality goes, it was great. If I were to suggest anything, I'd say I think touch controls should be enabled by default on small screens, but not needed.

Also, I had trouble actually testing the functionality of the joysticks because seemingly the resolution of my Pixel 7 is too low and the config GUI was getting cut off (I couldn't close it). Ended up using a sim and it seemed fine.

To support more mobile devices, we may consider adding a fullscreen button, and UI scaling options.

No longer a team member

[BrandonPacewic]

Think at this point these changes are good to go in. There is still lots of problems with getting everything working on mobile (particularly when it comes to smaller screens) however at this point the necessary changes are out of scope.

Few things to note for mobile updates that are needed:

• Gamepad/Tough Control Checkbox Conflict AARD-1945.
• Prevent Joint Auto-Configuration on New Touch Control Setup AARD-1946.
• Implement Tough Control Top Auction UI AARD-1947.
• Implement Tough Control Joint Slider UI AARD-1948.
• General Mobile Device UI Updates (for small screen scaling issues +extra) AARD-1950.

That being said, these issues already existed and this PR is one more step to getting everything working on mobile. Glad to get the last major PR from last year done. 📵