Enable mouse/pen aim + touch click play style

[Susko3]

• Alternative to / closes Fix pen input not being friendly with touch osu-framework#6509
• Closes Add basic test coverage for pen & touch playstyle #31680
• Fixes Cannot play osu! with pen + touch input #31570

This PR enables a playstyle where a hovering mouse or pen is used for position, and finger touches are used for clicking (as a keyboard). This is enabled by a new flag (trackPositionOfIndirectTouches), which will prevent touches from "stealing" the cursor position. The flag is dynamic: if a user directly touches a hitcircle with their finger, the game will assume the previous playstyle of aiming with a finger and using fingers for clicking. But if a user then hovers their pen or mouse, the game will instantly switch to the playstyle described in this PR.

The implementation is not perfect (see TestFirstTouchBrokenWhenMousePressed), but it's possible to work around some of the issues (see TestFirstTouchWorksWhenMousePressedAndMouseButtonsDisabled). It's recommended that users who wish to aim with the pen pressing down on the tablet surface, and don't want to use the pen for clicking, disable "mouse clicks during gameplay". (This is currently under the "Mouse" section, as there is no "Pen" section in input settings.)

The changes are scoped to OsuTouchInputMapper, so it only affect touch input (i.e. mouse/keyboard/tablet input will not regress).

Testing

A previous version of this branch was tested by frenzi and Walavouchey.

As seen in the Walavouchey replay, a common user "error" is accidentally directly tapping on a circle. Here, a hitcircle happened to appear exactly under where the user is trying to tap for a click, stealing the position from the pen, resulting in a sliderbreak.

This is not a new issue, as the same thing can happen when using touch instead of a pen for aiming.

A user can fix this by not tapping on the playfield, maybe best accomplished by resizing or moving the playfield using Screen scaling.

The following logic could be updated (in a separate PR) to fix this (perhaps it should consider notelock):

osu/osu.Game.Rulesets.Osu/OsuInputManager.cs

Lines 45 to 50 in ff0c0d5

	public bool CheckScreenSpaceActionPressJudgeable(Vector2 screenSpacePosition) =>
	// This is a very naive but simple approach.
	//
	// Based on user feedback of more nuanced scenarios (where touch doesn't behave as expected),
	// this can be expanded to a more complex implementation, but I'd still want to keep it as simple as we can.
	NonPositionalInputQueue.OfType<DrawableHitCircle.HitReceptor>().Any(c => c.CanBeHit() && c.ReceivePositionalInputAt(screenSpacePosition));

[frenzibyte]

Seems fine conceptually. I still would've preferred a rewrite of OsuTouchInputMapper that explicitly recognizes pen input as a "pointer" so as to avoid these extra set of conditionals but there are blockers in the way that require reassessment of how the o!f input subsystem works and I'm pretty much full with all of this stuff already. @peppy requesting second-hand code review on this.

[frenzibyte]

I would probably rewrite this to actually test expectations and put an ignore marker on the test itself. Feels wrong to assert wrong behaviour.

[Susko3]

I'm not sure what's the correct behaviour here. Maybe I was too harsh in naming the test "broken". This behaviour is correct in the sense that it closely matches what would happen if you pressed your pen and clicked the keyboard X. With touch input, it's a necessity to release the tracked touch, but pen input supports hover, the user explicitly chose to click with the pen—they should disable pen clicks if they don't want them.

Hmm, maybe tapping with the pen pressed down should press the OsuAction.RightButton without releasing the left. This would emulate clicking with the C key.

[Susko3]

Something like this—pressing the pen down will completely prevent touches from triggering the left button:

diff --git a/osu.Game.Rulesets.Osu/UI/OsuTouchInputMapper.cs b/osu.Game.Rulesets.Osu/UI/OsuTouchInputMapper.cs
index d6eec41584..d2087c9f04 100644
--- a/osu.Game.Rulesets.Osu/UI/OsuTouchInputMapper.cs
+++ b/osu.Game.Rulesets.Osu/UI/OsuTouchInputMapper.cs
@@ -80,12 +80,12 @@ protected override void OnTouchMove(TouchMoveEvent e)
 
         protected override bool OnTouchDown(TouchDownEvent e)
         {
-            OsuAction action = trackedTouches.Any(t => t.Action == OsuAction.LeftButton)
+            OsuAction action = trackedTouches.Any(t => t.Action == OsuAction.LeftButton) || osuInputManager.PressedActions.Contains(OsuAction.LeftButton)
                 ? OsuAction.RightButton
                 : OsuAction.LeftButton;
 
             // Ignore any taps which trigger an action which is already handled. But track them for potential positional input in the future.
-            bool shouldResultInAction = osuInputManager.AllowGameplayInputs && !tapsDisabled.Value && trackedTouches.All(t => t.Action != action);
+            bool shouldResultInAction = osuInputManager.AllowGameplayInputs && !tapsDisabled.Value && trackedTouches.All(t => t.Action != action) && !osuInputManager.PressedActions.Contains(action);
 
             // If we can actually accept as an action, check whether this tap was on a circle's receptor.
             // This case gets special handling to allow for empty-space stream tapping.

[frenzibyte]

That diff further leads to the idea that OsuTouchInputMapper is purposefully not wanting to be aware about the pen's state, and instead substitutes that with OsuInputManager.PressedActions. I don't think this is good, having a set of flags alongside checking PressedActions besides trackedTouches all to support pen input smells like the class is in need for a top-down refactor. I would keep the behaviour broken, update the test to match the expected behaviour achieved by the diff above, and mark it ignored.

[Walavouchey]

this branch + latest framework master logs replay

1. pen dragging and touch tapping work just as before
2. hovering while touch tapping doesn't work (i switch to this ~25s into the map and fail). cursor position doesn't update in this state, even after lifting the pen completely and continuing to tap the screen video
3. in general, hovering moves the cursor relatively (like a mouse) while dragging moves it absolutely (like a tablet should be) video

edit: latest framework master (with ppy/osu-framework#6511) + this pr works pretty well again

[peppy]

i can't make much sense of this sorry. it's adding one too many layers to OsuTouchInputManager.