v3.0.91 breaks ALT+key on wayvnc

[TobiaszCudnik]

v3.0.91 breaks ALT+key on wayvnc.

Not sure whos fault is that, but thats what Im getting on alt+g or alt+5:

ERROR: ../wayvnc/src/keyboard.c: 400: Failed to look up keyboard symbol: infinity

Works well in v3.0.3.

wayvnc https://github.com/any1/wayvnc

Thanks

[dcommander]

Which client platform are you using?

[TobiaszCudnik]

Sorry, forgot to mention, its macos with a default en_US locale on both client and server.

Im happy to create a detailed ticket on the wayvnc side, but I wanted to touch base here first.

Also, previously broken 'ctrl+shift+back quote' started to work in the new version.

[dcommander]

The main thing that changed between TurboVNC 3.0.3 and 3.1 beta1 vis-a-vis keyboard handling was the implementation of the QEMU Extended Key Event RFB extension. The original RFB protocol handled key events by sending X11 keysyms from client to server, but X11 keysyms depend on the keyboard map (which depends on the locale.) QEMU Extended Key Event allows raw keycodes to be sent from client to server instead, which effectively allows the server to handle keyboard mapping. However, this would only be relevant if wayvnc implements that extension. I'll try to set up wayvnc and reproduce the issue.

[dcommander]

Which compositor are you using? And how did you start the compositor? I don't know anything about wlroots and am having trouble bootstrapping an environment to reproduce the issue.

[TobiaszCudnik]

The safest way is to start sway headless with software rendering, like so:
export SWAYSOCK=/tmp/sway-ipc.sock
env WLR_BACKENDS=headless MESA_LOADER_DRIVER_OVERRIDE=swrast sway
Then add a virtual screen:
swaymsg create_output
And start wayvnc:
wayvnc --log-level=trace
Then press meta+enter.

Default keybindings here https://i3wm.org/docs/refcard.html. Should work anywhere, including VMs, but obviously fairly slow without a GPU.

[dcommander]

On Fedora, the sway command failed with libEGL fatal: did not find extension DRI_IMAGE_DRIVER version 1. I had to remove MESA_LOADER_DRIVER_OVERRIDE=swrast to make the failure go away, then I had to set WAYLAND_DISPLAY=wayland-1 before running wayvnc.

Apparently wayvnc does support the QEMU Extended Key Event extension, so I suspect that the issue is in their implementation of that extension, as the issue is not observed with other VNC servers that implement the extension. You can pass -noserverkeymap to /opt/TurboVNC/bin/vncviewer or set serverkeymap=0 in ~/.vnc/default.turbovnc as a workaround. That will disable the TurboVNC Viewer's use of the extension, even if the server supports it, so effectively the viewer will behave like TurboVNC 3.0.x.

[dcommander]

NOTE: You can also create a .turbovnc file specifically for your wayvnc connection, set serverkeymap=0 in that file, and drag it onto the TurboVNC Viewer icon to launch it. That would allow you to use the QEMU Extended Key Event extension with other VNC servers.

[TobiaszCudnik]

After creating an issue for wayvnc the conclusion seems to be that none of the problems exist when using a recent TigerVNC macos client.

Client	alt+g	ctrl+shift+backtick
TurboVNC v3.0.3	OK	BROKEN
TurboVNC v3.0.91	BROKEN	OK
TigerVNC v1.13	OK	OK

Unfortunately the speed of TigerVNC is noticeably slower in my case and would love to have this issue solved.

NOTE: You can also create a .turbovnc file specifically for your wayvnc connection

Not related to this issue, but nomachotkeys=1 doesnt have an effect when placed in ~/.vnc/default.turbovnc, but does work when passed directly as an parameter to the binary. serverkeymap=0 works both ways.

[dcommander]

After creating an issue for wayvnc the conclusion seems to be that none of the problems exist when using a recent TigerVNC macos client.

Client alt+g ctrl+shift+backtick
TurboVNC v3.0.3 OK BROKEN
TurboVNC v3.0.91 BROKEN OK
TigerVNC v1.13 OK OK
Unfortunately the speed of TigerVNC is noticeably slower in my case and would love to have this issue solved.

The macOS TigerVNC Viewer has had a severe performance issue for many years, and this has been reported to the TigerVNC developers on multiple occasions. I haven't compared the macOS TigerVNC Viewer with the macOS TurboVNC Viewer since TurboVNC 2.1.x and TigerVNC 1.8.x, but at that time, the TigerVNC Viewer had similar decoding performance to the TurboVNC Viewer (actually faster if you used multithreaded decoding, which TurboVNC doesn't yet support), but its blitting performance was about 11x slower.

As to the keyboard issues, both TurboVNC v3.1 beta and TigerVNC v1.13 support the QEMU Extended Key Event extension. That extension allows a raw keyboard scan code to be sent in addition to the X11 keysym. For LAlt-G, TurboVNC sends a sequence of keycode 0x38/keysym 0xffe9 (XK_Alt_L) followed by keycode 0x22/keysym 0x00a9 (XK_copyright). TigerVNC sends a sequence of keycode 0x38/keysym 0xff7e (XK_Mode_switch) followed by keycode 0x22/keysym 0x00a9 (XK_copyright). IMHO, TigerVNC isn't behaving correctly, because the left Alt key would produce an XK_Alt_L keysym if that key was fired on a local X session running on the server machine.

Regardless, the VNC server is supposed to use the raw scan code along with the server-side keyboard map to determine the keyboard symbol or modifier. The X11 keysym is only a hint, and in the case of Wayland, the VNC server should arguably ignore the X11 keysym, since X11 keysyms probably aren't relevant for Wayland. However, WayVNC doesn't appear to ignore the X11 keysym. Thus, even though TurboVNC sends exactly the same keycodes as TigerVNC, apparently WayVNC looks at the X11 keysym and thus interprets the left Alt key differently.

Curiously, the RAlt-G key sequence produces the same error when sent from TurboVNC, even though in that case TurboVNC sends exactly the same keycodes and keysyms as TigerVNC. (The error doesn't occur when using TigerVNC.) As far as I can tell, TurboVNC isn't doing anything wrong.

As for Ctrl-Shift-`, the TurboVNC v3.0.x Viewer (or the TurboVNC v3.1.x Viewer with the ServerKeymap parameter disabled) sends a sequence of keysym 0xffe3 (XK_Control_L), 0xffe1 (XK_Shift_L), and 0x0060 (XK_grave), which is incorrect. The last key symbol should be 0x007e (XK_asciitilde). This is a known Java limitation, unfortunately. Java correctly reports the shifted key code when Shift is pressed, but on macOS, it incorrectly reports the unshifted key code when Shift and Control are pressed simultaneously. That is one of several reasons why I implemented the QEMU Extended Key Event extension.

tl;dr: I still assert that WayVNC's behavior is at the core of the issue. It just works accidentally with TigerVNC because WayVNC pays attention to the X11 keysym that is sent along with the keycode, but arguably it shouldn't.

NOTE: You can also create a .turbovnc file specifically for your wayvnc connection

Not related to this issue, but nomachotkeys=1 doesnt have an effect when placed in ~/.vnc/default.turbovnc, but does work when passed directly as an parameter to the binary. serverkeymap=0 works both ways.

It works if you specify it as machotkeys=0. Reversing the parameter syntax by prefixing the parameter name with no is a backward compatibility hack that only works on the command line. The documentation for connection info files (in the TurboVNC Viewer usage screen) specifies that those files only support <param>=<value> syntax, as opposed to the other syntactical forms that are acceptable on the command line. The confusion is probably because the MacHotkeys parameter is hidden, and in #345 (comment), I wasn't clear that it was necessary to use forward rather than reverse syntax when specifying the parameter in a connection info file. I have updated my comment to make that more clear.

[dcommander]

I am investigating this further at the source code level, since I realized that the keyboard handling is part of neatvnc, not wayvnc. I will let you know what I find.

[dcommander]

Mystery solved and bug fixed. Refer to b4fd6ae for details.

[TobiaszCudnik]

I can confirm that all the problematic key combos are now working on v3.1 (build 20231117).

Thanks!