X11 with VNC, Fatal server error: (EE) Failed to activate virtual core keyboard: 2(EE) , termux workaround

[deliciouslytyped]

Update: termux not needed with #75 (comment)

I've been fiddling with #34 , and made something akin to progress, but there are some issues;

• strace doesnt work, so I'm not sure how to figure out why Xvfb isn't working. GDB works but I don't know how to deal with multithreaded-processed things with it. strace doesn't work #76
• the hybridization approach seems to leave everything visible in VNC hanged, whatever that means.*

cc @ShamrockLee

As a temporary workaround for #34, I tried installed and running X11 in termux.
I successfully installed and ran Xvfb, and connected nix-built applications to the remote display via the $DISPLAY variable over the loopback device. So a hybrid termux/nix-on-droid approach seems to work*.

A thrown together tutorial (no authentication, etc, the point was just to see if I can even get this to work):

• Install termux and nix-on-droid from F-Droid
• install a VNC app like VNC Viewer (by RealVNC) from the play store, or MultiVNC from F-Droid. Sadly the UI design of the former seems to work somewhat better. For example: the MultiVNC keyboard overlaps with the screen area for me, which is irritating Feature request: shift the canvas up/down when the keyboard is toggled bk138/multivnc#76 (comment)
• keep installing stuff until termux is happy; the main things needed are tigervnc, providing the vncserver and x0vncserver commands, which spawn an Xvnc process, and the X11 stuff which provides Xvnc, Xvfb, etc.
• in nix, load a shell with something like awesomewm and xclock for testing
• TERMUX: run Xvfb to start an X server. It seems not to expose a network port by default so we make it listen on a port, and we disable host based authentication (TODO: check the auth disabling is needed): Xvfb -listen tcp -ac &
• TERMUX: run x0vncserver to attach a vnc server to an existing display (TODO: check how to do it without password) (TODO: check if setting env var is needed) (TODO: the passwd file is created by running vncserver probably?): DISPLAY=:0 x0vncserver -rfbauth /data/data/com.termux/files/home/.vnc/passwd
  Edit: NIX: start a nix-shell with x11vnc and run DISPLAY=127.0.0.1:0 x11vnc -passwd whatever -rfbport 5901 -noshm -forever see X11 with VNC, Fatal server error: (EE) Failed to activate virtual core keyboard: 2(EE) , termux workaround #75 (comment)
• NIX: export DISPLAY=127.0.0.1:0 && awesome & xclock &
• VNC CLIENT: connect to 127.0.0.1:5900 with the set password (or whatever port, check with netstat) and enjoy your frozen windows

TODO: script that sshes to the termux and handles everythig in one place

[deliciouslytyped]

With regards to the apparent "hanging", I failed to notice that xclock didn't have a second hand, of course it looked stuck. :P
So there is probably some sort of virtual input device issue. xclock can be started with an update time < 30 seconds and a second hand will be added.

[deliciouslytyped]

xinput shows:

$ DISPLAY=127.0.0.1:0 xinput
⎡ Virtual core pointer                          id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ Xvfb mouse                                id=6    [slave  pointer  (2)]
⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
    ↳ Xvfb keyboard                             id=7    [slave  keyboard (3)]

[deliciouslytyped]

See also https://wiki.archlinux.org/index.php/TigerVNC and https://wiki.archlinux.org/index.php/X11vnc . I'm going to try with x11vnc, maybe it will do something better resulting in things working.

[deliciouslytyped]

x11vnc works for the above! You just have to disable the shmem feature (* see error below and noshm in https://jlk.fjfi.cvut.cz/arch/manpages/man/x11vnc.1) to be able to use the display over the network (remember, the X11 server is running on the termux side).

• NIX: DISPLAY=127.0.0.1:0 x11vnc -passwd whatever -rfbport 5901 -noshm -forever

*shmem error when trying to access a remote display:

26/08/2020 05:19:39 shmget(scanline) failed.                                                                          
26/08/2020 05:19:39 shmget: No such file or directory 

It remains an open question what went wrong with no input with x0vncserver.

[t184256]

That's some determination! I suggest you to make it into a wiki article, even if you don't consider that finished.

Couple of notes:

strace doesnt work

AFAIK, strace or anything using ptrace doesn't work in proot in general, something about ptrace not being able to nest. There is a chance of something has changed since, but I don't recall that changing.

install a VNC app like VNC Viewer (by RealVNC) from the play store, or MultiVNC from F-Droid.

My favourite has always been bVNC. Just look at this beauty --- active since forever and still going strong.

Xvfb + x11vnc

Sorry, I didn't exactly get that: why this combination instead of a single vnc server like tigervnc?

[deliciouslytyped]

I think the Xvfb + x11vnc combination was kind of accidental. I didn't figure out if I can pass arguments on to X11 when starting the vnc server (probably yes?) - so I just started Xvfb appropriately and ran x0vncserver; but then there were the input problems so I tried switching to x11vnc because it seemed fancier, and then it worked.

Edit: thanks for the client suggestion, I'll check it out! Always happy to have tool recommendations instead of haphazard searching.

Edit2: the vnc server and x11 are separate because the current variant needs the vnc server started from nix (x11vnc isnt in termux), while x11 needs to run in termux - but it might work if I try going back to vncserver with my new knowledge, let's see what happens.

Edit3: OK, I'm not sure right now where this leaves me, but it seems using vncserver in termux (without a separately prepared Xvfb session) works properly. However bVNC says something about encryption problems but VNC Viewer works.

Edit4: I rediscovered the first reason that I separated the VNC and X components: I hoped it would help ease debugging, back when I thought I might find a way to debug the X11 error. Running vncviewer (on nix) also yields the X error in the title, presumably through Xvnc.

[deliciouslytyped]

By the way, xterm doesn't work for some reason (window doesn't show up) - but other terminals like urxvt (rxvt-unicode package) work.

[deliciouslytyped]

With regards to debugging, guided by the hint of https://unix.stackexchange.com/questions/314335/how-to-run-xvfb-without-root/316240 ( https://ericdraken.com/running-xvfb-on-a-shared-host-without-x/ ), which has the same error, but different goal - and from which as of yet I haven't been able to gain much insight - I realized that we should be able to intercept the input and output of xkbcomp by writing a wrapper script that captures these to some file. Hopefully this will lead to diagnosing some fixable issue. I'm waiting for the build on my phone. (The build failed. I'm trying again, hopefully it was nondeterministic.)

Alternative ideas were:

• trying to use LD_PRELOAD to do some sort of tracing. (general example https://stackoverflow.com/questions/37911702/overriding-execve-with-ld-preload-only-works-sometimes)

But I finally figured out how to get past some Issues I had with multiprocess/threaded GDB debugging*, which does work**.
This would have worked if it was just a matter of getting the commandline for the xkbmap call, but data seems to be passed on stdin and I wasn't sure how to dump that without a lot of hassle, which made me think of the wrapper script alternative.

*https://sourceware.org/gdb/onlinedocs/gdb/Background-Execution.html
https://web.stanford.edu/class/cs107/resources/gdb_coredump1.pdf
https://www-zeuthen.desy.de/unix/unixguide/infohtml/gdb/Non_002dStop-Mode.html

**The general procedure is to set a breakpoint on Popen, set the breakpoint command to print (char *)$x0 to print the first function argument per ARM ABI (https://en.wikipedia.org/wiki/Calling_convention#ARM_(A64) , https://stackoverflow.com/a/4264953 , https://stackoverflow.com/a/233339), set gdb to not detach from child processes, and then to run through the application manually continuing a thread where necessary.

Code (and a cloneable repo) is at https://github.com/freedesktop/xorg-xserver/blob/f33cb4264387ed14a586ba080885b4d21e4aa48b/xkb/ddxLoad.c#L139

Tangentially,
The Xserver man page is mentioned as the base for Xorg and Xvfbboth. Strangely enough only Xorg actually seems to provide functionality(namely -logverbose and -logfile) for changing log levels and log file locations. Does this mean Xvfb simply has nothing worth changing, or just that the interface is, irritatingly, crippled?

[deliciouslytyped]

TODO: figure out why Xvfb + x0vncserver had input issues.
It's probably some mis- or lack of, configuration. Someone suggested that perhaps evdev isn't loaded by default.

[deliciouslytyped]

TL;DR: wasted too much time thinking xkbcomp was even being run at all.

After some more GDB shenanigans; it turns out x11 has it's own Popen function, which causing a premature exit at https://github.com/freedesktop/xorg-xserver/blob/2902b78535ecc6821cc027351818b28a5c7fdbdc/os/utils.c#L1433 .

Running nix-shell -p python37 --run "python -c ''import os; os.setgid(os,getgid())" yields

Traceback (most recent call last):
  File "<string>", line 1, in <module>
OSError: [Errno 38] Function not implemented

Which is probably a proot issue(TODO proot docs?). I will try to stub out setgid with a noop and LD_PRELOAD to get past this.
Similarly for the UID functions.

Edit: Hey! I think it works!
I suppose I should have looked into my hunch earlier - that Popen was being called but not getting as far as exec (but it seemed absurd and tedious to debug at the time). To add to the above, it was a bit less of a pain after running gef (https://github.com/hugsy/gef), and I just nexti through the function after a Popen breakpoint. This allowed me to match the execution path to the source (maybe I should have recompiled with debug symbols), showing the premature exit after the setgid().
Based on the previously mentioned SO post, I wrote nop stubs for LD_PRELOAD in the form of the following:

id.c:

#include <sys/types.h>
#include <stdio.h>

int setgid(gid_t gid){ printf("WARNING: setgid stubbed\n"); return 0; };
int setuid(uid_t uid){ printf("WARNING: setuid stubbed\n"); return 0; };

compiled with: gcc id.c -std=c99 -o id.so -shared -fPIC
and loaded via: LD_PRELOAD=./id.so Xvfb, yielding:

[nix-shell:~]$ LD_PRELOAD=./id.so Xvfb :8
_XSERVTransmkdir: Owner of /tmp/.X11-unix should be set to root
/tmp/comp.9Dl3dMgJh/
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning:          Unsupported high keycode 372 for name <I372> ignored
>                   X11 cannot support keycodes above 255.
>                   This warning only shows for the first high keycode.
Errors from xkbcomp are not fatal to the X server
^C

□

Edit 2: for some reason the printfs don't seem to actually show up anywhere.
Anyway, here's the wrapper script for posterity:

let
  ov = self: super: {
    xorg = super.xorg // {
      xkbcomp = super.writeShellScriptBin "xkbcomp" ''
        tbasedir=$(mktemp -d comp.XXXXXXXXX -p /tmp )/
        echo $tbasedir
        tee ''${tbasedir}stdin.txt | {
        ${super.xorg.xkbcomp}/bin/xkbcomp "$@"
        } 1> >(tee ''${tbasedir}stdout.txt ) 2> >(tee ''${tbasedir}stderr.txt >&2 )
        '';
      };
    };
  #I didn't check but Xvfb probably uses an xkbcomp built with it since they are the same codebase, so we have to do a two-staged build to get one with a wrapper
  ov2 = self: super: {
    xorg = super.xorg // {
      xorgserver = super.xorg.xorgserver.overrideAttrs (old: {
        configureFlags = old.configureFlags ++ [ "--with-xkb-bin-directory=${super.xorg.xkbcomp}/bin" ];
        });
      };
    };
in
with import <nixpkgs> { overlays = [ ov ov2 ]; };
  mkShell {
    buildInputs = [ rxvt-unicode xorg.xorgserver xorg.xkbcomp xorg.xinput xorg.xclock awesome xterm xdotool x11vnc nano gdb which ];
    }

TODO: why does this lead to that particularly unhelpful error message?

[deliciouslytyped]

Updated tut:

• install nix-on-droid from fdroid
• run nix-shell x11.nix --run "myx :1" where x11.nix is:

with import <nixpkgs> {};
let
  id_c = writeText "id.c" ''
    #include <sys/types.h>
    #include <stdio.h>

    int setgid(gid_t gid){ printf("WARNING: setgid stubbed"); return 0; }
    int setuid(uid_t uid){ printf("WARNING: setuid stubbed"); return 0; }
    '';

  id_so = runCommand "id.so" { buildInputs = [ gcc ]; } ''
    mkdir -p $out
    gcc -std=c99 -shared -fPIC ${id_c} -o $out/id.so
    '';

  myx = writeShellScriptBin "myx" ''
    export DISPLAY=$1 #TODO dynamic?
    LD_PRELOAD=${id_so}/id.so ${xorg.xorgserver}/bin/Xvfb $1 -ac -listen tcp &
    sleep 5
    ${x11vnc}/bin/x11vnc -display $1 -passwd test -rfbport 5902 -noshm -forever &  #not sure why noshm still needed
    awesome &
    urxvt -e env TERM=xterm tmux & #TODO probably wrong
    '';
in mkShell { buildInputs = [ myx awesome rxvt-unicode tmux ]; }

• be happy, with the applications that dont break in this environment 🎉

Demo:

[deliciouslytyped]

It could have been a bit of a shortcut to check the termux packaging, which in fact involves some similar patches:
https://github.com/termux/x11-packages/blob/3dcfccb30af147709c1e650aec9926b87cb2336f/packages/xorg-server-xvfb/xorg-server-1.20.1_os_utils.c.patch#L26

• but this is obvious mainly in hindsight (also ps doesn't work; /proc problems? #77 (comment)).