unfs3 probably broken

[sjorge]

Looks like unfs3 package is broken on Solaris

2018-07-12T21:51:11+02:00 apollo unfsd[925368]: [ID 952037 user.error] svc_tli_create: could not do t_sync: Illegal file descriptor

Is all that is logged and it does not appear to start.

[jclulow]

This is an unfortunately complicated problem.

If you attempt to use the standard NFS port (2049), unfs3 will create a regular BSD style TCP socket and bind it to the specified port. It will then pass that file descriptor to svctcp_create(), part of the RPC suite of functions. On Solaris and illumos systems, that function comes from libnsl.so.1 and appears to require not a BSD socket but a TLI socket, an unfortunate remnant of UNIX history.

Note in the autoconf scripts for unfs3 there is a mention of librpcsoc.so.1, one of the SunOS/UCB compatibility libraries that we don't ship in SmartOS. This library contains an implementation of svctcp_create() which does allow the use of BSD sockets. I believe that the configure script doesn't end up using it for our pkgsrc build because it's not available on the system, but the behaviour you see above is then only visible at runtime.

One way to work around this, though it's not perfect (because it involves the use of dynamic port numbers which confuses at least the Linux NFS client) is to use the -u flag to unfsd. This will use a random port for the NFS and MOUNT services; i.e., it passes RPC_SOCKANY rather than a specific file descriptor, so there is no socket type mismatch.

There are two likely paths forward for this issue:

1. fix illumos to accept BSD sockets as well as TLI sockets for the RPC functions, though this means unfsd will still be broken until you upgrade to a platform image with this fix
2. fix unfsd to be able to create and bind a TLI socket instead of a BSD socket, which would mean we can use unfsd even on older illumos systems in advance of this fix

There's a third option, which is probably also not great, which is to create a pkgsrc package with the librpcsoc.so.1 compatibility bits in it and ship it outside the core OS -- but I don't know how hard it would be to extricate it from illumos-gate and build it elsewhere, or if it makes use of private interfaces with the rest of the OS.

[jclulow]

Note in particular that the compatibility librpcsoc.so.1 contains what appear to be wrappers around the core RPC functionality from libnsl.so.1; e.g., see https://github.com/joyent/illumos-joyent/blob/master/usr/src/ucblib/librpcsoc/svc_tcp.c#L87-L168

This function calls at least svc_xprt_alloc() which does not seem to be documented, and xprt_register(), which does. Some more digging would be required to see how safe it would be to ship a copy of this library in pkgsrc.

[sjorge]

I actually tried specifying a udp port but that did not help. This is probably related why some packages no longer build without patching because they can't find rpcgen on newer platforms. (See #126) Maybe it is worth wrapping some of the remove bits up in an optional package somehow.

[jclulow]

@sjorge Specifying a port won't help, because from the perspective of the RPC routines the same thing is happening. Whether unfsd attempts to use its specific default (2049), or whether you provide a specific port, in both cases unfsd is binding a BSD style socket and attempting to pass it to the RPC routines, which doesn't work for the reason I described above.

If instead you use the -u flag I mentioned, the software will pass RPC_SOCKANY to get a random port number. From the manual page:

       -u     Use an unprivileged port for NFS and MOUNT service. Normally,
              unfsd will use port number 2049, which is the standard port for
              NFS.  When this option is in effect, arbitrary ports chosen by
              the RPC library will be used. You may need to use this option
              when running unfsd from a normal user account.

This is not related to rpcgen -- rather, it's a long-standing issue with unfsd when built against the regular libnsl.so.1 RPC routines. As far as I know, we've never shipped the ancient compatibility librpcsoc.so.1 in the SmartOS platform image so this problem has existed in the unfsd package since we started shipping pkgsrc packages.