RandomAccess.Write throws System.IO.IOException: Invalid argument when buffers count is greater than 1024

[miroslavp]

Description

Even though this exception sounds very similar to #108322 (comment) , it is actually a very different issue. It occurs in the Unix implementation of the RandomAccess.WriteGatherAtOffset method
This is the stack trace

System.IO.IOException: Invalid argument
at System.IO.RandomAccess.WriteGatherAtOffset(SafeFileHandle handle, IReadOnlyList`1 buffers, Int64 fileOffset)

Looking at the implementation I have noticed that we call Interop.Sys.PWriteV passing the buffersCount as a parameter

runtime/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs

Lines 199 to 202 in 01aa3d9

	fixed (Interop.Sys.IOVector* pinnedVectors = &MemoryMarshal.GetReference(vectors))
	{
	bytesWritten = Interop.Sys.PWriteV(handle, pinnedVectors, buffersCount, fileOffset);
	}

which in turn calls the native implementation

runtime/src/native/libs/System.Native/pal_io.c

Line 2000 in c049b85

while ((count = pwritev(fileDescriptor, (struct iovec*)vectors, (int)vectorCount, (off_t)fileOffset)) < 0 && errno == EINTR);

According to the documentation of pwritev here http://man.he.net/man2/pwritev we cannot pass buffersCount greater than 1024

NOTES
       POSIX.1  allows  an  implementation  to  place a limit on the number of
       items that can be passed in iov.  An implementation can  advertise  its
       limit  by  defining IOV_MAX in <limits.h> or at run time via the return
       value from sysconf(_SC_IOV_MAX).  On modern Linux systems, the limit is
       1024.  Back in Linux 2.0 days, this limit was 16.

Reproduction Steps

Call RandomAccess.Write(handle, buffers, fileOffset) by passing list of buffers where buffers are more than 1024

Expected behavior

It should save the buffers to file

Actual behavior

It throws System.IO.IOException: Invalid argument

Regression?

Not tested on .NET6

Known Workarounds

You can split the buffers into smaller lists (<=1024 elements) and call RandomAccess.Write multiple times with the proper offset

Configuration

Which version of .NET is the code running on?
.NET7
What OS and version, and what distro if applicable?
Ubuntu 22.04.3 LTS under WSL2
What is the architecture (x64, x86, ARM, ARM64)?
x64
Do you know whether it is specific to that configuration?
It is specific to the Unix implementation of the RandomAccess.WriteGatherAtOffset method

Other information

Haven't tested it, but this should also affect Read(SafeFileHandle handle, IReadOnlyList<Memory<byte>> buffers, long fileOffset) which calls ReadScatterAtOffset(SafeFileHandle handle, IReadOnlyList<Memory<byte>> buffers, long fileOffset)

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

[NicoAvanzDev]

Hi @miroslavp according to the documentation that you've linked we can see the following statements just below the "note" section:

Historical C library/kernel differences
To deal with the fact that IOV_MAX was so low on early versions of
Linux, the glibc wrapper functions for readv() and writev() did some
extra work if they detected that the underlying kernel system call
failed because this limit was exceeded. In the case of readv(), the
wrapper function allocated a temporary buffer large enough for all of
the items specified by iov, passed that buffer in a call to read(2),
copied data from the buffer to the locations specified by the iov_base
fields of the elements of iov, and then freed the buffer. The wrapper
function for writev() performed the analogous task using a temporary
buffer and a call to write(2).

   The need for this extra effort in the glibc wrapper functions went away
   with Linux 2.2 and later.  However, glibc continued to provide this be-
   havior until version 2.10.  Starting with glibc version 2.9, the  wrap-
   per  functions  provide  this behavior only if the library detects that
   the system is running a Linux kernel older than version 2.6.18 (an  ar-
   bitrarily  selected  kernel  version).  And since glibc 2.20 (which re-
   quires a minimum Linux kernel version of  2.6.32),  the  glibc  wrapper
   functions always just directly invoke the system calls.

Given the context cited above there are two possible solutions in my opinion:

1. Reimplement the wrapper code in the native part of pal_io.c just like glibc was used to work with version < 2.10
2. Properly handle the error where buffers are larger than IOV_MAX and fail correctly

I'd like to have an opinion from @adamsitnik that was really helpful in the previous issue.

Thanks guys for your time !