Canonicalize pin paths #4274

dthaler · 2025-03-10T19:36:30Z

Description

Pinning related APIs should treat pin paths the same as filesystem paths and canonicalize them accordingly
Fix documentation error with ebpf_get_next_pinned_object_path() where the start_path cannot be null.
Added a test. Note that in doing so, I found and filed issue pinned_path_count is always 0 #4273 and added a TODO for that issue in the test where it should be tested.

Note: on Linux, pin paths are case sensitive, and so any programs ported from Linux may assume case sensitivity.
On Windows, it varies by filesystem, where Windows itself supports both case sensitive filesystems and case insensitive filesystems. The latter is the normal filesystem on Windows, but if we later support a BPF filesystem, we can make it be case sensitive, for ease of porting from Linux. As such, this PR uses case sensitive pin paths. This is not a change for ebpf-for-windows, which already had case-sensitive pin paths.

Testing

This PR contains associated tests.

Documentation

This PR includes a design doc added to the docs directory.

Installation

No impact.

scripts/check_binary_dependencies_ebpfapi_dll_regular_debug.txt

lmb · 2025-03-17T10:34:13Z

tests/end_to_end/end_to_end.cpp

+    REQUIRE(map_fd > 0);
+
+    // Verify pin path canonicalization.
+    _verify_canonical_path(map_fd, "\\ebpf\\1", "C:\\ebpf\\1");


The C: prefix is kind of unexpected, is there a way to avoid that? The reason is that in Go the canonical way to handle paths is the filepath package. So to derive the pin path of an object we do:

filepath.Join("/sys/fs/bpf", "./../test")

The result of that function is \sys\fs\test. I guess this would work fine as long as we never iterate over pins, but it'd be nicer if the common use of the language native API would match up with the canonicalisation. Another way of saying this is: in the absence of a real BPF filesystem on Windows I think it'd be better to do purely lexical processing of the path.

Another question: what influences the C: in the output filename? Does it change based on the working directory of the calling process?

After a quick search, https://en.cppreference.com/w/cpp/filesystem/path/lexically_normal or similar might work? Lexical processing doesn't answer the question what should happen to relative paths like foo/bar, but maybe the right answer is to only accept absolute paths for pinning right now? (So ones starting in \.)

The C: prefix is kind of unexpected, is there a way to avoid that? The reason is that in Go the canonical way to handle paths is the filepath package.

Let's discuss more in the meeting. The ":" prefix is the common Windows filesystem mechanism. How does the filepath package deal with normal filesystem paths then?

https://pkg.go.dev/path/filepath says that drive letters are supported on Windows, and so it should not be unexpected.

changed to "BPF:" per analysis in the PinPaths.md doc that is now part of this PR.

docs/PinPaths.md

Signed-off-by: Dave Thaler <[email protected]>

lmb · 2025-03-19T09:13:15Z

docs/PinPaths.md

+* Pins can be enumerated or removed via normal filesystem APIs/utilities
+* Pin paths are case-sensitive
+* The path component separator is a forward slash ('/')
+* Pin paths begin with "/sys/fs/bpf/" and relative paths like "mymap" are relative to that prefix.


Not necessarily, this is just a convention that systemd and the popular user space libraries follow. It is possible to mount a separate BPF fs at say /var/run/myapp/bpffs and then pin into that.

lmb · 2025-03-19T09:16:01Z

docs/PinPaths.md

+the driver implements it as such.
+
+As for portability, canonicalization could ensure that Linux style paths like
+"/sys/fs/bpf/my/pin/path" would be canonicalized to "BPF:\my\pin\path" allowing


My gut feeling would be to not remove the sys/fs/bpf prefix. If you do want to remove it, what should happen to paths that do not have it?

lmb · 2025-03-19T09:16:38Z

docs/PinPaths.md

+
+As for portability, canonicalization could ensure that Linux style paths like
+"/sys/fs/bpf/my/pin/path" would be canonicalized to "BPF:\my\pin\path" allowing
+portability in that respect.  But querying the pin path on an object


Incomplete sentence?

lmb · 2025-03-19T09:39:43Z

docs/PinPaths.md

+
+1. RemoteFile: Avoid confusion with remote paths.
+
+1. FileAPIs: Paths should work with Windows file APIs in the future once a BPF filesystem is implemented.


Is this a necessary requirement? I had a very brief search on how this would be achieved on Windows and my impression is that a filesystem is a significant piece of work?

At this point, we already have support for calling into the user space APIs in C and Go. I believe most other libraries will provide an API for unpinning (rather than expecting the user to do straight up file system calls). Do we gain much by switching to a real filesystem?

P.S. If we want to keep the filesystem route open we probably need to take https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file#naming-conventions into account.

Whether we do it or not is really up to future demand and priorities, but the point is to not preclude it if we do need it in the future.

lmb · 2025-03-19T09:58:21Z

docs/PinPaths.md

+
+### UNC (UNC path)
+
+Example: \\BPF\my\pin\path


There is also \\?\ which would disable all path parsing and would forward the path to the filesystem verbatim. And \\.\ which points into the object namespace, but I don't know whether we could hook that somehow?

lmb · 2025-03-19T10:00:36Z

docs/PinPaths.md

+
+### Virtual (Windows file system path with another virtual drive for BPF)
+
+Example: BPF:\my\pin\path


Sorry for being a windows noob, is it possible to have multi letter drives? Never seen one of those.

Also, would this drive show up in the explorer?

It's not a drive, but a virtual device (aka DOS device name). There are a bunch of these in windows, like CON:, PRN:, NUL:, COM1:, etc. And drivers can add new ones.

dthaler requested review from Alan-Jowett, dv-msft, LakshK98, matthewige, mikeagun, mtfriesen, poornagmsft, saxena-anurag, shankarseal and vpidatala94 as code owners March 10, 2025 19:36

dthaler requested a review from lmb March 10, 2025 19:36

dthaler mentioned this pull request Mar 10, 2025

pinned_path_count is always 0 #4273

Open

dthaler changed the title ~~Canonicalize paths~~ Canonicalize pin paths Mar 10, 2025

Alan-Jowett previously approved these changes Mar 11, 2025

View reviewed changes

dthaler dismissed Alan-Jowett’s stale review via 3c38a22 March 11, 2025 18:27

Alan-Jowett reviewed Mar 12, 2025

View reviewed changes

scripts/check_binary_dependencies_ebpfapi_dll_regular_debug.txt Outdated Show resolved Hide resolved

dthaler force-pushed the canonicalize-paths branch from 888ff28 to 8717508 Compare March 14, 2025 15:55

lmb reviewed Mar 17, 2025

View reviewed changes

saxena-anurag reviewed Mar 17, 2025

View reviewed changes

docs/PinPaths.md Outdated Show resolved Hide resolved

dthaler force-pushed the canonicalize-paths branch 4 times, most recently from e45e561 to 53c55e1 Compare March 18, 2025 02:34

dthaler added 3 commits March 18, 2025 08:50

Canonicalize pin paths using filesystem canonicalization rules

e2e2fe0

Signed-off-by: Dave Thaler <[email protected]>

Update some bpftool_tests

16fa5b0

Signed-off-by: Dave Thaler <[email protected]>

Add decision to PinPaths.md

a20d9c8

Signed-off-by: Dave Thaler <[email protected]>

dthaler force-pushed the canonicalize-paths branch from 81b2d93 to a20d9c8 Compare March 18, 2025 15:54

dthaler requested review from lmb, Alan-Jowett and saxena-anurag March 18, 2025 15:54

lmb reviewed Mar 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Canonicalize pin paths #4274

Canonicalize pin paths #4274

dthaler commented Mar 10, 2025 •

edited

Loading

lmb Mar 17, 2025

lmb Mar 17, 2025

dthaler Mar 17, 2025

dthaler Mar 17, 2025

dthaler Mar 18, 2025

lmb Mar 19, 2025

lmb Mar 19, 2025

lmb Mar 19, 2025

lmb Mar 19, 2025

lmb Mar 19, 2025

dthaler Mar 19, 2025

lmb Mar 19, 2025

lmb Mar 19, 2025

dthaler Mar 19, 2025


		1. RemoteFile: Avoid confusion with remote paths.

		1. FileAPIs: Paths should work with Windows file APIs in the future once a BPF filesystem is implemented.


		### Virtual (Windows file system path with another virtual drive for BPF)

		Example: BPF:\my\pin\path

Canonicalize pin paths #4274

Are you sure you want to change the base?

Canonicalize pin paths #4274

Conversation

dthaler commented Mar 10, 2025 • edited Loading

Description

Testing

Documentation

Installation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dthaler commented Mar 10, 2025 •

edited

Loading