xcapture v3 and xtop first release

What's Changed

The first proper release of xcapture eBPF thread level activity tracker + sampler and a reference implementation xtop tool, showing how to use this data for dimensional performance analysis.

The xcapture binary uses modern eBPF, therefore it can be compiled once, into a single executable binary and copied to any other server with a sufficiently high GLIBC and kernel version (5.11+). Both x86_64 and aarch64 should work.

If you love downloading and running binaries from random GitHub accounts, you can download the xcapture (or xcapture-oldkernel) binary directly from this release:

wget https://github.com/tanelpoder/0xtools/releases/download/v3.0.3/xcapture
chmod u+x ./xcapture
sudo ./xcapture

Since this is a first and major release, lots of things have changed, I will cover the functionality and details in future blog posts.

New Contributors

• @rebhi made their first contribution in #47
• @grcevski made their first contribution in #54
• @Christoph-Lutz made their first contribution in #53
• @sjuxax made their first contribution in #57

Full Changelog: v2.0.3...v3.0.3

0xtools v3.0.0-beta1

New functionality

• xcapture now reads some data from userspace memory, can can sample userspace stack traces
• xtop text UI latency histograms, heatmaps, stack trace reporting, libaio/io_uring SQ/CQ size etc
• new tool: xstack for passive sampling of user/kernel stacks without injecting tracpoints or probes
• the beta version requires RHEL 5.14+ kernel or otherwise 5.18+ kernel (will backport some functionality)

What's Changed

• Fix sqlexecmany function by @rebhi in #47
• Fix uninitialized vars and sys call crash by @grcevski in #54
• Fix: task_struct backwards-compatibility with kernel versions < 5.14 by @Christoph-Lutz in #53
• Fix: task_struct backwards-compatibility with kernel versions < 5.14 (2) by @Christoph-Lutz in #55
• Use a /tmp file instead of RAM if duration exceeds 60 seconds by @rebhi in #49
• Fix signed div/mod compile error. by @sjuxax in #57

New Contributors

• @rebhi made their first contribution in #47
• @grcevski made their first contribution in #54
• @Christoph-Lutz made their first contribution in #53
• @sjuxax made their first contribution in #57

Full Changelog: v2.0.3...v3.0.0-beta1

v2.0.3-beta

This is still a beta / tech-nerd preview release. Please don't run it in busy production environments yet.

The main noteworthy change is that now there's a -o option to switch xcapture-bpf to always-on CSV output mode.

./xcapture-bpf -o /some/dirname

This will make xcapture log the 1HZ samples into hourly threads_* files in the output directory and if the stack capture is enabled (its enabled by default in -o mode), it will write any newly seen stack traces into a stacks_* file.

Unlike the hourly files with 1HZ thread sample history, a single stack file is created at xcapture startup (with a startup timestamp in it) and it appends newly seen, not yet written stacks into the end of the single file.

This is likely the final release of the BCC-based beta / developer preview. I've hit too many limitations sooner than I had thought and will now focus on writing a proper final version of this tool (that I can be proud of!) using libbpf and a better frontend. Use the current v2 release for testing and exploring what's possible!

Full Changelog: v2.0.2...v2.0.3

v2.0.2-beta

First major release of the next xcapture-bpf beta tool.
Updates and fixes for the classic 0x.tools, psn, xcapture (/proc).

What's Changed

Lots!

New Contributors

• @skoehler-soocs made their first contribution in #39

Full Changelog: v1.2.4...v2.0.2

v1.2.4 bugfixes

Fixed a bug where wrong system call names were shown on some versions/platforms (due to looking up wrong unistd.h file)

Full Changelog: v1.2.3...v1.2.4

v.1.2.3

• Supports arm64 and ppc64le platforms (unistd.h lookup for syscall names)

• Added more "single file descriptor" system calls to psn (for reporting the accessed file name)

• xcapture supports basic container namespace reporting (use -c nspid or -o output_dir as CSV output reports NSPID by default)

• psn with python3 now prints extra newlines in output as intended

• added a rudimentary tool vmtop for showing snapshots of /proc/vmstat activity

• also added rpmbuild files, but it's not working well yet, as rpmbuild seems to want to use a hard-coded python binary name instead of /usr/bin/env python (will look into this before next release)

What's Changed

• Add unistd.h path of CentOS7 ppc64le release by @xcnix in #28
• add few syscalls to report dir as filename by @timurakhmadeev in #17
• Support systemd and rpmbuild by @hungrybirder in #29

New Contributors

• @xcnix made their first contribution in #28
• @timurakhmadeev made their first contribution in #17
• @hungrybirder made their first contribution in #29

Full Changelog: v1.1.3...v1.2.3

v1.1.3

Changed make uninstall to use rm rf for removing a python lib dir

v1.1.2

Fixed a make install issue caused by another fix in v1.1.1 (need automated test flow!)

v1.1.1

Fixed a python path issue so that pSnapper works both from an installed location (/usr/bin) or from within the source tarball directory.

v1.1.0

• Python3 support for psn
• Changed license from GPL 3.0 to GPL 2.0 or higher
• xcapture can do more frequent sampling now (sleep <1s)
• Using semantic versioning now
• Accepted first PR from other people (thanks @lefred , @tcdale @ckujau)