Rebase the required Linux kernel on top of latest released version

[keryell]

The Linux kernel required to run this xdna-driver is based on an old release candidate of Linux 6.7 but Linux 6.7.2 has been released since with a lot of fixes (https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.7.2).
It would be nice to have the latest version on https://github.com/AMDESE/linux to minimize troubles.
Technically I should open this issue on https://github.com/AMDESE/linux but that GitHub repository does not allow opening issue. Strange vision of open-source... ;-)

[maxzhen]

We have no control over https://github.com/AMDESE/linux. But we need the change from that repository. For the team controlling https://github.com/AMDESE/linux, they have other reasons / dependencies to choose this as base. So, this is not as flexible as you want.

[superm1]

The dependency on patches in that repo is temporary, the patches in it are on their way upstream. Some are making it for 6.8, but a few will go out to 6.9 (or later depending upon code review).

[K2]

The 6.7.2 PR I sent #7 was against a patched ESE tree into 6.7.2. The patch from https://cdn.kernel.org/pub/linux/kernel/v6.x/patch-6.7.2.xz applies cleanly against the amdese tree with only 2 reg files, the primary Makefile subversion # and an HP driver one line fix to reapply it.

[keryell]

I am back into this with my Ubuntu 23.10 laptop.
To have a stable mainline kernel, I have rebased iommu_sva_v4_v6.7-rc8 Linux on top of latest v6.7.4 as a my iommu_sva_v4_v6.7.4 branch.
Then the problem to have something nicely working kernel is to start with a good .config. I got some inspiration from https://github.com/zabbly/linux and https://pkgs.zabbly.com/kernel/stable/pool/main/l/linux-zabbly-6.7.4-amd64-202402052246-ubuntu22.04/linux-image-6.7.4-zabbly%2B_6.7.4-amd64-202402052246-ubuntu22.04_amd64.deb

So now my minimal recipe for the kernel is:

git clone --depth=1 --branch iommu_sva_v4_v6.7.4 [email protected]:keryell/linux.git
cd linux
wget --output-document=.config https://gist.github.com/keryell/885c6963ec65bee6a03664e7379c876f/raw/9ed39d5adb07896f16a89def999aacd755a345cd/gistfile1.txt
make -j `nproc` bindeb-pkg
sudo apt install ../linux-headers-6.7.4+iommu-sva-v4+_6.7.4-00064-g7541606e8262-1_amd64.deb ../linux-image-6.7.4+iommu-sva-v4+_6.7.4-00064-g7541606e8262-1_amd64.deb

After installing XRT and this driver, the examples works but there are some driver issues at the end of the application #16

[keryell]

My new kernel recipe with a branch https://github.com/keryell/linux/tree/v6.7%2Biommu_sva_v4%2Bpasid_flush fixing #16 by applying suggested #16 (comment) :

git clone --depth=1 --branch v6.7+iommu_sva_v4+pasid_flush [email protected]:keryell/linux.git
cd linux
wget --output-document=.config https://gist.github.com/keryell/d5be08aa3c10484bdb4dc3ddbafc2347/raw/d6606403d9f27efa423a75f6c7a99bfde18dfeef/gistfile1.txt
make -j `nproc` bindeb-pkg
sudo apt install ../linux-headers-6.7.4+iommu-sva-v4+pasid-flush+_6.7.4-00065-gcd6023b0d8ef-2_amd64.deb ../linux-image-6.7.4+iommu-sva-v4+pasid-flush+_6.7.4-00065-gcd6023b0d8ef-2_amd64.deb 

@maxzhen @hegdevasant Any reason for not pushing such a branch to https://github.com/AMDESE/linux while waiting for the next version?

[keryell]

Since Linux kernel v6.7.5 is out, I have rebased iommu-sva-v4+pasid-flush on top of it.
The new recipe:

git clone --depth=1 --branch v6.7+iommu_sva_v4+pasid_flush [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/f71aa5d86983fbc4491a363a67f8c6ad/raw/ac4800c94b91acee3e28c2afbd560df5b05a1c21/.config
make -j `nproc` bindeb-pkg
sudo apt install ../linux-headers-6.7.5+iommu-sva-v4+pasid-flush+_6.7.5-00065-g1288eba56e68-4_amd64.deb ../linux-image-6.7.5+iommu-sva-v4+pasid-flush+_6.7.5-00065-g1288eba56e68-4_amd64.deb

[hegdevasant]

My new kernel recipe with a branch https://github.com/keryell/linux/tree/v6.7%2Biommu_sva_v4%2Bpasid_flush fixing #16 by applying suggested #16 (comment) :

git clone --depth=1 --branch v6.7+iommu_sva_v4+pasid_flush [email protected]:keryell/linux.git
cd linux
wget --output-document=.config https://gist.github.com/keryell/d5be08aa3c10484bdb4dc3ddbafc2347/raw/d6606403d9f27efa423a75f6c7a99bfde18dfeef/gistfile1.txt
make -j `nproc` bindeb-pkg
sudo apt install ../linux-headers-6.7.4+iommu-sva-v4+pasid-flush+_6.7.4-00065-gcd6023b0d8ef-2_amd64.deb ../linux-image-6.7.4+iommu-sva-v4+pasid-flush+_6.7.4-00065-gcd6023b0d8ef-2_amd64.deb 

@maxzhen @hegdevasant Any reason for not pushing such a branch to https://github.com/AMDESE/linux while waiting for the next version?

Note that I push code to AMDESE/linux so that its easy for upstream reviewers to fetch and review it as it has many dependencies, etc. I don't intent to rebase these branches. As I rework I rebase on latest upstream / iommu-next tree and push it. Like I have reworked/rebase patches on top of v6.8-rc2 (amdese/iommu_sva_part4_v6_v6.8_rc2).

-Vasant

[keryell]

Thanks @hegdevasant for at least pushing your work on https://github.com/AMDESE/linux
But is there no incentive to make this more palatable to end-users?

[keryell]

It is time for a new kernel, since Linux kernel https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/?h=v6.7.6 is out, I have rebased iommu-sva-v4 + pasid-flush on top of it.
The new recipe:

git clone --depth=1 --branch v6.7+iommu_sva_v4+pasid_flush [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/38b1d8bc34518e2fe0e3f438675437df/raw/bd4e45575e87fa2acda3f0b5a4c92f1f243ee789/.config
make -j `nproc` bindeb-pkg
# To adapt according to the real names
sudo apt install ../linux-headers-6.7.6+iommu-sva-v4+pasid-flush+_6.7.6-00065-gebd26810c997-1_amd64.deb ../linux-image-6.7.6+iommu-sva-v4+pasid-flush+_6.7.6-00065-gebd26810c997-1_amd64.deb

[maxzhen]

We have moved to 6.8 kernel. Please see the README for details.

[keryell]

We have moved to 6.8 kernel. Please see the README for details.

Great news! Is this included in the main 6.8 release? @superm1 suggested 1.5 month ago it might no be complete.
If not, I need to rebase this on top of the stable 6.8 which should come pretty soon.

[superm1]

We have moved to 6.8 kernel. Please see the README for details.

Great news! Is this included in the main 6.8 release? But @superm1 suggested 1.5 month ago it might no be complete.
If not, I need to rebase this on top of the stable 6.8 which should come pretty soon.

Not everything for SVA support is in upstream 6.8, so yes you'll need the remaining IOMMU SVA patches.

[keryell]

The kernel of the day since Linux kernel git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/?h=v6.7.9 is out. I have rebased iommu-sva-v4 + pasid-flush on top of it.
The new recipe:

git clone --depth=1 --branch v6.7+iommu_sva_v4+pasid_flush [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/c4865bddd2bd27dbef86195571b770af/raw/3cd09c2c7cfaaa9be09cbcf472422e1acfd9ad29/.config
make -j `nproc` bindeb-pkg
sudo apt install ../linux-headers-6.7.9+iommu-sva-v4+pasid-flush+_6.7.9-00064-gf823452a96f0-1_amd64.deb ../linux-image-6.7.9+iommu-sva-v4+pasid-flush+_6.7.9-00064-gf823452a96f0-1_amd64.deb

Note that since the latest xdna-driver does not support 6.7 anymore, I use 51887d3 as the latest version supporting 6.7.
I hope the final 6.8 version will land soon so I can forget about 6.7.

[keryell]

We are currently in a grey zone since this driver no longer work on 6.7 kernels but not yet on stable 6.8 kernel...

[keryell]

Since Linux kernel https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/?h=v6.8 is out, I have rebased https://github.com/AMDESE/linux/tree/iommu_sva_part4_v6_v6.8_rc2 on top of it into https://github.com/keryell/linux/tree/v6.8-iommu-sva-part4-v6
Having all the pieces working has been quite more complicated that with 6.7.
The current recipe is:

git clone --depth=1 --branch v6.8-iommu-sva-part4-v6 [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/43108736e05daf7270a9f082819ef001/raw/1a4a4e158fd08ddfca12d623146fa0ad0a1f51f1/.config
make -j `nproc` bindeb-pkg
sudo apt reinstall ../linux-headers-6.8.0+iommu-sva-part4-v6+_6.8.0-00051-g559cc476caf5-1_amd64.deb ../linux-image-6.8.0+iommu-sva-part4-v6+_6.8.0-00051-g559cc476caf5-1_amd64.deb ../linux-libc-dev_6.8.0-00051-g559cc476caf5-1_amd64.deb

Then a port of XRT for Linux 6.8 is required, from Xilinx/XRT#8005
Once it is installed, you can install the latest https://github.com/amd/xdna-driver

[hegdevasant]

Note that we have forked different tree for IOMMU project under amdese. We have started pushing changes to https://github.com/AMDESE/linux-iommu/ tree.

Latest version v7 is at : https://github.com/AMDESE/linux-iommu/tree/iommu_sva_part4_v7_v6.8_rc7 (This is based on top of iommu/next tree ..which is targeted for v6.9 and SVA patches).

-Vasant

[keryell]

@hegdevasant thanks for the information. I will try this new branch.

[keryell]

I have rebased https://github.com/AMDESE/linux-iommu/tree/iommu_sva_part4_v7_v6.8_rc on top of v6.8 where I had to fix a minor merge conflict in https://github.com/keryell/linux/tree/v6.8-iommu-sva-part4-v7
Current XDNA on top of XRT for 6.8 Xilinx/XRT#8005 works with the examples.
To compile and install the kernel on Ubuntu:

git clone --depth=1 --branch v6.8-iommu-sva-part4-v7 [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/74c24ec7a989abf1a30504642fccaaeb/raw/f2825cdf702f5551e81ea41356d8e0dd6dc827d8/.config
make -j `nproc` bindeb-pkg
sudo apt reinstall ../linux-headers-6.8.0+iommu-sva-part4-v7+_6.8.0-00104-g3c67b10dc48b-3_amd64.deb ../linux-image-6.8.0+iommu-sva-part4-v7+_6.8.0-00104-g3c67b10dc48b-3_amd64.deb ../linux-libc-dev_6.8.0-00104-g3c67b10dc48b-3_amd64.deb

[keryell]

@hegdevasant it looks like the simplest solution is to give me a write access to either https://github.com/AMDESE/linux-iommu or https://github.com/AMDESE/linux to host my rebased branch for real™ normal™ users™ of AMD products.
Then I will push some branches on top if the latest stable kernel up to the point that everything is up-streamed.

[keryell]

As Linux 6.8.2 is out with a few AMD patches which might be useful, I have rebased the kernel to be used as:

git clone --depth=1 --branch v6.8.2-iommu-sva-part4-v7 [email protected]:keryell/linux.git
cd linux
wget https://gist.github.com/keryell/36cc169c5e948382302b990550fe2f08/raw/2f435d86559804b60c7eeeeee1464d7f514b95de/.config
make -j `nproc` bindeb-pkg
sudo apt reinstall ../linux-headers-6.8.2+iommu-sva-part4-v7+_6.8.2-00094-g70d90b9ab1a9-1_amd64.deb ../linux-image-6.8.2+iommu-sva-part4-v7+_6.8.2-00094-g70d90b9ab1a9-1_amd64.deb ../linux-libc-dev_6.8.2-00094-g70d90b9ab1a9-1_amd64.deb

This will work directly with 758e848 and https://github.com/Xilinx/XRT since Xilinx/XRT#8005 has been merged.
Working with current xdna-driver would require #31 to be addressed.

[keryell]

I have pushed my branches on https://github.com/AMD-SW/linux and
https://github.com/AMD-SW/linux/tree/v6.8.2-iommu-sva-part4-v7%2Bpmc-10ms-delay works rather well after 1 week.

[keryell]

There is now a stable Linux version with required patches and XRT+XDNA have been updated accordingly.