Dealing with the large size of the repository

[Ivan-Velickovic]

As we add more platforms and examples to the VMM repository, naturally the size of it will grow as we are storing Linux kernel images and root file systems. Currently it sits at ~500MB which can definitely be reduced with a variety of ways. However, even with reducing the sizes it will always be in the order of megabytes, which is far too large for something that is ultimately a library that you use within another project.

At the same time, we cannot expect everyone who wants to get started and actually run the VMM examples to build the Linux kernel and root file system themselves (even though we include the config files for reproducibility). So as a long-term solution I see two options, either we have some server that contains the binaries that people wget from or have the build system grab them automatically. Or we have a submodule that contains the artifacts and so if people want to use the examples they do git clone --recursive, for those that only want to use the library they do git clone. Not quite happy with either solution, open to suggestions.

At the time of writing the codebase is still changing a lot and so this issue won't be dealt with for a while.

[wom-bat]

I'm really hoping we can use distro-packaged images at some point, when the virtIO system is mature. Then only a few 'special' images will have to be packaged here.

[wom-bat]

I now have code to grab an image and extract kernel and initrd from it. Still working on how to integrate that.

[Ivan-Velickovic]

I have been working in this https://github.com/au-ts/libvmm/tree/slim branch and made some progress.

What kind of images are you extracting from?

[Ivan-Velickovic]

This has been partially done now, all images have been removed from the repository and now the repo is significantly smaller:

git clone git@github.com:au-ts/libvmm --single-branch

now takes up 7.5MB of disk space, rather than the previous ~1.5GB. Much more reasonable.

The main problem with this of course is that we had to rewrite the commit history in addition to transitioning to hosting the example images outside of the repo. The main consequence is branches now don't point to commits on main and that all the PRs got closed automatically by GitHub when I did the force push that rewrote the commit history.

I have started to revive all the open PRs but am still in the process of it, for now there exists an old_main branch that has the commit history before it was rewritten to not have any of the images. This means that while that branch exists, any people pointing to particular commits of libvmm e.g via a git submodule will still work.

However, I encourage everyone to move to 0.1.0 since it is tagged.

For updating existing branches, I recommend first rebasing them on old_main, and then cherry-picking the commits the branch adds on top of main.

I will leave old_main up for a month or a couple months but ultimately to have the whole repository take up a small amount of space, we will have to rebase every branch or delete any we don't intend to rebase.