Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define the initial NixOS configuration of webforge in a flake with CI support #33

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

btlogy
Copy link
Member

@btlogy btlogy commented Jan 29, 2025

Duplicates #32 which implements #31 and adds support for CI with 99 extra LoC, which partly implements #34.

The CI supports is actually automating the 3 first commands described in #31:

nix flake check
nix build .#nixosConfigurations.webforge.config.system.build.toplevel
nixos-rebuild dry-activate --flake .#webforge --target-host [email protected]

The automated deployment will come later (with a lot more LoC).

Signed-off-by: Benoit Donneaux <[email protected]>
Signed-off-by: Benoit Donneaux <[email protected]>
@btlogy btlogy force-pushed the 31.initial-webforge-nix-with-ci branch from dab11ea to 4bfd3e9 Compare February 13, 2025 16:32
Co-authored-by: Florian Sesser <[email protected]>
Signed-off-by: Benoit Donneaux <[email protected]>
@btlogy btlogy force-pushed the 31.initial-webforge-nix-with-ci branch from 4bfd3e9 to efd74ff Compare February 13, 2025 16:39
Signed-off-by: Benoit Donneaux <[email protected]>
@btlogy btlogy requested a review from hacklschorsch February 13, 2025 16:44
@btlogy
Copy link
Member Author

btlogy commented Feb 13, 2025

@hacklschorsch was right: the initial way this PR was dealing with nix eval to dynamically create a matrix job was over-complicated for only a few host in this project...
We might find a better way to re-introduce this later (if ever needed).
But for now, we just repeat the sysname, hostname and domainname in nix and in GHA, and that's fine if it is easier to maintain.

Comment on lines +44 to +50
echo "matrix<<end_of_matrix" >> $GITHUB_OUTPUT
cat >> $GITHUB_OUTPUT <<-EOF
{"include":[
{"target":"webforge","hostname":"webforge","domain":"tahoe-lafs.org"}
]}
EOF
echo "end_of_matrix" >> $GITHUB_OUTPUT
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do the matrix and end_of_matrix markers do? I'm not seeing them anywhere else in the code.

I searched the GHA docs another, not believing there's no better way, but came up black. :/

Copy link
Member Author

@btlogy btlogy Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matrix is the name of the output and end_of_matrix is a marker unique enough to tell GHA when the output value starts and stops.

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#multiline-strings

Comment on lines +78 to +80
# collect garbage until Nix store size (in bytes) is at most this number
# before trying to save a new cache
gc-max-store-size-linux: 1073741824
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do if we can't collect enough garbage to land at 1 GB? delete stuff we might still need?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand from their own explanation, I believe the GC will mostly stop before the store reaches 1GB when there is nothing left to collect!?
Will the content left be useful enough, I do not know for sure...
But back when the caching was working (something recently changed), I'm pretty sure it was saving quite some time...
I'm afraid we will have to wait to test and dig again...

But since there is only 1 of 2 servers in this repository, I suppose we could try w/o this GC and may not fill the 10GB cache too fast.

Comment on lines +81 to +82
# do purge caches
purge: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the default not to purge them? That is weird

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what this purge achieve is to get rid of an entry of the cache we know to be already obsolete.
W/o this feature, the cache will be purge, but only based on the oldest entries.
This might be more useful if/when we would mix object in this cache (not only the nix store).
But even if you have 2 nixConfigurations here, for instance webforge and testgrid.
If you change and build 5 time the config for testgrid, you might want to purge those rapidly to maximize the chance to still have a cache hit for webforge which you may have not changed.

Comment on lines +83 to +88
# purge all versions of the cache
purge-prefixes: ${{ runner.os }}-Nix-${{ matrix.target }}-
# created more than 0 seconds ago relative to the start of the `Post Restore` phase
purge-created: 0
# except the version with the `primary-key`, if it exists
purge-primary-key: never
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we need that many lines to purge everything but a cache that actually fits?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like that.
As commented above, the idea is to purge the cache entry we know to be obsolete and keeping the others as long as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants