Indicating why each fixture archive in `.gitignore` is untracked

[EliahKagan]

Summary 💡

Background

Many generated archives of Git repository fixtures produced by gix-testtools running fixture scripts are committed, which allows them to be reused on systems that may be unable to produce them, and also facilitates more robust testing on systems that can produce them, since we can verify that the test works both with a fixture generated on another machine and a fixture generated on the current machine.

For tests whose associated fixtures intentionally incorporate information about the current system, it would be incorrect to track the archives. In some other cases, it would be ideal to track the archives but we can't yet do so because of limitations in the test suite or fixture scripts that could eventually be overcome. Therefore various archives are listed in .gitignore files.

Proposal

I think we should somehow keep track of why each fixture in any .gitignore file is listed there, because:

• Some may not work across systems due to bugs in the fixture, other test code, or even in the code under test. Keeping track of what we know about why a fixture is in .gitignore would help reveal this.
• For others, it may be possible to impove them so they can be committed. This would lead to a more robust test suite, for the reason described above.

Motivation 🔦

Conceptual clarity

Besides the points above, I think documenting this would make it easier to reason about the relationship between repositories maintained in fixtures and the test code that uses those repositories. (This is a lesser rationale than the above, but it is also what motivated me to suggest that we do this.)

The fixture accompanying the regression test for #2227 introduced in #2229 is in .gitignore. The correct configuration that should be written when doing a shallow clones depends only on the repository being cloned, but I think this was made harder to reason about by the test repository always being created on the machine running the test.

That fixture is reused across various tests and there may also be good reasons for it to be created anew each time. Furthermore, in this case I don't think using a committed archive would have caused the test to fail in a way that would have revealed #2311. (If it would have, then that would indicate a bug in gix-testtools not adequately sanitizing/resetting the configuration when running the fixture scripts--though the ability to reveal such bugs is another benefit.) Rather, it's only the effect of making clear what is and isn't expected to vary based on what system generates the fixtures that I think might've helped.

Plan

If there's no objection, then I'll eventually try to do this, though I'm not sure when, which is why I'm opening this issue for it so I don't forget or in case someone else wants to take it on.

[Byron]

Thanks a lot for bringing this up, there is absolutely no object to document the exceptions marked via .gitignore files, and I am looking forward to seeing this work happen eventually.
Maybe this could even be automated with the right prompt and a little preparation. If one would remove the ignores, create the archives on macOS or Windows, and then run them on Linux, they should fail. Copilot could be instructed to fix this by adding the ignore lines back, along with a concise description of the reasoning behind it. It will still be some work to review and validate, but maybe it does save time or allows to focus on what matters.