chore: stop tracking simulator-regenerated datasets#137
Merged
Conversation
Untrack the ~220 files under dataset/ that are regenerated by the
simulator scripts in scripts/{imaging,interferometer,multi,group,
cluster,point_source,weak}/simulator.py and friends. They were
already covered by the dataset/ rule in .gitignore but had been
committed before that rule existed, so git kept tracking them and
test runs left them perpetually dirty (currently 143 staged files
+ more unstaged).
Kept tracked (real observational data and simulator inputs):
- dataset/imaging/cosmos_web_ring/ (JWST COSMOS-Web; loaded by
scripts/imaging/start_here.py and scripts/multi/start_here.py)
- dataset/imaging/slacs1430+4105/ (HST SLACS lens; used by plot
guides under scripts/guides/plot/examples/)
- dataset/group/102021990_NEG650312660474055399/ (real catalog
object; loaded by scripts/group/start_here.py)
- dataset/interferometer/uv_wavelengths/sma.fits (synthetic SMA
uv-coverage read by interferometer + multi/imaging_and_inter
simulators and modeling guides)
- dataset/imaging/los_halos/los_halo_list.npy + los_sheet_values.npy
(line-of-sight halo data read by los_halos/simulator.py)
- dataset/.gitignore (the inner * + !.gitignore marker)
The dataset/ rule is rewritten as dataset/** plus !-pins for each
preserved path so the maintainer's intent is documented in
.gitignore. Note: the inner dataset/.gitignore (* + !.gitignore)
takes precedence for NEW files, so adding new real-data files in
the future requires `git add -f`. The !-pins are decorative for
new files but accurate for the currently-tracked set.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dataset/that are regenerated by simulators inscripts/{imaging,interferometer,multi,group,cluster,point_source,weak}/simulator.pyand friends. They were perpetually dirty ingit status(143 staged + many unstaged).dataset/rule in.gitignore, but had been committed before the rule existed, so git kept tracking them.dataset/imaging/cosmos_web_ring/— JWST COSMOS-Web; loaded byscripts/imaging/start_here.py,scripts/multi/start_here.pydataset/imaging/slacs1430+4105/— HST SLACS lens; used byscripts/guides/plot/examples/dataset/group/102021990_NEG650312660474055399/— real catalog object; loaded byscripts/group/start_here.pydataset/interferometer/uv_wavelengths/sma.fits— synthetic SMA uv-coverage; read byscripts/multi/features/imaging_and_interferometer/simulator.pyand several modeling/aggregator scriptsdataset/imaging/los_halos/los_halo_list.npy+los_sheet_values.npy— line-of-sight halo data; read byscripts/imaging/features/advanced/los_halos/simulator.pydataset/.gitignore— the inner* + !.gitignoremarker.gitignorechangeReplaces:
with:
This documents which datasets are deliberately tracked. Adding new real data later:
git add -f dataset/path/to/data.fitsand add a matching!dataset/...line to.gitignore. (The innerdataset/.gitignorestill ignores everything by default, so-fis required for new files in keep-dirs.)Effect
git statuswill go silent for the simulator-output dataset tree.start_here.pytutorials work out of the box.dataset/imaging/simple/); runningpython scripts/imaging/simulator.pyregenerates them.Test plan
dataset/imaging/cosmos_web_ring/,slacs1430+4105/,102021990_NEG.../,uv_wavelengths/sma.fits, andlos_halos/*.npyare all presentpython scripts/imaging/simulator.py, confirmdataset/imaging/simple/regenerates andgit statusstays cleanpython scripts/imaging/start_here.py(loads cosmos_web_ring), confirm it still finds the data🤖 Generated with Claude Code