ImportedFile: use BigAutoField for primary key #9669
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We could disable search indexing while we do the migration, but I don't think that should be required, we have 11M records, but to migrate the SphinxDomain model it took 15 min, and we had ~56M. ```python In [7]: ImportedFile.objects.count() Out[7]: 11527437 ``` So some 3 min of not being able to index new versions doesn't seem bad... There are two things that could happen: - The query times out and we don't index that version. - The query waits till the migration is done, nothing gets lost. But if we disable search indexing we definitely won't index new versions. We don't use those models outside search indexing, so doc serving and such shouldn't be affected. ref #9492
humitos
approved these changes
Aug 9, 2023
|
We may want to wait until we cleanup this table, right? |
|
If everything works as expected, we can wait till we have a smaller table, yeah. |
|
So, we have 12M of records now :D, so we should probably wait till we have run a re-index, that will delete old files. |
|
We have more files now 🙃 What should we do here? |
We are still tracking index files for new projects, doesn't look like this number will be decreasing on its own (old projects triggering builds), so a manual re-index is needed. |
|
This seems worth doing if we think we should do it :) |
|
We are at 36% capacity on IDs for ImportedFile, it should take some years to fill up, but better deal with that now than later when things are on fire. |
ericholscher
approved these changes
Jan 29, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We could disable search indexing while we do the migration, but I don't think that should be required, we have 11M records, but to migrate the SphinxDomain model it took 15 min, and we had ~56M.
So some 3 min of not being able to index new versions doesn't seem bad... There are two things that could happen:
But if we disable search indexing we definitely
won't index new versions.
We don't use those models outside search indexing, so doc serving and such shouldn't be affected.
ref #9492
📚 Documentation previews 📚
docs): https://docs--9669.org.readthedocs.build/en/9669/dev): https://dev--9669.org.readthedocs.build/en/9669/