Prevent glean metadata overrides form overwriting default metadata #280
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
akkomar
reviewed
Aug 21, 2024
|
I looked through mozilla-services/mozilla-pipeline-schemas#827 and the new settings look to match probe-scraper. The only issue is that the firefox ios sync pings were added exactly 400 days ago and the default retention for the app is 400 so partitions would start getting dropped from this change. We should manually set the retention for those in probe scraper before making this fix. I commented on each of the affected pings in the other pr |
whd
approved these changes
Aug 22, 2024
|
https://github.com/mozilla/probe-scraper/pull/795/files will land today after which I think it's safe to propagate this. |
|
I'll merge this and rerun schema generator and make sure it's working before the end of the day |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#266 introduced a bug where the metadata for a ping set at the app-level might overwrite the app defaults in
moz_pipeline_metadata_defaults. This is becauseGleanPing.apply_default_metadatais called twice and the first call puts thedefault_metadatadict in the ping and then the second call will update that dict in place so thedefault_metadatawill then have new values.For example, this can be seen in
pine.deletion-requestwhere the retention is set to 1130 but it's set to 10000 in probe-scraper.I made a pr against the generated schemas to show the changes this would make on the schemas mozilla-services/mozilla-pipeline-schemas#827. I'll go through those and make sure those are correct and won't unexpectedly delete things. All the changes are decreases in retention period except for pine deletion-requests and pageload which means that this bug shouldn't have deleted anything unexpectedly.