-
Notifications
You must be signed in to change notification settings - Fork 124
Metadata label and CI validation for table levels based on medallion architecture #8522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
… medallion architecture.
akkomar
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requesting changes because of existing approval with test failures.
| ] | ||
|
|
||
| if "level" in metadata: | ||
| level = metadata["level"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if you follow Sean's suggestion you could convert to AssetLevel here:
| level = metadata["level"] | |
| level = AssetLevel(metadata["level"]) |
This way invalid values fail at parse time and you don't need a separate validator above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very tidy, though I'd personally prefer the detailed debugging messages in the validator, in case of CI failures.
| return True | ||
|
|
||
|
|
||
| def validate_asset_level(query_dir, metadata): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about missing owners?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my opinion, it makes sense to require owners, at the minimum for level Gold.
Could you add it to Requirements for each level for which levels you think it makes sense?
Enables adding a metadata label to set the table level as one of [gold, silver, bronze].
Related Tickets & Documents
Reviewer, please follow this checklist