-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: support lists in 'resolve_keyed_by' #608
base: main
Are you sure you want to change the base?
Conversation
5daf787
to
8c4f563
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like this breaks on python 3.8?
Moving this back to draft to get it out of review queues. |
Oops, I forgot about this PR.. I'll get it into shape soon |
8c4f563
to
4ed10f0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this worth an addition to the "Keyed By" section of docs/concepts/transforms.rst, or an example somewhere?
4ed10f0
to
89a1539
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The doc improvements here are fantastic!
), | ||
), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some tests for error / corner cases would be welcome. Eg:
container: {"a": [{"b": 1}, {"c": 2}]}
subfield: a.b
@transforms.add | ||
def resolve_artifact_paths(config, tasks): | ||
for task in tasks: | ||
resolve_keyed_by(task, "worker.artifacts.path", task["label"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to bikeshed on this too much, so this is certainly not a blocker, but I have to admit that I find it a bit unintuitive that .path
accesses multiple list items (in addition to access plain dict keys). I wonder if jq-style syntax would be preferable here? Eg: worker.artifacts[].path
?
task["label"], | ||
platform=platform, | ||
) | ||
yield task |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yield task | |
yield new_task |
I tested this in Gecko and it resulted in an identical full graph