Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remediated default behavior, added documentation #378

Merged
merged 3 commits into from
Mar 28, 2025

Conversation

jroach-astronomer
Copy link
Member

@jroach-astronomer jroach-astronomer commented Mar 6, 2025

#295 outlined a documentation change that would better outline the workflow for defining and using a "default" block when creating DAGs with DAG Factory. In its most basic use, a default block at the top of a YAML file would be used, followed by the DAG-definitions leveraging these default values.

During the documentation and testing process, it was discovered that the previously-expected behavior was not being implemented. This was due to this code (https://github.com/astronomer/dag-factory/blob/main/dagfactory/dagfactory.py#L136C1-L140C18), which completely overrode default_config, rather than just the default_args key-value pair.

To handle this, both a change to the logic above as well the appropriate unit-tests were written. This change allowed for the behavior in #295 to be completely implemented and documented.

… default values
@jroach-astronomer jroach-astronomer requested a review from a team as a code owner March 6, 2025 17:06
@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.04%. Comparing base (0f8aad3) to head (aaa5fe7).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #378   +/-   ##
=======================================
  Coverage   94.04%   94.04%           
=======================================
  Files          11       11           
  Lines         890      890           
=======================================
  Hits          837      837           
  Misses         53       53           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jroach-astronomer
Copy link
Member Author

cc: @pankajastro - ready for review!

Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jroach-astronomer , thanks for these contributions!

My understanding is that there are two types of changes in this PR:

  1. Linting improvements
  2. Fix/change of behaviour of default config in the YAML file

How did you come up with these linting changes? Was it using the project's pre-commit? It would be great if we separated these from the (2)

@jroach-astronomer
Copy link
Member Author

Hi @jroach-astronomer , thanks for these contributions!

My understanding is that there are two types of changes in this PR:

  1. Linting improvements
  2. Fix/change of behaviour of default config in the YAML file

How did you come up with these linting changes? Was it using the project's pre-commit? It would be great if we separated these from the (2)

@tatiana, that's right, these changes were made using pre-commit. What's the best way to separate these changes out?

@tatiana tatiana added this to the DAG Factory 0.23.0 milestone Mar 25, 2025
@jroach-astronomer
Copy link
Member Author

@pankajastro , I've resolved our conversations and made the changes you've requested.

@pankajastro
Copy link
Collaborator

Hi @jroach-astronomer, Look like there are some conflicts. Could you please resolve them?

@jroach-astronomer
Copy link
Member Author

Hi @jroach-astronomer, Look like there are some conflicts. Could you please resolve them?

Merge conflict resolved!

Copy link
Collaborator

@pankajastro pankajastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for improving it.

@pankajastro pankajastro merged commit ae99563 into astronomer:main Mar 28, 2025
68 checks passed
@pankajastro pankajastro mentioned this pull request Mar 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants