Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move GitDagBundle to providers #47636

Merged
merged 14 commits into from
Mar 26, 2025

Conversation

ephraimbuddy
Copy link
Contributor

This commit moves GitDagBundle into the providers package. As part of the move, a new GIT provider is created and like operators, transfers etc, we now have bundles package. The bundle package is where the GitDagBundles lives.

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch 2 times, most recently from c33cd18 to c123c9b Compare March 12, 2025 12:49
@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch 3 times, most recently from 23fa0ba to 6bc6572 Compare March 13, 2025 12:43
@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

I dismissed the alerts from CodeQL as false positive.

For the compat tests - git provider should only be for airflow 3 so you should also add git to the list of excluded providers in src/airflow_breeze/global_constants.py (PROVIDERS_COMPATIBILITY_TESTS_MATRIX)

@ephraimbuddy
Copy link
Contributor Author

I dismissed the alerts from CodeQL as false positive.

For the compat tests - git provider should only be for airflow 3 so you should also add git to the list of excluded providers in src/airflow_breeze/global_constants.py (PROVIDERS_COMPATIBILITY_TESTS_MATRIX)

Thanks! Almost got stuck at that! @dstandish is also fixing false positive at #47685

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch 2 times, most recently from 0af257c to 927c584 Compare March 13, 2025 16:19
@jscheffl
Copy link
Contributor

Looks good in my view.

Shall we add some config (don't know where) to ensure this provider is installed per default in Airflow Docker images released?

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from 927c584 to d29890f Compare March 18, 2025 11:31
Copy link
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want this provider to be listed in the PRE_INSTALLED_PROVIDERS ?

@jscheffl
Copy link
Contributor

jscheffl commented Mar 18, 2025

Do we want this provider to be listed in the PRE_INSTALLED_PROVIDERS ?

Strong YES. I assume very many people will use this as git sync is used often as standard in helm chart as well...

(which contradicts a bit with #47927 :-D)

@potiuk
Copy link
Member

potiuk commented Mar 20, 2025

Strong YES. I assume very many people will use this as git sync is used often as standard in helm chart as well...

Hmm. I think it should be rather in https://github.com/apache/airflow/blob/main/prod_image_installed_providers.txt

@jscheffl
Copy link
Contributor

Strong YES. I assume very many people will use this as git sync is used often as standard in helm chart as well...

Hmm. I think it should be rather in https://github.com/apache/airflow/blob/main/prod_image_installed_providers.txt

Ups, okay, sorry, mis-conception on my side. Strong YES so that the install is in the Docker, don't need to be in the PIP Distribution directly.

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from d29890f to df69535 Compare March 21, 2025 10:14
@ephraimbuddy ephraimbuddy marked this pull request as ready for review March 21, 2025 10:17
@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from df69535 to fb44fda Compare March 21, 2025 11:45
@potiuk
Copy link
Member

potiuk commented Mar 22, 2025

I think you need to rebase now after airflow-core move and run this:

breeze release-management prepare-provider-documentation --reapply-templates-only git

That should regenerate all the "build" documentation in provider package using the latest templates.

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from fb44fda to 595c3c2 Compare March 24, 2025 06:41
@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch 2 times, most recently from fcf91e9 to 92eed23 Compare March 24, 2025 21:31
@ephraimbuddy
Copy link
Contributor Author

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch 2 times, most recently from 007040d to bb75e96 Compare March 25, 2025 21:36
@potiuk
Copy link
Member

potiuk commented Mar 25, 2025

One small thing to fix.

@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from bb75e96 to abef0e9 Compare March 25, 2025 22:31
@ephraimbuddy ephraimbuddy force-pushed the add-gitbundle-provider branch from 5045074 to a617218 Compare March 26, 2025 13:56
@ephraimbuddy ephraimbuddy merged commit 08d9dc2 into apache:main Mar 26, 2025
148 checks passed
@ephraimbuddy ephraimbuddy deleted the add-gitbundle-provider branch March 26, 2025 15:29
pankajkoti pushed a commit to astronomer/airflow that referenced this pull request Mar 28, 2025
* Move `GitDagBundle` to providers

This commit moves GitDagBundle into the providers package. As part of
the move, a new `GIT` provider is created and like `operators`, `transfers`
etc, we now have `bundles` package. The bundle package is where the `GitDagBundles`
lives.

* fixup! Move `GitDagBundle` to providers

* Update documentation

* Fix static checks

* add git to compatibility tests matrix

* Update doctree

* Update with recent changes

* Fix doc, update version and add provider to prod image preinstalled providers

* fixup! Fix doc, update version and add provider to prod image preinstalled providers

* Fix doc

* update breeze command hash

* Fix incompatible providers regex removal syntax

* Remove providers mistakenly added in pyproject.toml

* Update providers/git/pyproject.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

5 participants