Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run migration test from AF2.10.5 to 3.0 #48512

Merged
merged 8 commits into from
Apr 8, 2025

Conversation

ephraimbuddy
Copy link
Contributor

@ephraimbuddy ephraimbuddy commented Mar 28, 2025

Improve the testing framework to validate database migrations from
Airflow 2.10.5 to 3.0, with verification in CI. This ensures smooth
upgrade paths for users.

Key fixes/Improvement:

  • Fix foreign key inconsistencies in the dataset-to-asset migration
  • Add support for different constraint naming patterns between ORM and migrations observed in Airflow 2
  • Handle database-specific edge cases for PostgreSQL and MySQL

This ensures databases can be successfully migrated regardless of whether they
were initially created through ORM or migration files.

closes: #45479

@ephraimbuddy ephraimbuddy added the full tests needed We need to run full set of tests for this PR to merge label Mar 28, 2025
@ephraimbuddy ephraimbuddy reopened this Mar 28, 2025
@ephraimbuddy ephraimbuddy force-pushed the data-migration branch 4 times, most recently from c0ce8cb to ca40517 Compare March 31, 2025 16:28
@ephraimbuddy ephraimbuddy force-pushed the data-migration branch 4 times, most recently from 3dbbe44 to 18db94e Compare April 7, 2025 15:31
Improve the testing framework to validate database migrations from
Airflow 2.10.5 to 3.0, with verification in CI. This ensures smooth
upgrade paths for users.

Key fixes/Improvement:
- Fix foreign key inconsistencies in the dataset-to-asset migration
- Add support for different constraint naming patterns between ORM and migrations observed in Airflow 2
- Handle database-specific edge cases for PostgreSQL and MySQL

This ensures databases can be successfully migrated regardless of whether they
were initially created through ORM or migration files.

closes: apache#45479
@phanikumv phanikumv merged commit 5f07a59 into apache:main Apr 8, 2025
96 checks passed
@phanikumv phanikumv deleted the data-migration branch April 8, 2025 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add migration test in CI that initializes Airflow 2.10, upgrades and runs 3.0 migration tests
5 participants