Skip to content

Replace generic AirflowException in AzureBatchOperator input validation - #71981

Merged
potiuk merged 1 commit into
apache:mainfrom
baha-bouali:follow-up-exception-file
Sep 8, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
baha-bouali:follow-up-exception-file

Conversation

@baha-bouali

@baha-bouali baha-bouali commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #71071

CONTEXT: get the operators/batch.py entry in generated/known_airflow_exceptions.txt removed entirely.

What changed

AzureBatchOperator._check_inputs() had 8 bare raise AirflowException(...) calls. Each now raises a specific exception instead, defined in a new providers/microsoft/azure/exceptions.py:

Condition Exception
vm_publisher not provided AzureBatchVmPublisherMissingError
use_latest_verified_vm_image_and_sku without a complete image spec AzureBatchLatestImageSpecIncompleteError
vm_publisher without vm_sku/vm_offer/vm_node_agent_sku_id AzureBatchVmImageSpecIncompleteError
Neither target_dedicated_nodes nor enable_auto_scale AzureBatchPoolSizingMissingError
enable_auto_scale combined with explicit node counts AzureBatchPoolSizingConflictError
enable_auto_scale without auto_scale_formula AzureBatchAutoScaleFormulaMissingError
batch_job_release_task without batch_job_preparation_task AzureBatchJobPreparationTaskMissingError
Required pool/job/task parameters missing AzureBatchRequiredParametersMissingError

All 8 subclass AirflowException, so existing except AirflowException handling around this operator is unaffected. Message strings are unchanged — only the exception types differ.

The allowlist entry is now gone rather than reduced: the checker deletes the line when the count reaches 0.

image

Testing

  • New tests/unit/microsoft/azure/test_exceptions.py: 24 cases covering inheritance, raising, and catchability as AirflowException.
  • operators/test_batch.py: the 19 existing tests passed unmodified after the change, confirming base-class compatibility. They're now tightened to assert the specific exception, and 4 previously untested validation branches have coverage: 23 tests total.
Was generative AI tooling used to co-author this PR?
  • Yes
  • Assited by Claude Sonnet 5. Reviewed, written and tested by the submitter

Signed-off-by: baha-bouali <baha.bouali2020@gmail.com>
@potiuk
potiuk force-pushed the follow-up-exception-file branch from eeafb36 to 950a94f Compare September 8, 2026 22:01
@potiuk
potiuk merged commit a4320d9 into apache:main Sep 8, 2026
156 of 157 checks passed
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Sep 11, 2026
…on (apache#71981)

Signed-off-by: baha-bouali <baha.bouali2020@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants