Skip to content

Fix EMR Serverless delete operator skipping deletion when deferrable - #73323

Merged
potiuk merged 2 commits into
apache:mainfrom
sgoel2be24-cyber:fix-emr-serverless-delete-deferrable
Sep 20, 2026
Merged

potiuk merged 2 commits into
apache:mainfrom
sgoel2be24-cyber:fix-emr-serverless-delete-deferrable

Conversation

@sgoel2be24-cyber

Copy link
Copy Markdown
Contributor

EmrServerlessDeleteApplicationOperator(deferrable=True) never deleted the application. The inherited stop step defers, so the code after super().execute() that calls DeleteApplication was never reached. The task then resumed in the delete operator's execute_complete with the stop trigger's success event, logged "deleted successfully" and went green, leaving the application in place (only stopped). The non-deferrable path was unaffected.

The stop operator now defers to an overridable stop_complete_method_name (default execute_complete, so its behaviour is unchanged) in both places it waits on EmrServerlessStopApplicationTrigger, including the force_stop path. The delete operator sets it to delete_stopped_application, which validates the stop event, then calls DeleteApplication and defers on EmrServerlessDeleteApplicationTrigger as before. Deferrable and non-deferrable modes now share the same delete code.

The existing test_delete_application_deferrable only asserted that TaskDeferred was raised, which is why it passed with the bug. It now also checks the trigger, the resume method and that delete isn't called before the stop completes. New tests cover the resume-then-delete step, the force_stop chain and a failed stop event. All of them fail on main.

Note: task instances already deferred on the stop trigger by an older provider version during an upgrade will still resume in execute_complete, since the stop and delete trigger events are identical and can't be told apart.

Tested locally:

  • pytest on the amazon provider's EMR operator and trigger tests: 166 passed
  • The reproduction script from the issue now shows stop → defer → delete → defer → complete
  • prek pre-commit stage passes, apart from check-template-fields-valid, which needs Docker and wasn't run locally (no template fields changed); mypy on emr.py passes

closes: #72123


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Sep 18, 2026

@vincbeck vincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks valid

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified end to end: on main the inherited execute() defers before delete_application is ever reached and the delete operator's own execute_complete then logs "deleted successfully" on the stop trigger's event; the stop_complete_method_name hook routes the resume to delete_stopped_application, so both the plain and force_stop deferrable chains now actually delete.

Checked specifically:

  • _delete_application carries the whole old tail of execute() verbatim; the raise AirflowException("Application deletion failed ...") is a pure relocation, not a new usage.
  • The only modified existing test (test_delete_application_deferrable) is strengthened, not loosened.
  • All three new tests fail on main (delete_stopped_application does not exist there), so they are real regression tests.
  • Non-deferrable path is unchanged and still covered by the pre-existing test_delete_application_* tests.

Smaller observations

  • Two nits inline (a comment that restates the parent, and one autospec). I'll push fixups for both so this can merge without another round-trip.
  • The upgrade-path caveat in the description is accurate and unavoidable; no change needed.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.

Comment thread providers/amazon/src/airflow/providers/amazon/aws/operators/emr.py Outdated
Comment thread providers/amazon/tests/unit/amazon/aws/operators/test_emr_serverless.py Outdated
sgoel2be24-cyber and others added 2 commits September 20, 2026 15:33
With deferrable=True the parent stop step defers, so the delete call placed
after super().execute() was never reached. The task then resumed in the
delete operator's execute_complete with the stop trigger's success event,
logged that the application was deleted and succeeded, silently leaving the
application in place.

closes: apache#72123
Review nits: the comment in delete_stopped_application now records why
super().execute_complete is used (this class overrides execute_complete
for the delete trigger's event), and the cancel_running_jobs patch in the
force_stop test uses autospec=True.

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the fix-emr-serverless-delete-deferrable branch from dd35b49 to 6baa30a Compare September 20, 2026 13:33
@potiuk
potiuk merged commit c5d7f60 into apache:main Sep 20, 2026
83 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EmrServerlessDeleteApplicationOperator never calls DeleteApplication when deferrable=True (silently leaks applications, task reports success)

3 participants