Skip to content

Fix LivyAsyncHook sending POST to Livy GET only endpoints - #71867

Merged
dabla merged 6 commits into
apache:mainfrom
Jie211:fix-livy-async-get-methods
Sep 16, 2026
Merged

dabla merged 6 commits into
apache:mainfrom
Jie211:fix-livy-async-get-methods

Conversation

@Jie211

@Jie211 Jie211 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

LivyAsyncHook.get_batch_state() and get_batch_logs() call run_method() without specifying method,
so they fall back to self.method, which LivyAsyncHook.__init__ hardcodes to "POST".

Livy only accepts GET on /batches/{id}/state and /batches/{id}/log,
so both requests fail with 405 Method Not Allowed.

If i use deferrable mode, the batch is submitted, but the first poll from the trigger fails.

    WARNING  [Try 1 of 3] Request to https://<livy>:8998/batches/<id>/state failed.
    {"Response": ["Method Not Allowed"], "Status Code": [405], "status": "error"}
    AirflowException: Batch <id> did not succeed with 'str' object has no attribute 'value'

Maybe this is come from this PR: #60458


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Jie211
Jie211 marked this pull request as ready for review August 20, 2026 09:48

@Nataneljpwd Nataneljpwd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@eladkal

eladkal commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

cc @dabla for review after #60458

@dabla
dabla merged commit 1fbaea8 into apache:main Sep 16, 2026
79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants