Fix deferrable Kubernetes 401s with a default exec-based kubeconfig

[rjgoyln]

Summary

AsyncKubernetesHook caches the kubeconfig it loads, and both the pod and job triggers keep that hook as a cached_property. This causes a problem with exec-based credentials such as aws eks get-token, which mint tokens that expire after about fifteen minutes. The cached token can therefore expire during a long-running deferrable task, causing subsequent Kubernetes API requests to fail with 401 Unauthorized.

#65212 fixed this for exec auth loaded through config_dict, kube_config, and kube_config_path, but missed the fall-through branch where kubernetes_asyncio loads the kubeconfig from ~/.kube/config or $KUBECONFIG.

This branch is used when:

• the operator does not set config_file,
• the worker does not set KUBECONFIG, and
• the connection does not provide a kubeconfig field.

This is also the typical setup when an EKS kubeconfig is mounted at the default location.

This PR applies the same caching decision to the default kubeconfig path: exec-based credentials are not cached, while static credentials remain cached. This allows the exec plugin to be invoked again and obtain a fresh token when the trigger polls later.

When $KUBECONFIG contains multiple existing files, the config is left uncached rather than reimplementing kubernetes_asyncio's merge rules. This adds one kubeconfig read per poll in that case.

In-cluster configuration continues to be cached because load_incluster_config() installs a refresh_api_key_hook that re-reads the rotated service account token.

Closes #61737

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

[potiuk]

The default-kubeconfig fall-through is the one branch #65212 missed, and this closes it the same way the other three already do.

I verified the new path resolution against kubernetes_asyncio's own semantics rather than taking it on trust: load_kube_config(config_file=None) falls back to KUBE_CONFIG_DEFAULT_LOCATION, which KubeConfigMerger then splits on the path separator, runs through expanduser, and filters by existence — the same three steps, in the same order, as _resolve_default_kubeconfig_path(). ENV_KUBECONFIG_PATH_SEPARATOR is os.pathsep on both platforms. So the exec detection reads exactly the file the loader used.

Also checked: the extracted _kubeconfig_file_uses_exec_auth() is behaviour-identical to the block it replaces (same read, same parse, same warning, same conservative True fallback), nothing else in the hunk was absorbed, no existing test was modified, and the two expected_cached=False cases cannot pass pre-fix, since the default branch set _config_loaded = True unconditionally. Treating an ambiguous default location as exec, rather than reimplementing the merge rules, is the right direction to be conservative in.

I rebased the branch on main — it was 457 commits behind — and pushed one wording fixup: _resolve_default_kubeconfig_path() returns a path and loads nothing, so its docstring now says so.

Smaller observations

• Three of the five new cases (default_kubeconfig_no_exec, ..._ignores_paths_that_do_not_exist, ..._expands_home_directory) assert _config_loaded is True, which is also the pre-fix result, so they pass on unmodified main. Worth keeping regardless — each flips if the expanduser or existence-filter logic regresses — just noting the revert-detectable signal lives in the two expected_cached=False cases.
• The tests patch async_config.KUBE_CONFIG_DEFAULT_LOCATION, which is the name _resolve_default_kubeconfig_path() itself reads, so this is right for what is being tested. Only worth knowing that the real load_kube_config() reads its own module-level copy, so the patch would stop steering anything if someone later un-mocks the loader in these tests.
• Optional follow-up, not for this PR: a multi-file $KUBECONFIG with purely static credentials is now never cached, so the kubeconfig is re-merged on every trigger poll. KubeConfigMerger(paths).config would give the merged view without reimplementing the merge rules, if that cost ever shows up.

---

This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.