Fix OpenLineage emitting duplicate START events for rescheduled sensors - #73144
Merged
kaxil merged 3 commits intoSep 15, 2026
Merged
Conversation
On Airflow 3 a sensor in reschedule mode emitted a START event on every poke instead of only the first. The guard meant to prevent this tested a dict key with `hasattr`, which is always false on a `TypedDict`, so the early return was unreachable and the check never fired. Airflow 2 was unaffected because its hook returns on `is_ti_rescheduled_already()` before reaching the check. The Airflow 3 hook has no DB-backed equivalent, so the context field is its only reschedule signal. Restore the guard in the Airflow 3 hook, scoped the way its Airflow 2 sibling scopes it: to sensors in reschedule mode. `task_reschedule_count` counts rows written for any operator, and a missing-DAG startup failure writes them before any listener hook fires, so an unscoped check would suppress the START of an attempt that had emitted nothing yet. Guarding in the hook rather than inside the emission closure matches the Airflow 2 branch and avoids forking the task runner once per suppressed poke. That path already builds the template context, so the check adds no work.
kaxil
requested review from
Lee-W,
amoghrajesh,
kacpermuda and
vatsrahul1001
September 14, 2026 18:28
Lee-W
approved these changes
Sep 15, 2026
kacpermuda
approved these changes
Sep 15, 2026
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Airflow 3, a sensor in
reschedulemode emits an OpenLineage START event on every pokeinstead of only the first. The run id is derived from
try_numberand a reschedule does notincrement it, so all of those events carry the same run id: downstream consumers see one run
entering RUNNING over and over for the life of the sensor.
The listener already has a guard meant to stop exactly this, but it tests a dict key with
hasattr:contextisairflow.sdk.definitions.context.Context, aTypedDict, so at runtime it is aplain dict with no attribute of that name. The condition is always false and the
returnbelow it is unreachable. The mixed access on the line gives it away:
hasattrtreatscontextas an object, then the subscript treats it as a dict, and only the second is right.Airflow 2 is unaffected, which is why this went unnoticed. Its hook calls
is_ti_rescheduled_already()against the
task_rescheduletable and returns before the dead check is ever reached. TheAirflow 3 hook has no equivalent and cannot have one: that helper is defined under
if not AIRFLOW_V_3_0_PLUS:and needs a SQLAlchemy session against the metadata DB, which atask runner has no access to. The
hasattrline is the only reschedule protection Airflow 3has, and it has never fired. It arrived that way in #45294, which split the hook by Airflow
version and swapped the
is_ti_rescheduled_already()call inside the emission closure for thehasattrcheck, adding the working guard back only on the Airflow 2 branch.The revived guard is scoped to reschedule-mode sensors, not to the row count alone.
task_reschedule_countanswers "have rows been written for this task instance", which is notthe same question as "did we already emit a START".
_maybe_reschedule_startup_failure()writes those rows for any operator when a worker cannot see the DAG file, up to
[workers] missing_dag_retries(default 3), and it runs insidestartup()before theon_task_instance_runninghook fires. So a plainPythonOperatorthat hit bundle-sync lagreaches its first real attempt with a non-zero count and nothing emitted yet. A guard keyed on
the count alone would suppress that attempt's START and then emit a COMPLETE for a run that
never entered RUNNING, losing the RUNNING transition, the START
eventTimethat makes runduration computable, and the inputs that extractors report at start. The Airflow 2 sibling
gates on
isinstance(task, BaseSensorOperator)andtask.reschedulebefore it ever looks atthe table;
getattr(task, "reschedule", False)is the same test without the import, sincerescheduleis a property on the sensor base.The check sits in the hook rather than in the emission closure, where the broken one was.
That closure runs under
_execute(..., use_fork=True), so leaving the guard there forks thetask runner once per suppressed poke to do nothing. Guarding in the hook matches where the
Airflow 2 branch guards, and costs nothing extra because
get_dag_run_dag_and_task_from_ti()on the line above already builds and caches the template context on Airflow 3.
The regression test drives the real Airflow 3 path, a
RuntimeTaskInstancewith a liveTIRunContext, rather than the mocked-context helper in the same file. Nothing in the suitehad ever set the count above zero, so the guard was never exercised in its firing state. The
non-sensor row is what pins the scoping: with the count-only form of the check it fails.