Skip to content

Make the Snowpark container job operator tests collect and pass - #72701

Closed
rjgoyln wants to merge 2 commits into
apache:mainfrom
rjgoyln:fix/snowpark-containers-missing-itertools-import
Closed

rjgoyln wants to merge 2 commits into
apache:mainfrom
rjgoyln:fix/snowpark-containers-missing-itertools-import

Conversation

@rjgoyln

@rjgoyln rjgoyln commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

The operator test module has never run. Two failures stack up:

  • itertools.count is called without importing itertools, so collection fails and all 45 tests in the module error.
  • The mocked clock jumps 20s per read against a 10s timeout, so the poll loop is past its deadline on the first check and logs a status it never observed.

Pinning the three clock reads the loop actually makes fixes both: the timeout fires after exactly one DESCRIBE SERVICE, and an implementation that grows a fourth read fails loudly instead of silently shifting the timeline.

The operator needs no change: two real monotonic() reads are microseconds apart, so the pre-poll deadline check can only trip for a non-positive timeout.

Left for a follow-up: the mocked hook returns a constant RUNNING, so assert_called_once_with("RUNNING") separates "polled at least once" from "never polled" rather than the last observed status from an earlier one. Covering that needs a status sequence and a longer deadline.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

@boring-cyborg boring-cyborg Bot added area:providers provider:snowflake Issues related to Snowflake provider labels Sep 8, 2026
The test module calls itertools.count without importing itertools, so
every test in it errors at collection and the snowflake provider suite is
red on main.

The mocked clock is wrong on its own terms too: it jumps 20s per read
against a 10s timeout, so the poll loop is already past its deadline on
the first check and reports a status it never observed. Pinning the three
reads the loop actually makes keeps the timeline explicit, and fails
loudly rather than silently shifting if the implementation grows another
one.
@rjgoyln
rjgoyln force-pushed the fix/snowpark-containers-missing-itertools-import branch from 22c04d6 to b416667 Compare September 8, 2026 12:30
@rjgoyln
rjgoyln marked this pull request as ready for review September 8, 2026 12:33
@rjgoyln
rjgoyln requested a review from potiuk as a code owner September 8, 2026 12:33

@Andrushika Andrushika left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix!
Nit: maybe assert that DESCRIBE count in the test directly, since the description relies on it.

The pinned clock already makes a second poll impossible, since it would
need a fourth monotonic() read, but that is a side effect of the mock's
length rather than something the test states. A reader of the timeout
case should not have to re-derive it.
@rjgoyln

rjgoyln commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

LGTM, thanks for the fix!
Nit: maybe assert that DESCRIBE count in the test directly, since the description relies on it.

Good catch, thanks! I’ll add a direct assertion for the DESCRIBE call count.

)
@mock.patch("time.sleep")
@mock.patch("time.monotonic", side_effect=itertools.count(0, 20))
@mock.patch("time.monotonic", side_effect=[0, 5, 10])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, after a closer look I think I was wrong here. The itertools.count in the original suggestion was on purpose, to keep the patched clock from running out if anything else reads time.monotonic during the test: #70103 (comment)

So maybe keep the unbounded counter, just with a step that fits the 10s timeout, e.g. itertools.count(0, 5) (0, 5, 10) plus the missing import itertools. count(0, 20) hits the deadline on the first check, which is the other failure you found.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @potiuk since it was your suggestion in #70103, in case I read it wrong. wdyt?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification! Since this has been addressed in #72709, I’ll close this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Andrushika I’m thinking of opening a separate one to add test coverage for #72709, since it currently doesn’t have a regression test. What do you think?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say no harm in adding it, but if we don’t think the extra coverage is necessary, I’d rather avoid opening another PR just to create noise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I got it. Thank you so much!

@rjgoyln rjgoyln closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:snowflake Issues related to Snowflake provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants