← 输入框|github.com|刷新
Skip to content

[python] Give the Delta snapshot tests' token waits the sinks' 60 s - #7852

Merged
bestouff merged 1 commit into
mainfrom
issue-7654
Nov 5, 2026
Merged

bestouff merged 1 commit into
mainfrom
issue-7654

Conversation

@bestouff

@bestouff bestouff commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #7654.

A completion token is complete only once every output connector has processed the resulting changes (crates/connectors/src/controller/stats.rs:110). In CI the Delta sinks of test_output_snapshot.py write to an object store, so the two 30 s token waits also covered Parquet writes and Delta log commits over the network, while the sink polls right next to them already allow 60 s.

Wait Before After
push() helper in test_egress_send_snapshot 30 s 60 s
input_json in test_delta_output_send_snapshot_after_flag_flip 30 s 60 s
Sink polls in both tests 60 s unchanged

In the failing run the pipeline took 18 s to compile and 13 s to start before the token wait began, and the wait was still the only one in the file shorter than the work it measures.

A completion token is complete only once every output connector has
processed the resulting changes. In CI the Delta sinks of these tests
write to an object store, so the two 30 s token waits also covered four
Parquet writes and Delta log commits over the network, while the sink
polls right next to them already allow 60 s. On a slow amd64 runner the
token wait timed out after the pipeline had already taken 18 s to
compile and 13 s to start.

Fixes #7654.
@mythical-fred-oss

Copy link
Copy Markdown
Contributor

LGTM. This is a test-only change: it raises two completion-token waits from 30 s to 60 s, so they match the sink polls next to them that already allow 60 s.

  • Ran: pre-commit run --files python/tests/runtime/test_output_snapshot.py. ruff and ruff-format passed.
  • Not run: the runtime test itself. It needs a pipeline-manager and an object store.
  • Flaky-test gate: passes. The waits poll a token with a deadline, so there is no fixed sleep and a passing run doesn't get slower. The only 30 s wait left (endpoint removal, around line 153) doesn't include Delta commits, so it's fine to leave it.
  • Docs, breaking changes, deps: not affected.

@bestouff
bestouff requested a review from ryzhyk November 5, 2026 09:26
@bestouff
bestouff added this pull request to the merge queue Nov 5, 2026
Merged via the queue into main with commit 8dc6b9a Nov 5, 2026
3 checks passed
@bestouff
bestouff deleted the issue-7654 branch November 5, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runtime Integration Tests (amd64): test_delta_output_send_snapshot_after_flag_flip times out waiting for input processing under slow CI

2 participants