← 输入框|github.com|刷新
Skip to content

[python] Test that a support bundle starts during a slow circuit step - #7935

Merged
blp merged 1 commit into
mainfrom
support-bundle-slow-step-test
Nov 11, 2026
Merged

blp merged 1 commit into
mainfrom
support-bundle-slow-step-test

Conversation

@blp

@blp blp commented Nov 11, 2026

Copy link
Copy Markdown
Member

Regression test for feldera/cloud#2163, which #7818 and feldera/cloud#2164 fixed.

A scalar UDF sleeps 45 s, so one circuit step stays busy. During that step the test requests a support bundle and asserts:

  1. The first byte arrives within 15 s.
  2. The step is still running when the first byte arrives, so the test really checks something.
  3. The full bundle has circuit_profile.json.gz with worker_profiles.

The CI python-multihost suite runs it with 2 hosts as well.

Validation

Server Result
v0.375.0 (before #7818), local k3d fails: first byte after 43.1 s
v0.379.0 or later not run locally. CI checks it

The UDF uses the scalar (Gen-1) form, so the test also compiles on servers from before Gen-2 UDFs.

-- Claude Code, at @blp's request.

@blp blp added QA Testing and quality assurance python Pull requests that update python code multihost Related to multihost or distributed pipelines Support Bundle Things that concerns support bundles collections labels Nov 11, 2026
@blp blp self-assigned this Nov 11, 2026
f"first byte after {first_byte_secs:.1f} s"
)
assert (pipeline.stats().global_metrics.total_processed_records or 0) == 0, (
"the step completed before the bundle started, so this test checks nothing"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gap, medium: processed == 0 is also true if the step has not started yet, for example if the sleep(2) on line 63 is too short on a busy CI host. In that case dump_json_profile returns at once, the first byte comes early, and the test passes without checking anything. A lower bound on the total download time shows that the bundle really waited for the step. This bound is safe because the bundle cannot finish before the step does:

bundle = first + b"".join(chunks)
total_secs = time.monotonic() - start
assert total_secs > STEP_SECS / 2, f"bundle finished after {total_secs:.1f} s, so it did not wait for the step"

Once this check is in place, the sleep(2) only makes it more likely that the step has started. Please also add a comment there explaining why it is needed (zero-tolerance rule on unexplained sleeps).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: a bundle requested before the step starts got its first byte early too, so the test passed without checking anything. ca87781 adds the check total_secs > STEP_SECS / 2 and a comment on the sleep(2).

To confirm, I ran a copy of the test with no record inserted, so no step was running. It failed with bundle complete after 0.3 s, so it did not wait for the step.

-- Claude Code, at @blp's request.

@mythical-fred-oss

Copy link
Copy Markdown
Contributor

Review summary (test-only PR, adds a regression test for streaming the support bundle)

Check Result
pre-commit run --files python/tests/platform/test_support_bundle.py (ruff, ruff-format) passed
Run the test itself not run: it needs pipeline-manager and the SQL compiler
Server timeouts checked against the 45 s step the manager's COLLECTION_TIMEOUT is 120 s, and collection_log.txt gets a line every 10 s, so neither the profile fetch nor the client's read timeout should trip
Flaky-test gate the timing thresholds (first byte < 15 s, step 45 s) leave plenty of margin
  • Gap, medium: the test can pass without testing anything if the step has not started when the bundle is requested (see the inline comment). Asserting a lower bound on the total download time fixes this and keeps the test deterministic.
  • Cost, low: the test spends about 45 s in the sleeping UDF on top of the UDF compile. That is fine for platform tests, but keep STEP_SECS as low as the margins allow.

LGTM once the inline comment is addressed.

# processes a record.
STEP_SECS = 45

# A bundle that starts later than this waited for the step.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not obvious to me that this is true.
the converse is false, though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. A late first byte can have other causes, and an early one does not prove that the step was running. The comment claimed more than the test checks. ca87781 changes it to say only what the limit is: the bundle sends bytes while it collects, so its first byte must not wait for the step.

The "was the step running" question now has its own check: the whole bundle must take more than STEP_SECS / 2.

-- Claude Code, at @blp's request.

A UDF that sleeps 45 s keeps one circuit step busy. A support bundle
requested during that step must send its first bytes within 15 s, and
must still hold the circuit profile when the step completes. The test
also checks that the bundle took more than half of the step time, so
that it fails if the bundle started before the step did.

Before #7818, the bundle sent nothing until it had all
of its data, so a proxy could close the idle connection
(feldera/cloud#2163). On v0.375.0 this test fails: the first byte
arrived after 43.1 s.

The CI multihost suite runs this test with 2 hosts too.

Signed-off-by: Ben Pfaff <blp@feldera.com>
@blp
blp force-pushed the support-bundle-slow-step-test branch from ca87781 to 7f5ac0c Compare November 11, 2026 17:43
@blp
blp added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 2b13a7c Nov 11, 2026
@blp
blp deleted the support-bundle-slow-step-test branch November 11, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

multihost Related to multihost or distributed pipelines python Pull requests that update python code QA Testing and quality assurance Support Bundle Things that concerns support bundles collections

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants