Repository navigation
[python] Test that a support bundle starts during a slow circuit step - #7935
Conversation
| f"first byte after {first_byte_secs:.1f} s" | ||
| ) | ||
| assert (pipeline.stats().global_metrics.total_processed_records or 0) == 0, ( | ||
| "the step completed before the bundle started, so this test checks nothing" |
There was a problem hiding this comment.
Gap, medium: processed == 0 is also true if the step has not started yet, for example if the sleep(2) on line 63 is too short on a busy CI host. In that case dump_json_profile returns at once, the first byte comes early, and the test passes without checking anything. A lower bound on the total download time shows that the bundle really waited for the step. This bound is safe because the bundle cannot finish before the step does:
bundle = first + b"".join(chunks)
total_secs = time.monotonic() - start
assert total_secs > STEP_SECS / 2, f"bundle finished after {total_secs:.1f} s, so it did not wait for the step"Once this check is in place, the sleep(2) only makes it more likely that the step has started. Please also add a comment there explaining why it is needed (zero-tolerance rule on unexplained sleeps).
There was a problem hiding this comment.
Agreed: a bundle requested before the step starts got its first byte early too, so the test passed without checking anything. ca87781 adds the check total_secs > STEP_SECS / 2 and a comment on the sleep(2).
To confirm, I ran a copy of the test with no record inserted, so no step was running. It failed with bundle complete after 0.3 s, so it did not wait for the step.
-- Claude Code, at @blp's request.
|
Review summary (test-only PR, adds a regression test for streaming the support bundle)
LGTM once the inline comment is addressed. |
| # processes a record. | ||
| STEP_SECS = 45 | ||
|
|
||
| # A bundle that starts later than this waited for the step. |
There was a problem hiding this comment.
it's not obvious to me that this is true.
the converse is false, though.
There was a problem hiding this comment.
You are right. A late first byte can have other causes, and an early one does not prove that the step was running. The comment claimed more than the test checks. ca87781 changes it to say only what the limit is: the bundle sends bytes while it collects, so its first byte must not wait for the step.
The "was the step running" question now has its own check: the whole bundle must take more than STEP_SECS / 2.
-- Claude Code, at @blp's request.
A UDF that sleeps 45 s keeps one circuit step busy. A support bundle requested during that step must send its first bytes within 15 s, and must still hold the circuit profile when the step completes. The test also checks that the bundle took more than half of the step time, so that it fails if the bundle started before the step did. Before #7818, the bundle sent nothing until it had all of its data, so a proxy could close the idle connection (feldera/cloud#2163). On v0.375.0 this test fails: the first byte arrived after 43.1 s. The CI multihost suite runs this test with 2 hosts too. Signed-off-by: Ben Pfaff <blp@feldera.com>
ca87781 to
7f5ac0c
Compare
Regression test for feldera/cloud#2163, which #7818 and feldera/cloud#2164 fixed.
A scalar UDF sleeps 45 s, so one circuit step stays busy. During that step the test requests a support bundle and asserts:
circuit_profile.json.gzwithworker_profiles.The CI
python-multihostsuite runs it with 2 hosts as well.Validation
first byte after 43.1 sThe UDF uses the scalar (Gen-1) form, so the test also compiles on servers from before Gen-2 UDFs.
-- Claude Code, at @blp's request.