Repository navigation
Tags
Tags
[dbsp] Check merged batches again after reopening them in merge propt… …ests The merge proptests checked a merged batch only through the reader the merge returned, which keeps the filter the merge built, so they could not catch a filter that decodes differently once a restart reopens the file. Both runners now reopen the merged batch from its file, as a restore does, and repeat the content and `seek_key_exact` checks on the reopened batch. Weight cancellation and the `retain_above` key filter often drop the smallest input key, the case that reloading used to get wrong. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com> (cherry picked from commit c9d0920) Backport note: on this release line an empty batch reopened from its file has no membership filter, while the merge reports the kind it would have built. The reopened-filter-kind checks therefore skip empty merged batches. An empty batch answers every probe correctly without a filter.
[manager] Do not relay an upstream `Content-Length` in `streaming_pro… …xy()`. From actix-web 4.12, `HttpResponseBuilder::streaming()` sends a sized body, not a chunked stream, if the response already has a `Content-Length`. `streaming_proxy()` copied every upstream header, so a multihost pipeline's circuit profile, which the coordinator sends with a length, came back sized. This made `test_circuit_profile_streaming` fail on multihost pipelines (feldera/cloud#2230). Now `streaming_proxy()` drops the upstream `Content-Length`, as the buffered proxy path already does. Signed-off-by: Ben Pfaff <blp@feldera.com>
[qa] Delete only stale topics in the Kafka Avro test's setup The setup read the variant id where it expected a topic's creation date, so it deleted every kafka_avro topic on the cluster, including those of runs still in progress against it, such as the other architecture's job. It now reads the date that each topic name carries. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
Preserve literals in prebuilt binary cache keys Blind namespace replacement can change SQL literals and conflate distinct programs. Normalize generated Rust identifiers and Cargo crate references structurally, including workspace members, while preserving literal data. Reject duplicate normalized archive paths instead of overwriting them. Cover namespace independence, literal collisions, Cargo references and malformed generated sources. The four prebuilt cache tests pass, and real SQL compiler output normalizes identically across pipeline IDs.
[pipeline-manager] Stream the support bundle while it is collected. The support bundle handler sent nothing until it had collected all data and built the whole ZIP file. A slow collection, such as the profile of a large multihost pipeline in a long commit, kept the response silent for minutes, so a proxy or the client closed it first. Now the response starts when the request is valid. When the bundle collects new data, its first entry is collection_log.txt, which gets a line every 10 s until the collection is complete. Then the collections and the metadata follow, each sent when it is written. The ZIP file uses data descriptors, which the profiler UI, unzip, Python, and the zip crate read. A failure after the response starts now cuts the response short instead of setting an error status. The collection itself reports failures inside the bundle, as before. The API docs no longer list 503 examples for a failed pipeline request, because the endpoint reports such failures inside the bundle. They describe collection_log.txt and the missing Content-Length. The operations guide and the changelog describe the change too. Part of feldera/cloud#2163. Signed-off-by: Ben Pfaff <blp@feldera.com>
[connectors] Assert the early health check only before the threshold The timer that reports a long retry loop unhealthy starts at the chunk's first retry, which is after the test calls `write_batch`. So an instant taken before that call bounds the timer: while less than the threshold has passed since then, no report can exist yet, and the assertion holds by construction. A runner that stalls past the threshold skips the check instead of failing it, so the test no longer races a wall clock. Refs #7828.
PreviousNext