Repository navigation
[connectors,dbsp] Use neutral names in tests that copied a reported workload - #7950
Merged
Merged
Conversation
The nested `after` struct of the column-mapping tests used column names copied from a table in a problem report. The tests need only the structure of that table: a nested struct whose children are column-mapped. The children now have neutral names (`item_id`, `vendor_id`, `vendor_name`, `event_time`, `status`, `amount`), and the comments describe the shape without its origin. The fixture versions are bumped, because a cached fixture is found by its path alone. The Unity Catalog fixture is a table that someone makes by hand: run `tests.platform.fixtures.unity_uniform` again and point DELTA_TABLE_TEST_UNITY_UNIFORM_TABLE at `uniform_iceberg_v2`. Signed-off-by: Xavier Bestel <xavier.bestel@feldera.com>
Contributor
|
LGTM. This PR only renames test fixtures. The fixture versions are bumped and no
|
The comment of gate_measures_only_the_collections_it_changes named the pipeline and the join that the costs came from. The test needs only the costs, so the comment now describes a cluster of 25 streams and a skewed join. Signed-off-by: Xavier Bestel <xavier.bestel@feldera.com>
lalithsuresh
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The nested
afterstruct of the Delta column-mapping tests used column names copied from a table in a problem report. The tests need only the structure of that table: a nested struct whose children are column-mapped. This PR gives the children neutral names and removes the origin from the comments.transaction__iditem_idtransaction__merchant_idvendor_idtransaction__merchant_namevendor_nametransaction__timeevent_timetransaction__statusstatustransaction__amountamountThe fixture versions are bumped, because a cached fixture is found by its path alone.
The second commit changes one comment in
crates/dbsp/src/operator/dynamic/balance/balancer.rs. The comment named the pipeline and the join that the test costs came from. It now describes a cluster of 25 streams and a skewed join. No code changes.Action before merge
The Unity Catalog fixture is a table that someone makes by hand.
test_delta_input_uniform_unity.pyfails in an environment that still points at the old table. To update it, from thepythondirectory:Then set
DELTA_TABLE_TEST_UNITY_UNIFORM_TABLEto theuniform_iceberg_v2name that the script prints.Testing
cargo test -p dbsp_connectors --lib project_to_logical_matches_by_field_idpasses.fixtures/uniform_iceberg.pybuilds the table locally, and the Parquet schema has the new names.