← 输入框|github.com|刷新
Skip to content

[SQL] Remove unused recursive circuits - #7954

Merged
mihaibudiu merged 1 commit into
mainfrom
issue7946
Nov 13, 2026
Merged

mihaibudiu merged 1 commit into
mainfrom
issue7946

Conversation

@mihaibudiu

Copy link
Copy Markdown
Contributor

Fixes #7946.

@mihaibudiu
mihaibudiu marked this pull request as ready for review November 12, 2026 18:31
@mythical-fred-oss

Copy link
Copy Markdown
Contributor

LGTM. The fix is correct. FindDeadCode keeps a nested operator only when one of its outputs is reachable, so it is safe to drop the whole circuit in RemoveOperators. The size() > 0 check in CheckPersistentIds is a good guard against the sha256("") id collision.

What I ran

Check Result
./build.sh ✅
RecursiveTests (with the extra test below) ✅ 15/15
PersistentIdTests, MetadataTests, SerializationTests ✅ 95/95
pre-commit run --files <changed> ✅ (no hooks apply)

missing (medium): the issue's actual repro is not tested. issue7946 covers recursive views that no view reads. It does not cover the user's path, where PropagateEmptySources makes the views dead through a LEFT JOIN with an empty left side. It also does not check that the remaining circuit still runs. I ran this test locally and it passes:

+    @Test
+    public void issue7946LeftJoinEmpty() {
+        String sql = """
+                CREATE TABLE t(src BIGINT, dst BIGINT);
+                DECLARE RECURSIVE VIEW p1(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p1 AS SELECT src, dst FROM t UNION SELECT p1.src, t.dst FROM p1 JOIN t ON t.src = p1.dst;
+                DECLARE RECURSIVE VIEW p2(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p2 AS SELECT src, dst FROM t UNION SELECT p2.src, t.dst FROM p2 JOIN t ON t.src = p2.dst;
+                CREATE LOCAL VIEW empty_left AS
+                SELECT CAST(NULL AS BIGINT) AS id FROM (VALUES (1)) AS one(x) WHERE FALSE;
+                CREATE MATERIALIZED VIEW o AS
+                SELECT l.id, r.dst FROM empty_left AS l LEFT JOIN p1 AS r ON l.id = r.src
+                UNION ALL
+                SELECT l.id, r.dst FROM empty_left AS l LEFT JOIN p2 AS r ON l.id = r.src
+                UNION ALL
+                SELECT src, dst FROM t;""";
+        var ccs = this.getCCS(sql);
+        ccs.visit(new CircuitVisitor(ccs.compiler) {
+            @Override
+            public void postorder(DBSPNestedOperator operator) {
+                Assert.fail("Unexpected recursive circuit " + operator);
+            }
+        });
+        ccs.step("INSERT INTO t VALUES(1, 2);", """
+                 id | dst | weight
+                ------------------
+                 1  | 2   | 1""");
+    }

bug (medium), not caused by this PR: a related DeadCode crash. It happens when the second view in a mutual recursion is LOCAL and is read only through the back edge. The compiler fails with an NPE in CircuitCloneVisitor.typecheckBackEdges. It fails the same way with this PR's RemoveOperators change reverted, and it compiles when b is not LOCAL. My guess is that FindDeadCode does not follow the edge from a view declaration back to its view, but I have not traced it. Worth a follow-up issue:

CREATE TABLE t(src BIGINT, dst BIGINT);
DECLARE RECURSIVE VIEW a(src BIGINT, dst BIGINT);
DECLARE RECURSIVE VIEW b(src BIGINT, dst BIGINT);
CREATE LOCAL VIEW a AS SELECT src, dst FROM t UNION SELECT b.src, t.dst FROM b JOIN t ON t.src = b.dst;
CREATE LOCAL VIEW b AS SELECT src, dst FROM a;
CREATE MATERIALIZED VIEW o AS SELECT * FROM a;

@anandbraman anandbraman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test looks good. Thanks!

@mythical-fred-oss

Copy link
Copy Markdown
Contributor

Review: LGTM. The fix matches the root cause in #7946: RemoveOperators now drops a DBSPNestedOperator that is not in keep, and CheckPersistentIds adds a guard against empty nested circuits.

What I ran

Command Result
./build.sh ✅
mvn test -Dtest=RecursiveTests, with the 2 tests below added ✅ 16/16
mvn test -Dtest=CteToLocalViewsTests,IncrementalRecursiveTests,NonLinearRecursionTests,MultiCrateTests,CircuitGraphsTests,DeferCarriedColumnsTests,SerializationTests ✅ 58/58

Missing (medium): the test does not cover the issue's original repro. issue7946 only covers recursive views that nothing reads. The user's case was different: the views are read, but only through a LEFT JOIN whose left side is always empty, so they become dead only after PropagateEmptySources. There is also no case that removes one dead circuit and keeps a live one, or that checks results end to end. Both pass with this PR. Suggested tests:

Test diff
diff --git a/sql-to-dbsp-compiler/SQL-compiler/src/test/java/org/dbsp/sqlCompiler/compiler/sql/recursive/RecursiveTests.java b/sql-to-dbsp-compiler/SQL-compiler/src/test/java/org/dbsp/sqlCompiler/compiler/sql/recursive/RecursiveTests.java
index b8a5763..fa0a714 100644
--- a/sql-to-dbsp-compiler/SQL-compiler/src/test/java/org/dbsp/sqlCompiler/compiler/sql/recursive/RecursiveTests.java
+++ b/sql-to-dbsp-compiler/SQL-compiler/src/test/java/org/dbsp/sqlCompiler/compiler/sql/recursive/RecursiveTests.java
@@ -534,6 +534,63 @@ public class RecursiveTests extends BaseSQLTests {
         cc.visit(visitor);
     }
 
+    @Test
+    public void issue7946LeftJoinEmpty() {
+        // p1 and p2 are read, but only through a LEFT JOIN with an always-empty left side
+        String sql = """
+                CREATE TABLE t(src BIGINT, dst BIGINT);
+                DECLARE RECURSIVE VIEW p1(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p1 AS SELECT src, dst FROM t UNION SELECT p1.src, t.dst FROM p1 JOIN t ON t.src = p1.dst;
+                DECLARE RECURSIVE VIEW p2(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p2 AS SELECT src, dst FROM t UNION SELECT p2.src, t.dst FROM p2 JOIN t ON t.src = p2.dst;
+                CREATE LOCAL VIEW empty_left AS
+                SELECT CAST(NULL AS BIGINT) AS id FROM (VALUES (1)) AS one(x) WHERE FALSE;
+                CREATE MATERIALIZED VIEW o AS
+                SELECT l.id, r.dst FROM empty_left AS l LEFT JOIN p1 AS r ON l.id = r.src
+                UNION ALL
+                SELECT l.id, r.dst FROM empty_left AS l LEFT JOIN p2 AS r ON l.id = r.src
+                UNION ALL
+                SELECT src, dst FROM t;""";
+        var ccs = this.getCCS(sql);
+        ccs.visit(new CircuitVisitor(ccs.compiler) {
+            @Override
+            public void postorder(DBSPNestedOperator operator) {
+                Assert.fail("Unexpected recursive circuit " + operator);
+            }
+        });
+        ccs.stepWeightOne("INSERT INTO t VALUES(1, 2);", """
+                 id | dst
+                ----------
+                 1  | 2""");
+    }
+
+    @Test
+    public void issue7946OneDeadOneLive() {
+        // p1 is dead, p2 is used: only p2's circuit must survive
+        String sql = """
+                CREATE TABLE t(src BIGINT, dst BIGINT);
+                DECLARE RECURSIVE VIEW p1(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p1 AS SELECT src, dst FROM t UNION SELECT p1.src, t.dst FROM p1 JOIN t ON t.src = p1.dst;
+                DECLARE RECURSIVE VIEW p2(src BIGINT, dst BIGINT);
+                CREATE LOCAL VIEW p2 AS SELECT src, dst FROM t UNION SELECT p2.src, t.dst FROM p2 JOIN t ON t.src = p2.dst;
+                CREATE VIEW o AS SELECT * FROM p2;""";
+        var ccs = this.getCCS(sql);
+        int[] count = new int[1];
+        ccs.visit(new CircuitVisitor(ccs.compiler) {
+            @Override
+            public void postorder(DBSPNestedOperator operator) {
+                count[0]++;
+            }
+        });
+        Assert.assertEquals(1, count[0]);
+        ccs.stepWeightOne("INSERT INTO t VALUES(1, 2), (2, 3);", """
+                 src | dst
+                ----------
+                 1   | 2
+                 2   | 3
+                 1   | 3""");
+    }
+
     @Test
     public void testRecursive2() {
         // Non-incremental test

Fixes #7946.

Also, CSE merges the outputs of a recursive circuit that map to the same stream.

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
@mihaibudiu
mihaibudiu added this pull request to the merge queue Nov 13, 2026
Merged via the queue into main with commit 5c20d53 Nov 13, 2026
1 check passed
@mihaibudiu
mihaibudiu deleted the issue7946 branch November 13, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[SQL] Two dead recursive LOCAL views get the same persistent id: DeadCode leaves empty nested circuits

2 participants