← 输入框|github.com|刷新
Skip to content

Add VoltDB - #1375

Merged
mrigger merged 3 commits into
sqlancer:mainfrom
VoltDB:upstream-voltdb-provider
Nov 15, 2026
Merged

mrigger merged 3 commits into
sqlancer:mainfrom
VoltDB:upstream-voltdb-provider

Conversation

@jwiebema1

Copy link
Copy Markdown
Contributor

This adds a provider for VoltDB, an in-memory, partitioned, ACID SQL database. We have run it against VoltDB for several weeks, single nodes and clusters, and it has found more than seventy bugs, most of them wrong results.

What it adds

  • The provider (src/sqlancer/voltdb): schema, data and typed expression generators over VoltDB's SQL. It covers replicated and partitioned tables, indexes, materialized views, stored procedures and streams, and its connection goes through the VoltDB JDBC driver (org.voltdb:voltdbclient).
  • The common oracles: NoREC, TLP (WHERE, DISTINCT, HAVING, aggregates) and PQS. The provider evaluates VoltDB's expressions itself for PQS.
  • Oracles for VoltDB's own features:
    • parameterized statements, materialized views, LIMIT/OFFSET, subqueries, window functions, set operations, derived tables and CTEs;
    • stored procedures, schema changes and the plan cache, geospatial functions;
    • differential query plans (PLANS) and streams.
  • Oracles that need a cluster set up for them, each behind an option: permissions with security on, durability across a whole-cluster crash, XDCR convergence, and a mirror cluster running another version (DIFF).
  • src/sqlancer/voltdb/README.md describes each oracle and option.

Changes to shared code

They are in the first commit, kept apart from the provider:

  • GlobalState and SQLQueryAdapter.checkException pass an IgnoreMeException through instead of wrapping it in an AssertionError. A provider's connection can then skip a statement or a database. VoltDB's does this after a node failure, when a statement may or may not have committed.
  • CompositeTestOracle records the oracle that ran last before running it. getLastQueryString then names the oracle that failed, and its reproducer is returned, so --use-reducer works with more than one oracle.
  • SQLancerResultSet.getObject(int).

Testing

  • mvn verify passes, including checkstyle, PMD, SpotBugs and check_names.py (the provider has its entry).
  • TestVoltDB runs the main oracles against a server on localhost:21212 when VOLTDB_AVAILABLE=true, as the other servers' tests do.

Not in this PR

  • CI. VoltDB's public image, voltactivedata/volt-developer-edition, needs a license key, which a workflow would have to hold as a secret. We will follow up with a CI job once we have a key we can give the project, as CONTRIBUTING.md allows.
  • Ticket numbers. Code comments name issues in VoltDB's tracker, such as ENG-30583, which are not public. --avoid-known-bugs true (the default) steers the generators around those bugs. Each workaround's comment says what it avoids, so it can be removed once the bug is fixed.

We would be glad to adjust anything to fit the project's conventions.

…reduce with several oracles

- GlobalState and SQLQueryAdapter.checkException pass an IgnoreMeException through instead of wrapping it in an
  AssertionError, so that a provider's connection can skip the statement or the database (VoltDB's does after a
  node failure, when a statement may or may not have committed).
- CompositeTestOracle records the oracle that ran last before running it, so getLastQueryString names the oracle
  that failed, and returns that oracle's reproducer, so that --use-reducer works with more than one oracle.
- SQLancerResultSet.getObject(int).
A provider for VoltDB (https://www.voltactivedata.com), an in-memory, partitioned, ACID SQL database, with a typed
expression generator over VoltDB's SQL and the NoREC, TLP (WHERE, DISTINCT, HAVING, aggregates) and PQS oracles,
and oracles for VoltDB's own features: parameterized statements, materialized views, LIMIT, subqueries, window
functions, set operations, derived tables, stored procedures, schema changes, geospatial functions, query plans
(PLANS), durability across a cluster restart (DURABLE), XDCR convergence, permissions (PERMS) and streams.
src/sqlancer/voltdb/README.md describes them and the options.

TestVoltDB runs the main oracles against a server on localhost:21212 when VOLTDB_AVAILABLE=true. A CI job is to
follow: VoltDB's public Developer Edition image needs a license key.

@mrigger mrigger left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this is awesome! Excited to see the PR. It seems that the VoltDB implementation is more mature than the SQLancer implementations of most other database systems, including many more test oracles. Also great to hear that you have found bugs with it. I skimmed over all the changes, but did not do a full review (except for the changes modifying the common classes/interfaces).

Would it be possible to add a GitHub actions workflow so that we avoid introducing any regression to the VoltDB implementation?


// the failing oracle's reproducer, so that --use-reducer works with several oracles
@Override
public Reproducer<G> getLastReproducer() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this!

DIFF {
@Override
public TestOracle<VoltDBGlobalState> create(VoltDBGlobalState globalState) {
return new VoltDBDifferentialOracle(globalState, expressionErrors());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see so many (additional) test oracles!

* COUNT, SUM, MIN and MAX cover the rows up to the row and its peers, or the whole group without an ORDER BY. NULLs
* order first, and last with DESC. The query must also return the rows of the same query without the window function.
*/
public class VoltDBWindowOracle implements TestOracle<VoltDBGlobalState> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, @Yibo-Dong has been working on a new test oracle for window functions. I think he would be happy to share the approach or tool in case you are interested.

for (int i = 0; i < ROWS; i++) {
double w = 0.5 + r.getInteger(0, 450) / 100.0;
double h = 0.5 + r.getInteger(0, 450) / 100.0;
boxes.add(new Box(round(-170 + r.getInteger(0, 33_000) / 100.0),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have named constants for some of these.

Connection connection = connection(state, user);
VoltDBTable table = state.getSchema().getRandomTable(t -> !t.isView());
// a procedure check (3) costs two catalog updates, so one check in eight
switch (Randomly.fromOptions(0, 0, 1, 1, 2, 2, 0, 3)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess at some point we should improve the Randomly interface to add weights/probabilities.


public VoltDBExpressionGenerator(VoltDBGlobalState state) {
this.state = state;
this.avoidKnownBugs = state.getDbmsSpecificOptions().avoidKnownBugs;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for now, but it would be great to adopt the known-bug pattern in addition to the flag (see https://github.com/sqlancer/sqlancer/blob/main/CONTRIBUTING.md#unfixed-bugs).

@mrigger
mrigger merged commit a513c87 into sqlancer:main Nov 15, 2026
23 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants