Repository navigation
Add VoltDB - #1375
Add VoltDB#1375
Conversation
…reduce with several oracles - GlobalState and SQLQueryAdapter.checkException pass an IgnoreMeException through instead of wrapping it in an AssertionError, so that a provider's connection can skip the statement or the database (VoltDB's does after a node failure, when a statement may or may not have committed). - CompositeTestOracle records the oracle that ran last before running it, so getLastQueryString names the oracle that failed, and returns that oracle's reproducer, so that --use-reducer works with more than one oracle. - SQLancerResultSet.getObject(int).
A provider for VoltDB (https://www.voltactivedata.com), an in-memory, partitioned, ACID SQL database, with a typed expression generator over VoltDB's SQL and the NoREC, TLP (WHERE, DISTINCT, HAVING, aggregates) and PQS oracles, and oracles for VoltDB's own features: parameterized statements, materialized views, LIMIT, subqueries, window functions, set operations, derived tables, stored procedures, schema changes, geospatial functions, query plans (PLANS), durability across a cluster restart (DURABLE), XDCR convergence, permissions (PERMS) and streams. src/sqlancer/voltdb/README.md describes them and the options. TestVoltDB runs the main oracles against a server on localhost:21212 when VOLTDB_AVAILABLE=true. A CI job is to follow: VoltDB's public Developer Edition image needs a license key.
mrigger
left a comment
There was a problem hiding this comment.
Wow, this is awesome! Excited to see the PR. It seems that the VoltDB implementation is more mature than the SQLancer implementations of most other database systems, including many more test oracles. Also great to hear that you have found bugs with it. I skimmed over all the changes, but did not do a full review (except for the changes modifying the common classes/interfaces).
Would it be possible to add a GitHub actions workflow so that we avoid introducing any regression to the VoltDB implementation?
|
|
||
| // the failing oracle's reproducer, so that --use-reducer works with several oracles | ||
| @Override | ||
| public Reproducer<G> getLastReproducer() { |
| DIFF { | ||
| @Override | ||
| public TestOracle<VoltDBGlobalState> create(VoltDBGlobalState globalState) { | ||
| return new VoltDBDifferentialOracle(globalState, expressionErrors()); |
There was a problem hiding this comment.
Awesome to see so many (additional) test oracles!
| * COUNT, SUM, MIN and MAX cover the rows up to the row and its peers, or the whole group without an ORDER BY. NULLs | ||
| * order first, and last with DESC. The query must also return the rows of the same query without the window function. | ||
| */ | ||
| public class VoltDBWindowOracle implements TestOracle<VoltDBGlobalState> { |
There was a problem hiding this comment.
By the way, @Yibo-Dong has been working on a new test oracle for window functions. I think he would be happy to share the approach or tool in case you are interested.
| for (int i = 0; i < ROWS; i++) { | ||
| double w = 0.5 + r.getInteger(0, 450) / 100.0; | ||
| double h = 0.5 + r.getInteger(0, 450) / 100.0; | ||
| boxes.add(new Box(round(-170 + r.getInteger(0, 33_000) / 100.0), |
There was a problem hiding this comment.
Would be nice to have named constants for some of these.
| Connection connection = connection(state, user); | ||
| VoltDBTable table = state.getSchema().getRandomTable(t -> !t.isView()); | ||
| // a procedure check (3) costs two catalog updates, so one check in eight | ||
| switch (Randomly.fromOptions(0, 0, 1, 1, 2, 2, 0, 3)) { |
There was a problem hiding this comment.
I guess at some point we should improve the Randomly interface to add weights/probabilities.
|
|
||
| public VoltDBExpressionGenerator(VoltDBGlobalState state) { | ||
| this.state = state; | ||
| this.avoidKnownBugs = state.getDbmsSpecificOptions().avoidKnownBugs; |
There was a problem hiding this comment.
Fine for now, but it would be great to adopt the known-bug pattern in addition to the flag (see https://github.com/sqlancer/sqlancer/blob/main/CONTRIBUTING.md#unfixed-bugs).
This adds a provider for VoltDB, an in-memory, partitioned, ACID SQL database. We have run it against VoltDB for several weeks, single nodes and clusters, and it has found more than seventy bugs, most of them wrong results.
What it adds
src/sqlancer/voltdb): schema, data and typed expression generators over VoltDB's SQL. It covers replicated and partitioned tables, indexes, materialized views, stored procedures and streams, and its connection goes through the VoltDB JDBC driver (org.voltdb:voltdbclient).src/sqlancer/voltdb/README.mddescribes each oracle and option.Changes to shared code
They are in the first commit, kept apart from the provider:
GlobalStateandSQLQueryAdapter.checkExceptionpass anIgnoreMeExceptionthrough instead of wrapping it in anAssertionError. A provider's connection can then skip a statement or a database. VoltDB's does this after a node failure, when a statement may or may not have committed.CompositeTestOraclerecords the oracle that ran last before running it.getLastQueryStringthen names the oracle that failed, and its reproducer is returned, so--use-reducerworks with more than one oracle.SQLancerResultSet.getObject(int).Testing
mvn verifypasses, including checkstyle, PMD, SpotBugs andcheck_names.py(the provider has its entry).TestVoltDBruns the main oracles against a server onlocalhost:21212whenVOLTDB_AVAILABLE=true, as the other servers' tests do.Not in this PR
voltactivedata/volt-developer-edition, needs a license key, which a workflow would have to hold as a secret. We will follow up with a CI job once we have a key we can give the project, as CONTRIBUTING.md allows.ENG-30583, which are not public.--avoid-known-bugs true(the default) steers the generators around those bugs. Each workaround's comment says what it avoids, so it can be removed once the bug is fixed.We would be glad to adjust anything to fit the project's conventions.