← 输入框|github.com|刷新
Skip to content

Shortly run SQLancer on the individual DBMS as part of the CI #3

Closed

Description

@mrigger

It would be desirable to add smoke testing to the CI, to verify that the testing implementation for one of the DBMS is not obviously broken.

Activity

  1. mrigger commented on Jun 23, 2020

    @mrigger
    ContributorAuthor

    This seems to be more tricky than initially thought. For DuckDB and SQLite, running SQLancer might cause the JVM to crash when a memory error is triggered (see #1). Also for the other DBMS, it might be that bugs are still triggered on their latest version, either since a new bug was introduced or because an old one has not yet been fixed. An option would be to disable checking whether SQLancer finds a bug, but then we would also not detect any regression bugs in SQLancer.

    I think the optimal setup would be if the DBMS under test would, for each commit to it, pull the latest (release) version of SQLancer and run it for a while (e.g., 10 minutes), to prevent that any new bugs are introduced to the DBMS. SQLancer could, in turn, test the latest (release) version of the DBMS to check that SQLancer is not obviously broken.

  2. mrigger commented on Jul 16, 2020

    @mrigger
    ContributorAuthor

    Closing this, since SQLancer now runs tests for the DBMS it supports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions