Sporadically failing Metabase tests

Driver Development
1

Hello!
In our continuous integration build on GitHub Actions we run the Metabase test suite with our Exasol driver. Some of the tests fail sporadically, e.g. this one:

 FAIL in metabase.api.session-test/failure-threshold-per-request-source (session_test.clj:182)
The same as above, but ensure that throttling is done on a per request source basis.
expected: #"^Too many attempts! You must wait 1\d seconds before trying again\.$"
  actual: "Too many attempts! You must wait 9 seconds before trying again."

(see https://github.com/exasol/metabase-driver/issues/21)

Do you see the same test failures in your tests or is this caused by our test setup?

Our temporary solution was to skip these tests, but we would like to enable them once they are stable.

2

Hi @kaklakariada
Yeah, we're seeing these sometimes too - quite annoying, since it's just a couple of millisecond difference, which causes this problem.
We are working on a lot of these flaky tests currently, so hopefully you should soon be able to rebase master and then it should not be a problem anymore.

3

Hi @flamber
Great, thank you for the information!
Looking forward to the new release :smiley: