Running integration tests with deps.edn build without symlink

Hi all,

I would like to migrate the build of our Exasol driver to deps.edn, see this PR. Building the driver with clojure -X:build :project-dir "\"$(pwd)\"" works when the metabase sources are checked out next to the driver project (see build log).

Now I would like to run unit and integration tests. Until now I linked the driver into the Metabase sources, and started the tests with cd ../metabase && clojure -X:dev:ci:drivers:drivers-dev:test (see run-integration-tests.sh for details) but this generates warning message Use of :paths external to the project has been deprecated, please remove: modules/drivers/exasol/test (see build log).

Can you please give me an advise how to run unit and integration tests for my driver without linking it into the Metabase sources.

Thank you very much for your help!

Kind regards,
Christoph.

Hi Christoph.

This error is coming from clojure.tools.deps.alpha and seems to want to prevent linking outside of a project.

One solution I came up with is to create a jar out of the test resources and then test metabase with both jars on the classpath.

Making a test resources jar

Add the following two aliases to the deps.edn file:

  ;; want a jar with only the test sources in it
  :test-jar
  {:replace-deps  {}}

  ;; run with `clojure -T:build test-jar`
  :build-test-jar
  {:replace-paths ["test"]
   :replace-deps {}
   :extra-deps {io.github.clojure/tools.build {:git/tag "v0.8.2"
                                               :git/sha "ba1a2bf"}}
   :ns-default build}

We need a build.clj file in the root of the project directory

(ns build
  (:require [clojure.tools.build.api :as b]))

(def class-dir "target/classes")
(def test-basis (b/create-basis {:project "deps.edn"
                                 :aliases [:test-jar]}))
(defn clean [_]
  (b/delete {:path "target"}))

(defn test-jar
  "Build a jar containing test sources."
  [_]
  (clean nil)
  (b/write-pom {:class-dir class-dir
                :basis     test-basis
                :lib       'exasol-metabase-driver/test-sources  ;' close quot for preview in discourse
                :version   "1.0"})
  (b/copy-dir {:target-dir class-dir
               :src-dirs ["test"]})
  (b/jar {:class-dir class-dir
          :jar-file "exasol-test-sources.jar"}))

The gist of this is that we just want a jar with the test files so rather than deal with directories outside of the metabase project, we'll just have some namespaces we can find.

We can build this jar with clj -T:build-test-jar test-jar.

An issue with the current exasol driver

For whatever reason, the following call to (log-driver-version) is breaking the build.

(defn- log-driver-version []
  (log/info (u/format-color 'green (format "Loading Exasol Metabase driver %s, Exasol JDBC driver: %s"
                                           (get-driver-version) (get-jdbc-driver-version)))))

(log-driver-version)
^^^^^^^^^^^^^^^

When you run the tests with this line present, you get errors like

ERROR in metabase.driver.exasol-test/join-test (classloader.clj:128)
Uncaught exception, not in assertion.

      clojure.lang.ExceptionInfo: clojure/tools/logging/impl/LoggerFactory
         classloader: #object[clojure.lang.DynamicClassLoader 0x1ec4fdcf "clojure.lang.DynamicClassLoader@1ec4fdcf"]
      classpath-urls: ("file:/Users/dan/projects/work/metabase/plugins/exasol.metabase-driver.jar")
    system-classpath: ("/Users/dan/.m2/repository/amalloy/ring-buffer/1.3.1/ring-buffer-1.3.1.jar"
                       "/Users/dan/.m2/repository/amalloy/ring-gzip-middleware/0.1.4/ring-gzip-middleware-0.1.4.jar"
                       "/Users/dan/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar"
                       "/Users/dan/.m2/repository/aysylu/loom/1.0.2/loom-1.0.2.jar"
                       "/Users/dan/.m2/repository/better-cond/better-cond/2.0.1-SNAPSHOT/better-cond-2.0.1-SNAPSHOT.jar"
                       "/Users/dan/.m2/repository/bigml/histogram/4.1.4/histogram-4.1.4.jar"
                       "/Users/dan/.m2/repository/buddy/buddy-core/1.10.413/buddy-core-1.10.413.jar"
                       "/Users/dan/.m2/repository/buddy/buddy-sign/3.4.333/buddy-sign-3.4.333.jar"
                       "/Users/dan/.m2/repository/cheshire/cheshire/5.10.2/cheshire-5.10.2.jar"
                       "/Users/dan/.m2/repository/cider/cider-nrepl/0.26.0/cider-nrepl-0.26.0.jar"
                       ...)

I think this is because it is AOT compiling a reference to a particular logger and that's not valid when we actually use the jar. In any case, remove that offending line and then we build the jar as we did before:

clojure -X:build :project-dir "\"$(pwd)\""

Run the tests

So now we have:

  • a jar with test sources in it (clojure -T:build-test-jar test-jar creates exasol-test-sources.jar in the root directory)
  • a jar with the exasol driver in it (clojure -X:build :project-dir "\"$(pwd)\"" creates target/exasol.metabase-driver.jar the driver itself)

We just need to run the tests.

From the metabase directory, I run:

DRIVERS=exasol clj -Sdeps '{:deps {exasol/exasol-driver {:local/root "../metabase-driver/target/exasol.metabase-driver.jar"} exasol/exasol-tests {:local/root "../metabase-driver/exasol-test-sources.jar" }}}' -X:dev:test :only metabase.driver.exasol-test

This looks unwieldy but break it down to its constituent parts:

  • DRIVERS=exasol don't want to test h2 or anything else. Setting the driver i'm interested in testing
  • clj can use clojure or clj.
  • -Sdeps '{:deps {exasol/exasol-driver {:local/root "../metabase-driver/target/exasol.metabase-driver.jar"} exasol/exasol-tests {:local/root "../metabase-driver/exasol-test-sources.jar" }}}' Here i'm adding the two jars we created above onto the classpath. These can be moved anywhere you like for convenience. I am just referring to them where they are created.
  • X:dev:test: runs our test runner
  • :only metabase.driver.exasol-test: specify which namespace i want to test. I only want to test the exasol driver

For me I get the following output:

[ten lines or so of warnings about new Clojure 1.11.1 functions]
Finding tests took 26.2 s.
Running 14 tests

 0/14     0% [                                                  ]  ETA: --:--[exasol] DROP SCHEMA "CAM_153" CASCADE
[exasol] CREATE SCHEMA "CAM_153"
ERROR in metabase.driver.exasol-test/join-test (interface.clj:660)
Uncaught exception, not in assertion.

java.lang.Exception: In order to test exasol, you must specify the env var MB_EXASOL_TEST_HOST.
                   metabase.test.data.interface/db-test-env-var-or-throw    interface.clj:  660
                   metabase.test.data.interface/db-test-env-var-or-throw    interface.clj:  656

This looks bad at first glance but its a proper error from your driver:

In order to test exasol, you must specify the env var MB_EXASOL_TEST_HOST.

At the moment I'm not sure how to have an exasol instance running but this seems like it has the driver and tests on the classpath and it is trying to run some tests.

I get 14 of these errors all complaining about the env var for the exasol host not being set so it seems like all tests are running. Hopefully with that env var set all tests will pass.

Helpful links

Hi @dpsutton,
thank you very much for your great help! I got the tests running now locally. I removed all calls to log/info and log/warn to avoid the java.lang.NoClassDefFoundError for clojure/tools/logging/impl/LoggerFactory as you mentioned. However some tests fail with java.lang.NoClassDefFoundError: honeysql/format/ToSql because I use the hformat/to-sql function:

(defmethod hformat/fn-handler (u/qualified-name ::mod)
  [_ x y]
  (format "mod(%s, %s)" (hformat/to-sql x) (hformat/to-sql y)))

I would like to fix the root cause of these NoClassDefFoundErrors. Do you have an idea how to investigate this?

I start the tests with this command:

clojure -Sdeps {:deps { exasol/exasol-driver {:local/root "/Users/chp/git/metabase-driver/target/exasol.metabase-driver.jar"} exasol/exasol-tests {:local/root "/Users/chp/git/metabase-driver/target/exasol-test-sources.jar"} }} -X:dev:ci:drivers:drivers-dev:test

Tests in CI crash with exit code 255, but I will first try to get them running locally.

@kaklakariada I don't really have any context about what you're trying to do or know why you're trying to do things this way but it's not going to work.

You're building a JAR that AOT compiles all its classes. Somewhere in there is bytecode to import the interface honeysql.format.ToSql, but that interface is not included in your driver JAR because Metabase provides it (this is correct). However, you're running Metabase in a non-AOT manner. honeysql.format.ToSql doesn't exist as an interface when you launch Metabase with metabase -X:test... it gets defined dynamically when the honeysql.format namespace is loaded. So if your driver is loaded and it tries to import that interface before honeysql.format is loaded (before the ToSql interface gets created dynamically) it's going to fail.

Everything either has to all be AOTed (this is the case with production metabase.jar plus my-driver.jar) or nothing should be AOTed... otherwise you'll run in to stuff like this.

Why are you building a JAR for testing in the first place? Just add your exasol/exasol-driver as a :local/root dependency pointing toward the relevant source directory. Do the same for your tests. Add the underlying JDBC driver as :extra-deps. You do not need to be building a JAR in order to run the tests. It's a horribly slow feedback cycle at any rate.

If you really want to build a JAR then you'll either have to

  1. change your JAR to stop AOTing the classes (i.e., include Clojure sources but do not compile them) (at which point building a JAR is pretty much pointless)
  2. AOT compile Metabase and its dependencies (not sure if you need to include the test sources as well or not) into a JAR and then include that as an extra :local/root dep as well when running tests

But I would not recommend that.

Hi @camsaul,
thanks a lot for your suggestions, I got it working based on your recommendation to use :local/root dependencies.

When the driver jar is not installed in metabase/plugins, the tests fail with this exception:

java.lang.IllegalArgumentException: No method in multimethod 'connection-properties' for dispatch value: :exasol

It looks like the tests don't find resources/metabase-plugin.yaml. In deps.edn I add the resources directory to the path:

:paths
 ["src" "resources"]

Copying the built driver jar to metabase/plugins solves this, but I would like to avoid this.

Hi @kaklakariada,

You shouldn't add the resources directory to the classpath since Metabase won't know that that specific metabase-plugin.yaml pertains to your specific driver. It won't do anything with it anyway. For running tests you just need to have "src" and probably your "test" directory as well. You shouldn't need to copy any built drivers to the plugins directory; in fact, you don't need to build any drivers at all to run tests.

Instead you can set MB_DEV_ADDITIONAL_DRIVER_MANIFEST_PATHS when running tests so it will see your plugin manifest.

 ;; for hacking on 3rd-party drivers locally: set
 ;; `-Dmb.dev.additional.driver.manifest.paths=/path/to/whatever/metabase-plugin.yaml` or
 ;; `MB_DEV_ADDITIONAL_DRIVER_MANIFEST_PATHS=...` to have that plugin manifest get loaded during startup. Specify
 ;; multiple plugin manifests by comma-separating them.

Hi @camsaul,
Thank you very much for your help, specifying MB_DEV_ADDITIONAL_DRIVER_MANIFEST_PATHS worked :smiley: