@relferreira Permissions or ownership of the files in /plugins/
? There should be some errors during startup that should hint at the problem, but I’m not totally sure.
Hello @flamber
I couldn’t find any error during startup. Both methods logs:
04-06 23:29:44 INFO driver.impl :: Registered driver :sparksql-databricks (parents: [:hive-like]) 🚚
...
Metabase Initialization COMPLETE
Here is a diff of the permissions on both methods
Docker run (works)
bash-5.0# ls -lh
total 68K
drwxr-xr-x 1 root root 4.0K Feb 25 20:38 app
drwxr-xr-x 1 root root 4.0K Jan 13 22:57 bin
drwxr-xr-x 5 root root 360 Apr 6 23:29 dev
drwxr-xr-x 1 root root 4.0K Apr 6 23:29 etc
drwxr-xr-x 1 root root 4.0K Apr 6 23:29 home
drwxr-xr-x 1 root root 4.0K Jan 13 22:57 lib
drwxr-xr-x 2 root root 4.0K Jan 2 19:19 lib64
drwxr-xr-x 5 root root 4.0K Oct 21 13:39 media
drwxr-xr-x 2 metabase metabase 4.0K Apr 6 23:29 metabase.db
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 mnt
drwxr-xr-x 1 root root 4.0K Jan 2 19:21 opt
drwxr-xr-x 17 root root 544 Apr 6 18:09 plugins
dr-xr-xr-x 206 root root 0 Apr 6 23:29 proc
drwx------ 2 root root 4.0K Oct 21 13:39 root
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 run
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 sbin
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 srv
dr-xr-xr-x 13 root root 0 Apr 6 23:29 sys
drwxrwxrwt 1 root root 4.0K Apr 6 23:29 tmp
drwxr-xr-x 1 root root 4.0K Jan 2 19:20 usr
drwxr-xr-x 1 root root 4.0K Oct 21 13:39 var
bash-5.0# ls -lh plugins
total 140M
-rw-r--r-- 1 root root 529.7K Feb 25 12:36 bigquery.metabase-driver.jar
-rw-r--r-- 1 root root 454.6K Feb 25 12:36 druid.metabase-driver.jar
-rw-r--r-- 1 root root 1.0M Feb 25 12:36 google.metabase-driver.jar
-rw-r--r-- 1 root root 703.8K Feb 25 12:36 googleanalytics.metabase-driver.jar
-rw-r--r-- 1 root root 4.4M Feb 25 12:36 mongo.metabase-driver.jar
-rw-r--r-- 1 root root 67.6K Feb 25 12:36 oracle.metabase-driver.jar
-rw-r--r-- 1 root root 137.6K Feb 25 12:36 presto.metabase-driver.jar
-rw-r--r-- 1 root root 2.3M Feb 25 12:36 redshift.metabase-driver.jar
-rw-r--r-- 1 root root 16.9M Apr 6 12:12 simba-spark-jdbc41-2.6.3.1003.jar
-rw-r--r-- 1 root root 27.6M Feb 25 12:36 snowflake.metabase-driver.jar
-rw-r--r-- 1 root root 17.0M Apr 6 12:11 sparksql-databricks.metabase-driver.jar
-rw-r--r-- 1 root root 59.9M Feb 25 12:36 sparksql.metabase-driver.jar
-rw-r--r-- 1 root root 6.8M Feb 25 12:36 sqlite.metabase-driver.jar
-rw-r--r-- 1 root root 1.2M Feb 25 12:36 sqlserver.metabase-driver.jar
-rw-r--r-- 1 root root 49.6K Feb 25 12:36 vertica.metabase-driver.jar
custom Dockerfile (error described in topic)
bash-5.0# ls -lh
total 72K
drwxr-xr-x 1 root root 4.0K Apr 6 18:41 app
drwxr-xr-x 1 root root 4.0K Jan 13 22:57 bin
drwxr-xr-x 5 root root 360 Apr 6 23:27 dev
drwxr-xr-x 1 root root 4.0K Apr 6 23:27 etc
drwxr-xr-x 1 root root 4.0K Apr 6 23:27 home
drwxr-xr-x 1 root root 4.0K Jan 13 22:57 lib
drwxr-xr-x 2 root root 4.0K Jan 2 19:19 lib64
drwxr-xr-x 5 root root 4.0K Oct 21 13:39 media
drwxr-xr-x 2 metabase metabase 4.0K Apr 6 23:27 metabase.db
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 mnt
drwxr-xr-x 1 root root 4.0K Jan 2 19:21 opt
drwxrwxrwx 1 root root 4.0K Apr 6 19:46 plugins
dr-xr-xr-x 206 root root 0 Apr 6 23:27 proc
drwx------ 2 root root 4.0K Oct 21 13:39 root
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 run
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 sbin
drwxr-xr-x 2 root root 4.0K Oct 21 13:39 srv
dr-xr-xr-x 13 root root 0 Apr 6 23:27 sys
drwxrwxrwt 1 root root 4.0K Apr 6 23:27 tmp
drwxr-xr-x 1 root root 4.0K Jan 2 19:20 usr
drwxr-xr-x 1 root root 4.0K Oct 21 13:39 var
bash-5.0# ls -lh plugins
total 139M
-rw-r--r-- 1 root root 529.7K Feb 25 12:36 bigquery.metabase-driver.jar
-rw-r--r-- 1 root root 454.6K Feb 25 12:36 druid.metabase-driver.jar
-rw-r--r-- 1 root root 1.0M Feb 25 12:36 google.metabase-driver.jar
-rw-r--r-- 1 root root 703.8K Feb 25 12:36 googleanalytics.metabase-driver.jar
-rw-r--r-- 1 root root 4.4M Feb 25 12:36 mongo.metabase-driver.jar
-rw-r--r-- 1 root root 67.6K Feb 25 12:36 oracle.metabase-driver.jar
-rw-r--r-- 1 root root 137.6K Feb 25 12:36 presto.metabase-driver.jar
-rw-r--r-- 1 root root 2.3M Feb 25 12:36 redshift.metabase-driver.jar
-rw-r--r-- 1 root root 16.9M Apr 6 12:12 simba-spark-jdbc41-2.6.3.1003.jar
-rw-r--r-- 1 root root 27.6M Feb 25 12:36 snowflake.metabase-driver.jar
-rw-r--r-- 1 root root 17.0M Apr 6 12:11 sparksql-databricks.metabase-driver.jar
-rw-r--r-- 1 root root 59.9M Feb 25 12:36 sparksql.metabase-driver.jar
-rw-r--r-- 1 root root 6.8M Feb 25 12:36 sqlite.metabase-driver.jar
-rw-r--r-- 1 root root 1.2M Feb 25 12:36 sqlserver.metabase-driver.jar
-rw-r--r-- 1 root root 49.6K Feb 25 12:36 vertica.metabase-driver.jar
@relferreira Not sure why /app
is from April 6th - it should be April 2nd, when 0.35.1 was released.
Try just using docker cp
to copy the drivers into place - if that works, then I guess you need to investigate what’s going on with Docker, but then it’s likely not a problem with Metabase.
@flamber I updated the driver to work with Metabase 0.34:
Now everything works if I use the following Dockerfile:
FROM metabase/metabase-head:latest
ENV MB_DB_CONNECTION_TIMEOUT_MS=60000
COPY sparksql-databricks.metabase-driver.jar /app/plugins/
But using this one, it doesn’t connect to the database
FROM metabase/metabase
ENV MB_DB_CONNECTION_TIMEOUT_MS=60000
COPY sparksql-databricks.metabase-driver.jar /plugins/
Do you know why?
By the way, thanks a lot for the quick replies.
@chandan try the version in my repository, I think it is working fine now.
@relferreira
It would be really nice if you used version tags instead of nothing or latest
, since it’s much easier to see that you’re actually using the same version.
Why? Besides the obvious differences between the two Dockerfiles?
metabase/metabase-head
is not the same as metabase/metabase
- and is build differently.
Hello @flamber, I can obviously see the difference. I was politely asking how I can make de plugin system work using the metabase/metabase
one.
@relferreira Given the amount of people that posts logs, where the answer is in their own post, then I don’t know if you had noticed the differences.
I don’t know why it doesn’t read the plugins directory - it should - you can see that in the log during startup, and that’s where it will extract the built-in drivers too.
So given that it’s extracting correctly, I’m not sure what’s going on. Post the entire log from startup.
Have you tried to docker cp
into the official image?
@flamber with docker cp
it works
I think you are right, there is something wrong with my Docker.
Thank’s a lot for the help
Status update, a couple years later: I've tried all three versions of the community Databricks Spark SQL driver, and none managed to connect to my Databricks instance. When I try to add a database via their drivers, Metabase fails to connect:
-
fhsgoncalves's says
No suitable driver found for jdbc:spark://adb-350175....
-
relferreira's says
[Simba][SparkJDBCDriver](500164) Error initialized or created transport for authentication: Invalid status 21
- rajeshkumarravi's says the same thing, even though it has an updated Simba JDBC driver
(My Databricks instance is on Azure, and has compute and SQL clusters running.)
Hi there! I made a post on the DBX forum where I hope we can get some traction to get an official driver implemented: Metabase support - Databricks - 54086
Please go in there and make your voices heard and upvote etc etc.