Metabase Startup/Upgrade Database connection error

Get Help
1

Hi,

We were using a self-hosted Metabase Server v0.37.2 on Ec2 machine, and we tried upgrading it to 0.39.2 (latest version), but ran into some errors while loading.

So, we booted up an AMI backup instance from yesterday and day before yesterday (where Metabase was working perfectly fine) but started seeing ERROR driver.util :: Database connection error and were unable to connect. We also had a full backup from Nov 2020 (mv.db file as well as jar which was running on 0.36 version) and even that we are facing the same issue while starting the jar file.

We were using H2 database.

Here's the error logtrace:

2021-05-31 12:40:30,611 INFO metabase.core :: Setting up and migrating Metabase DB. Please sit tight, this may take a minute...
2021-05-31 12:40:30,612 WARN metabase.db :: WARNING: Using Metabase with an H2 application database is not recommended for production deployments. For production deployments, we highly recommend using Postgres, MySQL, or MariaDB instead. If you decide to continue to use H2, please be sure to back up the database file regularly. For more information, see https://metabase.com/docs/latest/operations-guide/migrating-from-h2.html
2021-05-31 12:40:30,615 INFO metabase.db :: Verifying h2 Database Connection ...
2021-05-31 12:40:30,620 INFO driver.impl :: Initializing driver :sql...
2021-05-31 12:40:30,621 INFO driver.impl :: Initializing driver :sql-jdbc...
2021-05-31 12:40:30,622 INFO driver.impl :: Initializing driver :h2...
2021-05-31 12:40:35,622 ERROR driver.util :: Database connection error
java.util.concurrent.TimeoutException: Timed out after 5.0 s
        at metabase.util$deref_with_timeout.invokeStatic(util.clj:315) ~[metabase.jar:?]
        at metabase.util$deref_with_timeout.invoke(util.clj:307) ~[metabase.jar:?]
        at metabase.util$do_with_timeout.invokeStatic(util.clj:321) ~[metabase.jar:?]
        at metabase.util$do_with_timeout.invoke(util.clj:318) ~[metabase.jar:?]
        at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:34) [metabase.jar:?]
        at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:23) [metabase.jar:?]
        at clojure.lang.RestFn.invoke(RestFn.java:442) [metabase.jar:?]
        at clojure.lang.Var.invoke(Var.java:393) [metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21793$fn__21794.invoke(db.clj:316) [metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21793.invoke(db.clj:315) [metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) [metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21791.invoke(db.clj:310) [metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) [metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_$fn__21829.invoke(db.clj:352) [metabase.jar:?]
        at metabase.util$do_with_us_locale.invokeStatic(util.clj:670) [metabase.jar:?]
        at metabase.util$do_with_us_locale.invoke(util.clj:656) [metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_.invokeStatic(db.clj:351) [metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_.invoke(db.clj:346) [metabase.jar:?]
        at metabase.db$setup_db_from_env_BANG_.invokeStatic(db.clj:365) [metabase.jar:?]
        at metabase.db$setup_db_from_env_BANG_.invoke(db.clj:358) [metabase.jar:?]
        at metabase.db$setup_db_BANG_$fn__21834.invoke(db.clj:375) [metabase.jar:?]
        at metabase.db$setup_db_BANG_.invokeStatic(db.clj:373) [metabase.jar:?]
        at metabase.db$setup_db_BANG_.invoke(db.clj:368) [metabase.jar:?]
        at metabase.core$init_BANG_.invokeStatic(core.clj:96) [metabase.jar:?]
        at metabase.core$init_BANG_.invoke(core.clj:75) [metabase.jar:?]
        at metabase.core$start_normally.invokeStatic(core.clj:140) [metabase.jar:?]
        at metabase.core$start_normally.invoke(core.clj:134) [metabase.jar:?]
        at metabase.core$_main.invokeStatic(core.clj:173) [metabase.jar:?]
        at metabase.core$_main.doInvoke(core.clj:167) [metabase.jar:?]
        at clojure.lang.RestFn.invoke(RestFn.java:397) [metabase.jar:?]
        at clojure.lang.AFn.applyToHelper(AFn.java:152) [metabase.jar:?]
        at clojure.lang.RestFn.applyTo(RestFn.java:132) [metabase.jar:?]
        at metabase.core.main(Unknown Source) [metabase.jar:?]
2021-05-31 12:40:35,630 ERROR metabase.core :: Metabase Initialization FAILED
java.lang.Exception: Timed out after 5.0 s
        at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:40) ~[metabase.jar:?]
        at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:23) ~[metabase.jar:?]
        at clojure.lang.RestFn.invoke(RestFn.java:442) ~[metabase.jar:?]
        at clojure.lang.Var.invoke(Var.java:393) ~[metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21793$fn__21794.invoke(db.clj:316) ~[metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21793.invoke(db.clj:315) ~[metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) ~[metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790$fn__21791.invoke(db.clj:310) ~[metabase.jar:?]
        at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_$fn__21829.invoke(db.clj:352) ~[metabase.jar:?]
        at metabase.util$do_with_us_locale.invokeStatic(util.clj:670) ~[metabase.jar:?]
        at metabase.util$do_with_us_locale.invoke(util.clj:656) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_.invokeStatic(db.clj:351) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG__STAR_.invoke(db.clj:346) ~[metabase.jar:?]
        at metabase.db$setup_db_from_env_BANG_.invokeStatic(db.clj:365) ~[metabase.jar:?]
        at metabase.db$setup_db_from_env_BANG_.invoke(db.clj:358) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG_$fn__21834.invoke(db.clj:375) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG_.invokeStatic(db.clj:373) ~[metabase.jar:?]
        at metabase.db$setup_db_BANG_.invoke(db.clj:368) ~[metabase.jar:?]
        at metabase.core$init_BANG_.invokeStatic(core.clj:96) ~[metabase.jar:?]
        at metabase.core$init_BANG_.invoke(core.clj:75) ~[metabase.jar:?]
        at metabase.core$start_normally.invokeStatic(core.clj:140) [metabase.jar:?]
        at metabase.core$start_normally.invoke(core.clj:134) [metabase.jar:?]
        at metabase.core$_main.invokeStatic(core.clj:173) [metabase.jar:?]
        at metabase.core$_main.doInvoke(core.clj:167) [metabase.jar:?]
        at clojure.lang.RestFn.invoke(RestFn.java:397) [metabase.jar:?]
        at clojure.lang.AFn.applyToHelper(AFn.java:152) [metabase.jar:?]
        at clojure.lang.RestFn.applyTo(RestFn.java:132) [metabase.jar:?]
        at metabase.core.main(Unknown Source) [metabase.jar:?]
Caused by: java.util.concurrent.TimeoutException: Timed out after 5.0 s
        at metabase.util$deref_with_timeout.invokeStatic(util.clj:315) ~[metabase.jar:?]
        at metabase.util$deref_with_timeout.invoke(util.clj:307) ~[metabase.jar:?]
        at metabase.util$do_with_timeout.invokeStatic(util.clj:321) ~[metabase.jar:?]
        at metabase.util$do_with_timeout.invoke(util.clj:318) ~[metabase.jar:?]
        at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:34) ~[metabase.jar:?]
        ... 28 more
2021-05-31 12:40:35,633 INFO metabase.core :: Metabase Shutting Down ...
2021-05-31 12:40:35,634 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
2021-05-31 12:40:35,640 INFO metabase.core :: Metabase Shutdown COMPLETE

Please let us know if there's any obvious thing we are doing wrong? There shouldn't be any issue with the DB file only because we are facing the same issue on yesterday's DB and Jar file, as well as one from November 2020.

2

Hi @ishaan-rajiv-db
Which command are you using to start Metabase with? How are you hosting Metabase - Docker, JAR?
I would highly recommend that you just get it working on 0.37.3 and then migrate away from H2:
https://www.metabase.com/docs/latest/operations-guide/migrating-from-h2.html
Then after you have migrated, then you can upgrade to latest release, 0.39.3:
https://github.com/metabase/metabase/releases/latest

3

Thanks for the quick revert @flamber

We are using the JAR command. java -jar metabase.jar

Issue is that we are not able to startup either version of Metabase (0.37.2, i made a typo earlier, or 0.39.2) with our current mv.db file.

Even our Nov 2020 mv.db file and JAR (0.36.x), untouched from November are not working with the same error, which ideally should not happen because we were using that mv.db and JAR in the past and that mv.db file is what has transformed to our current mv.db file.

4

@ishaan-rajiv-db Well, if you have reverted the entire instance to a backup and it's not working, then something is really wrong.

Download the metabase.db.mv.db to your local computer and see if you can get it to work there on 0.37.2.

I don't know how you made the backup, but if it was while Metabase was running, then it might have made a backup of the H2 that included a corruption.