Metabase service no longer starts

Hey all,

I’m running Metabase on Digitalocean (installed it using their 1-click apps) but since a few days it doesn’t start any longer. I also tried updating it today (so it’s currently running 0.37.3

The error seems related to the h2 database driver but I’ve got no idea how to solve it. Any thoughts?

Thanks in advance!


    metabase.sh[3521]: org.h2.jdbc.JdbcSQLException: General error: "java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2" [50000-197]
    metabase.sh[3521]:         at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.message.DbException.get(DbException.java:168) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.message.DbException.convert(DbException.java:307) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.openDatabase(Database.java:319) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.<init>(Database.java:280) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:66) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:179) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:140) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:28) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:351) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.Driver.connect(Driver.java:69) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1111) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:192) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:187) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invokeStatic(sql_jdbc.clj:36) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invoke(sql_jdbc.clj:34) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.MultiFn.invoke(MultiFn.java:234) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.util$can_connect_with_details_QMARK_$fn__22524.invoke(util.clj:35) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.util$do_with_timeout$fn__10218.invoke(util.clj:324) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.AFn.call(AFn.java:18) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    metabase.sh[3521]:         at java.lang.Thread.run(Thread.java:834) ~[?:?]
    metabase.sh[3521]: Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2
    metabase.sh[3521]:         at org.h2.mvstore.db.ValueDataType.compare(ValueDataType.java:105) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.compare(MVMap.java:670) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.Page.binarySearch(Page.java:325) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.put(MVMap.java:166) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.put(MVMap.java:124) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.TransactionStore.rollbackTo(TransactionStore.java:528) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.TransactionStore$Transaction.rollback(TransactionStore.java:827) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.MVTableEngine$Store.initTransactions(MVTableEngine.java:256) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.open(Database.java:779) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.openDatabase(Database.java:286) ~[metabase.jar:?]
    metabase.sh[3521]:         ... 35 more
    metabase.sh[3521]: 2020-12-05 10:06:48,574 ERROR metabase.core :: Metabase Initialization FAILED
    metabase.sh[3521]: java.lang.Exception: General error: "java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2" [50000-197]
    metabase.sh[3521]:         at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:40) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:23) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.RestFn.invoke(RestFn.java:442) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.Var.invoke(Var.java:393) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$fn__21781$verify_db_connection__21790$fn__21793$fn__21794.invoke(db.clj:316) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$fn__21781$verify_db_connection__21790$fn__21793.invoke(db.clj:315) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$fn__21781$verify_db_connection__21790$fn__21791.invoke(db.clj:310) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$fn__21781$verify_db_connection__21790.invoke(db.clj:307) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG__STAR_$fn__21829.invoke(db.clj:352) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.util$do_with_us_locale.invokeStatic(util.clj:670) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.util$do_with_us_locale.invoke(util.clj:656) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG__STAR_.invokeStatic(db.clj:351) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG__STAR_.invoke(db.clj:346) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_from_env_BANG_.invokeStatic(db.clj:365) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_from_env_BANG_.invoke(db.clj:358) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG_$fn__21834.invoke(db.clj:375) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG_.invokeStatic(db.clj:373) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.db$setup_db_BANG_.invoke(db.clj:368) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$init_BANG_.invokeStatic(core.clj:96) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$init_BANG_.invoke(core.clj:75) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$start_normally.invokeStatic(core.clj:140) [metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$start_normally.invoke(core.clj:134) [metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$_main.invokeStatic(core.clj:173) [metabase.jar:?]
    metabase.sh[3521]:         at metabase.core$_main.doInvoke(core.clj:167) [metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.RestFn.invoke(RestFn.java:397) [metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.AFn.applyToHelper(AFn.java:152) [metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.RestFn.applyTo(RestFn.java:132) [metabase.jar:?]
    metabase.sh[3521]:         at metabase.core.main(Unknown Source) [metabase.jar:?]
    metabase.sh[3521]: Caused by: org.h2.jdbc.JdbcSQLException: General error: "java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2" [50000-197]
    metabase.sh[3521]:         at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.message.DbException.get(DbException.java:168) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.message.DbException.convert(DbException.java:307) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.openDatabase(Database.java:319) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.<init>(Database.java:280) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:66) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:179) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:140) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:28) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:351) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.Driver.connect(Driver.java:69) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1111) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:192) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:187) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invokeStatic(sql_jdbc.clj:36) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invoke(sql_jdbc.clj:34) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.MultiFn.invoke(MultiFn.java:234) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.util$can_connect_with_details_QMARK_$fn__22524.invoke(util.clj:35) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.util$do_with_timeout$fn__10218.invoke(util.clj:324) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.AFn.call(AFn.java:18) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    metabase.sh[3521]:         at java.lang.Thread.run(Thread.java:834) ~[?:?]
    metabase.sh[3521]: Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2
    metabase.sh[3521]:         at org.h2.mvstore.db.ValueDataType.compare(ValueDataType.java:105) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.compare(MVMap.java:670) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.Page.binarySearch(Page.java:325) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.put(MVMap.java:166) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.MVMap.put(MVMap.java:124) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.TransactionStore.rollbackTo(TransactionStore.java:528) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.TransactionStore$Transaction.rollback(TransactionStore.java:827) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.mvstore.db.MVTableEngine$Store.initTransactions(MVTableEngine.java:256) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.open(Database.java:779) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.openDatabase(Database.java:286) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Database.<init>(Database.java:280) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:66) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.openSession(Engine.java:179) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:140) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.Engine.createSession(Engine.java:28) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:351) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103) ~[metabase.jar:?]
    metabase.sh[3521]:         at org.h2.Driver.connect(Driver.java:69) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
    metabase.sh[3521]:         at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1111) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.java.jdbc$query.invoke(jdbc.clj:1144) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:192) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:187) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invokeStatic(sql_jdbc.clj:36) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.sql_jdbc$fn__78135.invoke(sql_jdbc.clj:34) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.MultiFn.invoke(MultiFn.java:234) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.driver.util$can_connect_with_details_QMARK_$fn__22524.invoke(util.clj:35) ~[metabase.jar:?]
    metabase.sh[3521]:         at metabase.util$do_with_timeout$fn__10218.invoke(util.clj:324) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030) ~[metabase.jar:?]
    metabase.sh[3521]:         at clojure.lang.AFn.call(AFn.java:18) ~[metabase.jar:?]
    metabase.sh[3521]:         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    metabase.sh[3521]:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    metabase.sh[3521]:         at java.lang.Thread.run(Thread.java:834) ~[?:?]
    metabase.sh[3521]: 2020-12-05 10:06:48,594 INFO metabase.core :: Metabase Shutting Down ...
    metabase.sh[3521]: 2020-12-05 10:06:48,597 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
    metabase.sh[3521]: 2020-12-05 10:06:48,610 INFO metabase.core :: Metabase Shutdown COMPLETE
    systemd[1]: metabase.service: Main process exited, code=exited, status=1/FAILURE
    systemd[1]: metabase.service: Failed with result 'exit-code'.


Hi @EricCharles
Looks like you have gotten a corrupted H2 database:
https://www.metabase.com/docs/latest/troubleshooting-guide/application-database.html#metabase-h2-application-database-gets-corrupted
After you have fixed the corruption, then migration away from H2:
https://www.metabase.com/docs/latest/operations-guide/migrating-from-h2.html