Metabase Shut Down

Hi,

we are using Metabase on the Windows Server Machine and facing the Metabase Shut Down Issue (please see the below log). How we can recover it or find our saved question/dashboards? Please guide

Here is the log:

 "user.timezone" "UTC"}

10-05 13:34:41 ←[1mINFO metabase.plugins←[0m :: Loading plugins in C:\Metabase\p
lugins...
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Plugin 'Metabase BigQuery Dr
iver' depends on plugin 'Metabase Google Drivers Shared Dependencies'
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase BigQuery Driver dep
endency {:plugin Metabase Google Drivers Shared Dependencies} satisfied? false
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Plugins with unsatisfied dep
s: ["Metabase BigQuery Driver"]
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :druid...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :druid
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :google...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered abstract driver :google
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase BigQuery Driver dep
endency {:plugin Metabase Google Drivers Shared Dependencies} satisfied? true
10-05 13:34:42 ←[1mDEBUG plugins.initialize←[0m :: Dependencies satisfied; these
 plugins will now be loaded: ["Metabase BigQuery Driver"]
10-05 13:34:42 ←[1mWARN plugins.lazy-loaded-driver←[0m :: Warning: plugin manife
st for :bigquery does not include connection properties
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :bigquery...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered abstract driver :sql
Load driver :sql took 135.5 ms
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :bigquery (parents:
 [:sql :google])
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Plugin 'Metabase Google Anal
ytics Driver' depends on plugin 'Metabase Google Drivers Shared Dependencies'
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase Google Analytics Dr
iver dependency {:plugin Metabase Google Drivers Shared Dependencies} satisfied?
 true
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :googleanalytics...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :googleanalytics (p
arents: [:google])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :mongo...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :mongo
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase cannot initialize p
lugin Metabase Oracle Driver due to required dependencies. Metabase requires the
 Oracle JDBC driver in order to connect to Oracle databases, but we can't ship i
t as part of Metabase due to licensing restrictions. See https://metabase.com/do
cs/latest/administration-guide/databases/oracle.html for more details.

10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase Oracle Driver depen
dency {:class oracle.jdbc.OracleDriver} satisfied? false
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Plugins with unsatisfied dep
s: ["Metabase Oracle Driver"]
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :presto...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :presto (parents: [
:sql])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :redshift...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered abstract driver :sql-jdbc
(parents: [:sql])
Load driver :sql-jdbc took 6.9 ms
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :postgres (parents:
 [:sql-jdbc])
Load driver :postgres took 80.9 ms
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :redshift (parents:
 [:postgres])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :snowflake...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :snowflake (parents
: [:sql-jdbc])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :hive-like...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered abstract driver :hive-like
 (parents: [:sql-jdbc])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :sparksql...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :sparksql (parents:
 [:hive-like])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :sqlite...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :sqlite (parents: [
:sql-jdbc])
10-05 13:34:42 ←[1mDEBUG plugins.lazy-loaded-driver←[0m :: Registering lazy load
ing driver :sqlserver...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :sqlserver (parents
: [:sql-jdbc])
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase cannot initialize p
lugin Metabase Vertica Driver due to required dependencies. Metabase requires th
e Vertica JDBC driver in order to connect to Vertica databases, but we can't shi
p it as part of Metabase due to licensing restrictions. See https://metabase.com
/docs/latest/administration-guide/databases/vertica.html for more details.

10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Metabase Vertica Driver depe
ndency {:class com.vertica.jdbc.Driver} satisfied? false
10-05 13:34:42 ←[1mINFO plugins.dependencies←[0m :: Plugins with unsatisfied dep
s: ["Metabase Vertica Driver" "Metabase Oracle Driver"]
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :h2 (parents: [:sql
-jdbc])
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Registered driver :mysql (parents: [:
sql-jdbc])
10-05 13:34:42 ←[1mINFO metabase.core←[0m :: Setting up and migrating Metabase D
B. Please sit tight, this may take a minute...
10-05 13:34:42 ←[1mWARN metabase.db←[0m :: WARNING: Using Metabase with an H2 ap
plication database is not recommended for production deployments. For production
 deployments, we highly recommend using Postgres, MySQL, or MariaDB instead. If
you decide to continue to use H2, please be sure to back up the database file re
gularly. For more information, see https://metabase.com/docs/latest/operations-g
uide/migrating-from-h2.html
10-05 13:34:42 ←[1mINFO metabase.db←[0m :: Verifying h2 Database Connection ...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Initializing driver :sql...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Initializing driver :sql-jdbc...
10-05 13:34:42 ←[1mINFO driver.impl←[0m :: Initializing driver :h2...
10-05 13:34:42 ←[1mERROR driver.util←[0m :: Database connection error
org.h2.jdbc.JdbcSQLException: File corrupted while reading record: null. Possibl
e solution: use the recovery tool [90030-197]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
        at org.h2.message.DbException.get(DbException.java:168)
        at org.h2.mvstore.db.MVTableEngine$Store.convertIllegalStateException(MV
TableEngine.java:196)
        at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:168)
        at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:100)
        at org.h2.engine.Database.getPageStore(Database.java:2538)
        at org.h2.engine.Database.open(Database.java:709)
        at org.h2.engine.Database.openDatabase(Database.java:286)
        at org.h2.engine.Database.<init>(Database.java:280)
        at org.h2.engine.Engine.openSession(Engine.java:66)
        at org.h2.engine.Engine.openSession(Engine.java:179)
        at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157)
        at org.h2.engine.Engine.createSession(Engine.java:140)
        at org.h2.engine.Engine.createSession(Engine.java:28)
        at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.jav
a:351)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103)
        at org.h2.Driver.connect(Driver.java:69)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)

        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)

        at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271)
        at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250)
        at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411)
        at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj
:1111)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093)

        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
        at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(c
onnection.clj:159)
        at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connect
ion.clj:154)
        at metabase.driver.sql_jdbc$fn__73840.invokeStatic(sql_jdbc.clj:36)
        at metabase.driver.sql_jdbc$fn__73840.invoke(sql_jdbc.clj:34)
        at clojure.lang.MultiFn.invoke(MultiFn.java:234)
        at metabase.driver.util$can_connect_with_details_QMARK_$fn__22482.invoke
(util.clj:35)
        at metabase.util$do_with_timeout$fn__10195.invoke(util.clj:310)
        at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030)
        at clojure.lang.AFn.call(AFn.java:18)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo
lExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPo
olExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: File corrupted in chunk 532484, expe
cted page length 4..192, got -597500897 [1.4.197/6]
        at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:870)

        at org.h2.mvstore.Page.read(Page.java:652)
        at org.h2.mvstore.Page.read(Page.java:186)
        at org.h2.mvstore.MVStore.readPage(MVStore.java:1936)
        at org.h2.mvstore.MVMap.readPage(MVMap.java:698)
        at org.h2.mvstore.Page.getChildPage(Page.java:208)
        at org.h2.mvstore.Cursor.min(Cursor.java:129)
        at org.h2.mvstore.Cursor.hasNext(Cursor.java:36)
        at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:683)
        at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:664)
        at org.h2.mvstore.MVStore.<init>(MVStore.java:361)
        at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2934)
        at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:155)
        ... 39 more
10-05 13:34:42 ←[1mERROR metabase.core←[0m :: Metabase Initialization FAILED
java.lang.Exception: File corrupted while reading record: null. Possible solutio
n: use the recovery tool [90030-197]
        at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(uti
l.clj:40)
        at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.cl
j:23)
        at clojure.lang.RestFn.invoke(RestFn.java:442)
        at clojure.lang.Var.invoke(Var.java:393)
        at metabase.db$fn__21751$verify_db_connection__21760$fn__21763$fn__21764
.invoke(db.clj:309)
        at metabase.db$fn__21751$verify_db_connection__21760$fn__21763.invoke(db
.clj:308)
        at metabase.db$fn__21751$verify_db_connection__21760.invoke(db.clj:300)
        at metabase.db$fn__21751$verify_db_connection__21760$fn__21761.invoke(db
.clj:303)
        at metabase.db$fn__21751$verify_db_connection__21760.invoke(db.clj:300)
        at metabase.db$setup_db_BANG__STAR_$fn__21799.invoke(db.clj:345)
        at metabase.util$do_with_us_locale.invokeStatic(util.clj:656)
        at metabase.util$do_with_us_locale.invoke(util.clj:642)
        at metabase.db$setup_db_BANG__STAR_.invokeStatic(db.clj:344)
        at metabase.db$setup_db_BANG__STAR_.invoke(db.clj:339)
        at metabase.db$setup_db_from_env_BANG_.invokeStatic(db.clj:358)
        at metabase.db$setup_db_from_env_BANG_.invoke(db.clj:351)
        at metabase.db$setup_db_BANG_$fn__21804.invoke(db.clj:368)
        at metabase.db$setup_db_BANG_.invokeStatic(db.clj:366)
        at metabase.db$setup_db_BANG_.invoke(db.clj:361)
        at metabase.core$init_BANG_.invokeStatic(core.clj:75)
        at metabase.core$init_BANG_.invoke(core.clj:54)
        at metabase.core$start_normally.invokeStatic(core.clj:119)
        at metabase.core$start_normally.invoke(core.clj:113)
        at metabase.core$_main.invokeStatic(core.clj:152)
        at metabase.core$_main.doInvoke(core.clj:146)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at metabase.core.main(Unknown Source)
Caused by: org.h2.jdbc.JdbcSQLException: File corrupted while reading record: nu
ll. Possible solution: use the recovery tool [90030-197]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
        at org.h2.message.DbException.get(DbException.java:168)
        at org.h2.mvstore.db.MVTableEngine$Store.convertIllegalStateException(MV
TableEngine.java:196)
        at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:168)
        at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:100)
        at org.h2.engine.Database.getPageStore(Database.java:2538)
        at org.h2.engine.Database.open(Database.java:709)
        at org.h2.engine.Database.openDatabase(Database.java:286)
        at org.h2.engine.Database.<init>(Database.java:280)
        at org.h2.engine.Engine.openSession(Engine.java:66)
        at org.h2.engine.Engine.openSession(Engine.java:179)
        at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157)
        at org.h2.engine.Engine.createSession(Engine.java:140)
        at org.h2.engine.Engine.createSession(Engine.java:28)
        at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.jav
a:351)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103)
        at org.h2.Driver.connect(Driver.java:69)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)

        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)

        at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271)
        at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250)
        at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411)
        at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj
:1111)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093)

        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
        at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(c
onnection.clj:159)
        at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connect
ion.clj:154)
        at metabase.driver.sql_jdbc$fn__73840.invokeStatic(sql_jdbc.clj:36)
        at metabase.driver.sql_jdbc$fn__73840.invoke(sql_jdbc.clj:34)
        at clojure.lang.MultiFn.invoke(MultiFn.java:234)
        at metabase.driver.util$can_connect_with_details_QMARK_$fn__22482.invoke
(util.clj:35)
        at metabase.util$do_with_timeout$fn__10195.invoke(util.clj:310)
        at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030)
        at clojure.lang.AFn.call(AFn.java:18)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo
lExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPo
olExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: File corrupted in chunk 532484, expe
cted page length 4..192, got -597500897 [1.4.197/6]
        at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:870)

        at org.h2.mvstore.Page.read(Page.java:652)
        at org.h2.mvstore.Page.read(Page.java:186)
        at org.h2.mvstore.MVStore.readPage(MVStore.java:1936)
        at org.h2.mvstore.MVMap.readPage(MVMap.java:698)
        at org.h2.mvstore.Page.getChildPage(Page.java:208)
        at org.h2.mvstore.Cursor.min(Cursor.java:129)
        at org.h2.mvstore.Cursor.hasNext(Cursor.java:36)
        at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:683)
        at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:664)
        at org.h2.mvstore.MVStore.<init>(MVStore.java:361)
        at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2934)
        at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:155)
        ... 39 more
10-05 13:34:42 ←[1mINFO metabase.core←[0m :: Metabase Shutting Down ...
10-05 13:34:42 ←[1mINFO metabase.server←[0m :: Shutting Down Embedded Jetty Webs
erver
10-05 13:34:42 ←[1mINFO metabase.core←[0m :: Metabase Shutdown COMPLETE

C:\Metabase>

Hi @see2rizwan

It looks like you’re ended up with a corrupted application database - possibly because of a bad shutdown.
File corrupted while reading record: null. Possible solution: use the recovery tool

Make sure you have backups and then try recovering - otherwise revert to your last backup:
https://www.metabase.com/docs/latest/troubleshooting-guide/application-database.html

And if you are able to recover, then make sure you have another backup, and then migrate away from H2:
https://www.metabase.com/docs/latest/operations-guide/migrating-from-h2.html

Hi Flamber,

Thank you for your reply.

Yes, the Database is corrupted. I tried to recover with the command “java -cp metabase.jar org.h2.tools.Recover”. There is a text file now with the name “metabase.db.mv.txt”.

How I can proceed further? Sorry, I am not technical so my questions may be annoying.

Please guide. Thank you

@see2rizwan You need to find a technical person who can help you. I really hope you have backups, if so, then revert to a backup.
There’s no file ending with .txt, so if that’s the case, then something is really wrong.
The troubleshooting guide is written for Unix-based systems, so you might need to adjust the commands for Windows.