Metabase migration Error

Get Help
1

HI team,

My metabase current version was 0.32.
And now i am trying to upgrade to latest.

after stopping the old metabase and running the new one, I am getting the below error.
i was running the postgres db in different container for data persistency . But new Metabase is complaining field already exist (which i believe it should.)

Please suggest.

 Reason: liquibase.exception.DatabaseException: ERROR: column "last_analyzed" of relation "metabase_field" already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]

I can drop the field from metabase_field Table and retry the deployment. But I am not sure if that will work.

2

@devD
So you were running 0.32.0 and now youā€™re trying to upgrade to 0.32.4?
There isnā€™t any changes to the backend since 0.32.0, so it shouldnā€™t run any migrations.
Can just you post the log from start until the error?

EDIT: You can always make a backup of the database, drop the column, start Metabase to see if that fixes it. If not, then revert to the backup.

1 Like
3

Hi flamber,

below is the full log

Picked up JAVA_TOOL_OPTIONS: -Xmx16g
log4j:ERROR Could not find value for key log4j.appender.metabase
log4j:ERROR Could not instantiate appender named ā€œmetabaseā€.
04-10 17:46:04 INFO metabase.util :: Loading Metabaseā€¦
04-10 17:46:04 INFO metabase.util :: Maximum memory available to JVM: 14.2 GB
04-10 17:46:06 INFO util.encryption :: Saved credentials encryption is DISABLED for this Metabase instance. :unlock:
For more information, see https://metabase.com/docs/latest/operations-guide/start.html#encrypting-your-database-connection-details-at-rest
04-10 17:46:08 INFO metabase.driver :: Registered abstract driver :sql :truck:
04-10 17:46:11 INFO metabase.core :: Starting Metabase in STANDALONE mode
04-10 17:46:11 INFO metabase.server :: Launching Embedded Jetty Webserver with config:
{:port 3000, :host ā€œ0.0.0.0ā€}

04-10 17:46:11 INFO metabase.core :: Starting Metabase version v0.32.4 (f4dc152 release-0.32.0) ā€¦
04-10 17:46:11 INFO metabase.core :: System timezone is ā€˜GMTā€™ ā€¦
04-10 17:46:11 INFO metabase.plugins :: Loading plugins in /pluginsā€¦
Extract file /modules/sparksql.metabase-driver.jar -> /plugins/sparksql.metabase-driver.jar took 296 ms
Extract file /modules/bigquery.metabase-driver.jar -> /plugins/bigquery.metabase-driver.jar took 2 ms
Extract file /modules/sqlserver.metabase-driver.jar -> /plugins/sqlserver.metabase-driver.jar took 5 ms
Extract file /modules/sqlite.metabase-driver.jar -> /plugins/sqlite.metabase-driver.jar took 11 ms
Extract file /modules/vertica.metabase-driver.jar -> /plugins/vertica.metabase-driver.jar took 406 Āµs
Extract file /modules/redshift.metabase-driver.jar -> /plugins/redshift.metabase-driver.jar took 11 ms
Extract file /modules/presto.metabase-driver.jar -> /plugins/presto.metabase-driver.jar took 883 Āµs
Extract file /modules/druid.metabase-driver.jar -> /plugins/druid.metabase-driver.jar took 2 ms
Extract file /modules/mongo.metabase-driver.jar -> /plugins/mongo.metabase-driver.jar took 29 ms
Extract file /modules/google.metabase-driver.jar -> /plugins/google.metabase-driver.jar took 14 ms
Extract file /modules/oracle.metabase-driver.jar -> /plugins/oracle.metabase-driver.jar took 546 Āµs
Extract file /modules/snowflake.metabase-driver.jar -> /plugins/snowflake.metabase-driver.jar took 67 ms
Extract file /modules/googleanalytics.metabase-driver.jar -> /plugins/googleanalytics.metabase-driver.jar took 3 ms
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :hive-likeā€¦
04-10 17:46:11 DEBUG plugins.classloader :: Setting current thread context classloader to NEWLY CREATED classloader clojure.lang.DynamicClassLoader@1613d1cā€¦
04-10 17:46:11 INFO metabase.driver :: Registered abstract driver :sql-jdbc (parents: :sql) :truck:
Load driver :sql-jdbc took 31 ms
04-10 17:46:11 INFO metabase.driver :: Registered abstract driver :hive-like (parents: #{:sql-jdbc}) :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :sparksqlā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :sparksql (parents: #{:hive-like}) :truck:
04-10 17:46:11 INFO plugins.dependencies :: Plugin ā€˜Metabase BigQuery Driverā€™ depends on plugin ā€˜Metabase Google Drivers Shared Dependenciesā€™
04-10 17:46:11 INFO plugins.dependencies :: Metabase BigQuery Driver dependency {:plugin Metabase Google Drivers Shared Dependencies} satisfied? false
04-10 17:46:11 INFO plugins.dependencies :: Plugins with unsatisfied deps: [ā€œMetabase BigQuery Driverā€]
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :sqlserverā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :sqlserver (parents: #{:sql-jdbc}) :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :sqliteā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :sqlite (parents: #{:sql-jdbc}) :truck:
04-10 17:46:11 INFO plugins.dependencies :: Metabase cannot initialize plugin Metabase Vertica Driver due to required dependencies. Metabase requires the Vertica JDBC driver in order to connect to Vertica databases, but we canā€™t ship it as part of Metabase due to licensing restrictions. See https://metabase.com/docs/latest/administration-guide/databases/vertica.html for more details.

04-10 17:46:11 INFO plugins.dependencies :: Metabase Vertica Driver dependency {:class com.vertica.jdbc.Driver} satisfied? false
04-10 17:46:11 INFO plugins.dependencies :: Plugins with unsatisfied deps: [ā€œMetabase Vertica Driverā€ ā€œMetabase BigQuery Driverā€]
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :redshiftā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :postgres (parents: :sql-jdbc) :truck:
Load driver :postgres took 17 ms
04-10 17:46:11 INFO metabase.driver :: Registered driver :redshift (parents: #{:postgres}) :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :prestoā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :presto (parents: #{:sql}) :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :druidā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :druid :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :mongoā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :mongo :truck:
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :googleā€¦
04-10 17:46:11 INFO metabase.driver :: Registered abstract driver :google :truck:
04-10 17:46:11 INFO plugins.dependencies :: Metabase BigQuery Driver dependency {:plugin Metabase Google Drivers Shared Dependencies} satisfied? true
04-10 17:46:11 DEBUG plugins.initialize :: Dependencies satisfied; these plugins will now be loaded: [ā€œMetabase BigQuery Driverā€]
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :bigqueryā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :bigquery (parents: #{:sql :google}) :truck:
04-10 17:46:11 INFO plugins.dependencies :: Metabase cannot initialize plugin Metabase Oracle Driver due to required dependencies. Metabase requires the Oracle JDBC driver in order to connect to Oracle databases, but we canā€™t ship it as part of Metabase due to licensing restrictions. See https://metabase.com/docs/latest/administration-guide/databases/oracle.html for more details.

04-10 17:46:11 INFO plugins.dependencies :: Metabase Oracle Driver dependency {:class oracle.jdbc.OracleDriver} satisfied? false
04-10 17:46:11 INFO plugins.dependencies :: Plugins with unsatisfied deps: [ā€œMetabase Oracle Driverā€ ā€œMetabase Vertica Driverā€]
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :snowflakeā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :snowflake (parents: #{:sql-jdbc}) :truck:
04-10 17:46:11 INFO plugins.dependencies :: Plugin ā€˜Metabase Google Analytics Driverā€™ depends on plugin ā€˜Metabase Google Drivers Shared Dependenciesā€™
04-10 17:46:11 INFO plugins.dependencies :: Metabase Google Analytics Driver dependency {:plugin Metabase Google Drivers Shared Dependencies} satisfied? true
04-10 17:46:11 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :googleanalyticsā€¦
04-10 17:46:11 INFO metabase.driver :: Registered driver :googleanalytics (parents: #{:google}) :truck:
04-10 17:46:12 INFO metabase.driver :: Registered driver :mysql (parents: :sql-jdbc) :truck:
Load driver :mysql took 17 ms
04-10 17:46:12 INFO metabase.driver :: Registered driver :h2 (parents: :sql-jdbc) :truck:
Load driver :h2 took 13 ms
04-10 17:46:12 INFO metabase.core :: Setting up and migrating Metabase DB. Please sit tight, this may take a minuteā€¦
04-10 17:46:12 INFO metabase.db :: Verifying postgres Database Connection ā€¦
04-10 17:46:12 INFO metabase.driver :: Initializing driver :sqlā€¦
04-10 17:46:12 INFO metabase.driver :: Initializing driver :sql-jdbcā€¦
04-10 17:46:12 INFO metabase.driver :: Initializing driver :postgresā€¦
04-10 17:46:12 INFO metabase.db :: Verify Database Connection ā€¦ :white_check_mark:
04-10 17:46:12 INFO metabase.db :: Running Database Migrationsā€¦
04-10 17:46:12 INFO metabase.db :: Setting up Liquibaseā€¦
04-10 17:46:12 INFO metabase.db :: Liquibase is ready.
04-10 17:46:12 INFO metabase.db :: Checking if Database has unrun migrationsā€¦
04-10 17:46:13 INFO metabase.db :: Database has unrun migrations. Waiting for migration lock to be clearedā€¦
04-10 17:46:13 INFO metabase.db :: Migration lock is cleared. Running migrationsā€¦
04-10 17:46:13 ERROR changelog.ChangeSet :: Change Set migrations/000_migrations.yaml::32::agilliland failed. Error: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]
04-10 17:46:13 WARN metabase.util :: auto-retry metabase.db$run_schema_migrations_BANG_$fn__14468@918ca32: Migration failed for change set migrations/000_migrations.yaml::32::agilliland:
Reason: liquibase.exception.DatabaseException: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]
04-10 17:46:13 INFO metabase.db :: Setting up Liquibaseā€¦
04-10 17:46:13 INFO metabase.db :: Liquibase is ready.
04-10 17:46:13 INFO metabase.db :: Checking if Database has unrun migrationsā€¦
04-10 17:46:13 INFO metabase.db :: Database has unrun migrations. Waiting for migration lock to be clearedā€¦
04-10 17:46:13 INFO metabase.db :: Migration lock is cleared. Running migrationsā€¦
04-10 17:46:13 ERROR changelog.ChangeSet :: Change Set migrations/000_migrations.yaml::32::agilliland failed. Error: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]
04-10 17:46:13 ERROR metabase.core :: Metabase Initialization FAILED
liquibase.exception.MigrationFailedException: Migration failed for change set migrations/000_migrations.yaml::32::agilliland:
Reason: liquibase.exception.DatabaseException: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:637)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:53)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:83)
at liquibase.Liquibase.update(Liquibase.java:202)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.Liquibase.update(Liquibase.java:175)
at metabase.db$migrate_up_if_needed_BANG_.invokeStatic(db.clj:209)
at metabase.db$migrate_up_if_needed_BANG_.invoke(db.clj:196)
at metabase.db$migrate_BANG_$fn__14450.invoke(db.clj:313)
at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:799)
at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:769)
at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:834)
at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:769)
at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:782)
at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:769)
at metabase.db$migrate_BANG_.invokeStatic(db.clj:301)
at metabase.db$migrate_BANG_.invoke(db.clj:282)
at metabase.db$run_schema_migrations_BANG_$fn__14468.invoke(db.clj:446)
at metabase.util$do_with_auto_retries.invokeStatic(util.clj:428)
at metabase.util$do_with_auto_retries.invoke(util.clj:420)
at metabase.util$do_with_auto_retries.invokeStatic(util.clj:432)
at metabase.util$do_with_auto_retries.invoke(util.clj:420)
at metabase.db$run_schema_migrations_BANG_.invokeStatic(db.clj:445)
at metabase.db$run_schema_migrations_BANG_.invoke(db.clj:428)
at metabase.db$setup_db_BANG_$fn__14477.invoke(db.clj:468)
at metabase.util$do_with_us_locale.invokeStatic(util.clj:683)
at metabase.util$do_with_us_locale.invoke(util.clj:669)
at metabase.db$setup_db_BANG_.invokeStatic(db.clj:466)
at metabase.db$setup_db_BANG_.doInvoke(db.clj:460)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at metabase.core$init_BANG_.invokeStatic(core.clj:72)
at metabase.core$init_BANG_.invoke(core.clj:51)
at metabase.core$start_normally.invokeStatic(core.clj:118)
at metabase.core$start_normally.invoke(core.clj:112)
at metabase.core$_main.invokeStatic(core.clj:138)
at metabase.core$_main.doInvoke(core.clj:133)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at metabase.core.main(Unknown Source)
Caused by: liquibase.exception.DatabaseException: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists [Failed SQL: ALTER TABLE public.metabase_field ADD last_analyzed TIMESTAMP WITH TIME ZONE]
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:356)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:57)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:125)
at liquibase.database.AbstractJdbcDatabase.execute(AbstractJdbcDatabase.java:1229)
at liquibase.database.AbstractJdbcDatabase.executeStatements(AbstractJdbcDatabase.java:1211)
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:600)
ā€¦ 39 more
Caused by: org.postgresql.util.PSQLException: ERROR: column ā€œlast_analyzedā€ of relation ā€œmetabase_fieldā€ already exists
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:352)
ā€¦ 44 more
04-10 17:46:13 INFO metabase.core :: Metabase Shutting Down ā€¦
04-10 17:46:13 INFO metabase.core :: Metabase Shutdown COMPLETE

And below is my run command,
docker run -d -p 80:3000 --name metabase4 -e MB_DB_TYPE=postgres -e MB_DB_PORT=5432 -e MB_DB_USER=postgres -e MB_DB_PASS=docker -e MB_DB_DBNAME=postgres -e MB_DB_HOST=172.17.0.2 -e ā€œJAVA_TOOL_OPTIONS=-Xmx16gā€ metabase/metabase

one more quick question:
yesterday i tested same thing in local which work, but i guess it was 0.32.3

is there a way i can pull 0.32.3 version and test my deploy.
like ā€œdocker pull metabase/metabase:0.32.3ā€ is not working

4

It looks like something is completely corrupted in your database. Itā€™s trying to do migration for changeset 32, which is several years old.
Have you been downgrading versions - going back and forth between them, meaning using 0.30, then trying 0.32, then going to 0.31 and so on, with the same database?
You have to use the tags and then using the same tag, when running the container:
docker pull metabase/metabase:v0.32.3

1 Like
5

Hi Flamber,

Thanks for your help,
no i have not tried to downgrade version ever. I tried version v0.32.3 but same problem.

Is there a way to stop database migration as i need to run metabase by connecting to postgres.

this metabase were used by our BA and i will be in trouble if data are lost.

Please suggest

6

Use the env variable MB_DB_AUTOMIGRATE=false
https://www.metabase.com/docs/latest/operations-guide/start.html#running-metabase-database-migrations-manually

Iā€™m not sure what has happened to your database, but maybe the migration from H2 to Postgres went wrong. Without knowing exactly which versions you used with which databases, then itā€™s difficult to know. Your logs should have that information.

1 Like
7

@flamber
Great, great help for this flamber,

What i did is fired a simple H2 metabase copied the databasechangelog from h2 to my postgres DB.
For some reason i lost record in the table with id 32, which was throwing error.
Your hint of changeset 32 helped like a light bomb.

Thanks a lot

1 Like