Issue when upgrading from v0.41.4 to 0.43.2

guillaume.cadoret · April 27, 2023, 11:58am

Hello,

To solved SSO Google issue, ( see Metabase-specific SSO is broken; please bump · Issue #352 · pluralsh/plural-artifacts · GitHub) I update my metabase version (with docker) to 0.43.2

I encounter issue when I start metabase , Liquibase issue :



metabase    | 2023-04-27 14:03:27,748 INFO db.liquibase :: Database has unrun migrations. Waiting for migration lock to be cleared...
metabase    | 2023-04-27 14:03:28,540 INFO db.liquibase :: Migration lock is cleared. Running migrations...
metabase    | 2023-04-27 14:03:29,271 ERROR metabase.core :: Metabase Initialization FAILED
metabase    | liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for change set migrations/000_migrations.yaml::313::jeff303:
metabase    |      Reason: liquibase.exception.DatabaseException: ERROR: there is no unique constraint matching given keys for referenced table "core_user" [Failed SQL: (0) CREATE TABLE "public"."secret" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL, "version" INTEGER DEFAULT 1 NOT NULL, "creator_id" INTEGER, "created_at" TIMESTAMP with time zone NOT NULL, "updated_at" TIMESTAMP with time zone, "name" VARCHAR(254) NOT NULL, "kind" VARCHAR(254) NOT NULL, "source" VARCHAR(254), "value" BYTEA NOT NULL, CONSTRAINT "secret_pkey" PRIMARY KEY ("id", "version"), CONSTRAINT "fk_secret_ref_user_id" FOREIGN KEY ("creator_id") REFERENCES "public"."core_user"("id"))]
metabase    | 	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:126)
metabase    | 	at liquibase.Liquibase.lambda$null$0(Liquibase.java:265)
metabase    | 	at liquibase.Scope.lambda$child$0(Scope.java:180)
metabase    | 	at liquibase.Scope.child(Scope.java:189)
metabase    | 	at liquibase.Scope.child(Scope.java:179)
metabase    | 	at liquibase.Scope.child(Scope.java:158)
metabase    | 	at liquibase.Scope.child(Scope.java:243)
metabase    | 	at liquibase.Liquibase.lambda$update$1(Liquibase.java:264)
metabase    | 	at liquibase.Scope.lambda$child$0(Scope.java:180)
metabase    | 	at liquibase.Scope.child(Scope.java:189)
metabase    | 	at liquibase.Scope.child(Scope.java:179)
metabase    | 	at liquibase.Scope.child(Scope.java:158)
metabase    | 	at liquibase.Liquibase.runInScope(Liquibase.java:2405)
metabase    | 	at liquibase.Liquibase.update(Liquibase.java:211)
metabase    | 	at liquibase.Liquibase.update(Liquibase.java:197)
metabase    | 	at liquibase.Liquibase.update(Liquibase.java:193)
metabase    | 	at metabase.db.liquibase$migrate_up_if_needed_BANG_.invokeStatic(liquibase.clj:142)
metabase    | 	at metabase.db.liquibase$migrate_up_if_needed_BANG_.invoke(liquibase.clj:130)
metabase    | 	at metabase.db.setup$fn__35427$migrate_BANG___35432$fn__35433$fn__35434.invoke(setup.clj:66)
metabase    | 	at metabase.db.liquibase$fn__30951$do_with_liquibase__30956$fn__30957.invoke(liquibase.clj:59)
metabase    | 	at metabase.db.liquibase$fn__30951$do_with_liquibase__30956.invoke(liquibase.clj:51)
metabase    | 	at metabase.db.setup$fn__35427$migrate_BANG___35432$fn__35433.invoke(setup.clj:61)
metabase    | 	at metabase.db.setup$fn__35427$migrate_BANG___35432.invoke(setup.clj:40)
metabase    | 	at metabase.db.setup$fn__35486$run_schema_migrations_BANG___35491$fn__35492.invoke(setup.clj:119)
metabase    | 	at metabase.db.setup$fn__35486$run_schema_migrations_BANG___35491.invoke(setup.clj:113)
metabase    | 	at metabase.db.setup$fn__35538$setup_db_BANG___35543$fn__35544$fn__35547$fn__35548.invoke(setup.clj:145)
metabase    | 	at metabase.util$do_with_us_locale.invokeStatic(util.clj:716)
metabase    | 	at metabase.util$do_with_us_locale.invoke(util.clj:702)
metabase    | 	at metabase.db.setup$fn__35538$setup_db_BANG___35543$fn__35544$fn__35547.invoke(setup.clj:143)
metabase    | 	at metabase.db.setup$fn__35538$setup_db_BANG___35543$fn__35544.invoke(setup.clj:142)
metabase    | 	at metabase.db.setup$fn__35538$setup_db_BANG___35543.invoke(setup.clj:136)
metabase    | 	at metabase.db$setup_db_BANG_$fn__35573.invoke(db.clj:65)
metabase    | 	at metabase.db$setup_db_BANG_.invokeStatic(db.clj:60)
metabase    | 	at metabase.db$setup_db_BANG_.invoke(db.clj:51)
metabase    | 	at metabase.core$init_BANG__STAR_.invokeStatic(core.clj:98)
metabase    | 	at metabase.core$init_BANG__STAR_.invoke(core.clj:81)
metabase    | 	at metabase.core$init_BANG_.invokeStatic(core.clj:138)
metabase    | 	at metabase.core$init_BANG_.invoke(core.clj:133)
metabase    | 	at metabase.core$start_normally.invokeStatic(core.clj:150)
metabase    | 	at metabase.core$start_normally.invoke(core.clj:144)
metabase    | 	at metabase.core$_main.invokeStatic(core.clj:183)
metabase    | 	at metabase.core$_main.doInvoke(core.clj:177)
metabase    | 	at clojure.lang.RestFn.invoke(RestFn.java:397)
metabase    | 	at clojure.lang.AFn.applyToHelper(AFn.java:152)
metabase    | 	at clojure.lang.RestFn.applyTo(RestFn.java:132)
metabase    | 	at metabase.core.main(Unknown Source)
metabase    | Caused by: liquibase.exception.MigrationFailedException: Migration failed for change set migrations/000_migrations.yaml::313::jeff303:
metabase    |      Reason: liquibase.exception.DatabaseException: ERROR: there is no unique constraint matching given keys for referenced table "core_user" [Failed SQL: (0) CREATE TABLE "public"."secret" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL, "version" INTEGER DEFAULT 1 NOT NULL, "creator_id" INTEGER, "created_at" TIMESTAMP with time zone NOT NULL, "updated_at" TIMESTAMP with time zone, "name" VARCHAR(254) NOT NULL, "kind" VARCHAR(254) NOT NULL, "source" VARCHAR(254), "value" BYTEA NOT NULL, CONSTRAINT "secret_pkey" PRIMARY KEY ("id", "version"), CONSTRAINT "fk_secret_ref_user_id" FOREIGN KEY ("creator_id") REFERENCES "public"."core_user"("id"))]
metabase    | 	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:696)
metabase    | 	at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:56)
metabase    | 	at liquibase.changelog.ChangeLogIterator$2.lambda$null$0(ChangeLogIterator.java:113)
metabase    | 	at liquibase.Scope.lambda$child$0(Scope.java:180)
metabase    | 	at liquibase.Scope.child(Scope.java:189)
metabase    | 	at liquibase.Scope.child(Scope.java:179)
metabase    | 	at liquibase.Scope.child(Scope.java:158)
metabase    | 	at liquibase.changelog.ChangeLogIterator$2.lambda$run$1(ChangeLogIterator.java:112)
metabase    | 	at liquibase.Scope.lambda$child$0(Scope.java:180)
metabase    | 	at liquibase.Scope.child(Scope.java:189)
metabase    | 	at liquibase.Scope.child(Scope.java:179)
metabase    | 	at liquibase.Scope.child(Scope.java:158)
metabase    | 	at liquibase.Scope.child(Scope.java:243)
metabase    | 	at liquibase.changelog.ChangeLogIterator$2.run(ChangeLogIterator.java:93)
metabase    | 	at liquibase.Scope.lambda$child$0(Scope.java:180)
metabase    | 	at liquibase.Scope.child(Scope.java:189)
metabase    | 	at liquibase.Scope.child(Scope.java:179)
metabase    | 	at liquibase.Scope.child(Scope.java:158)
metabase    | 	at liquibase.Scope.child(Scope.java:243)
metabase    | 	at liquibase.Scope.child(Scope.java:247)
metabase    | 	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:65)
metabase    | 	... 45 more
metabase    | Caused by: liquibase.exception.DatabaseException: ERROR: there is no unique constraint matching given keys for referenced table "core_user" [Failed SQL: (0) CREATE TABLE "public"."secret" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL, "version" INTEGER DEFAULT 1 NOT NULL, "creator_id" INTEGER, "created_at" TIMESTAMP with time zone NOT NULL, "updated_at" TIMESTAMP with time zone, "name" VARCHAR(254) NOT NULL, "kind" VARCHAR(254) NOT NULL, "source" VARCHAR(254), "value" BYTEA NOT NULL, CONSTRAINT "secret_pkey" PRIMARY KEY ("id", "version"), CONSTRAINT "fk_secret_ref_user_id" FOREIGN KEY ("creator_id") REFERENCES "public"."core_user"("id"))]
metabase    | 	at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:397)
metabase    | 	at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:83)
metabase    | 	at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:151)
metabase    | 	at liquibase.database.AbstractJdbcDatabase.execute(AbstractJdbcDatabase.java:1279)
metabase    | 	at liquibase.database.AbstractJdbcDatabase.executeStatements(AbstractJdbcDatabase.java:1261)
metabase    | 	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:661)
metabase    | 	... 65 more
metabase    | Caused by: org.postgresql.util.PSQLException: ERROR: there is no unique constraint matching given keys for referenced table "core_user"
metabase    | 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
metabase    | 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365)
metabase    | 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355)
metabase    | 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490)
metabase    | 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408)
metabase    | 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:329)
metabase    | 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:315)
metabase    | 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:291)
metabase    | 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:286)
metabase    | 	at com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:75)
metabase    | 	at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:393)
metabase    | 	... 70 more
metabase    | 2023-04-27 14:03:29,284 INFO metabase.core :: Metabase Shutting Down ...
metabase    | 2023-04-27 14:03:29,291 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
metabase    | 2023-04-27 14:03:29,302 INFO metabase.core :: Metabase Shutdown COMPLETE

The last databasechanlog in my db is

description,comments
"createIndex indexName=idx_query_execution_card_id_started_at, tableName=query_execution",Added 0.41.3 -- Add index to QueryExecution card_id + started_at to fix performance issue

thank for help

GC

TonyC · April 28, 2023, 7:30am

Hi, Can you share the Admin -> Troubleshooting -> Diagnostic Info please

Additionally can you have a look at the table core_user in your application database? Since this script is failing to execute for some reason:

github.com

metabase/metabase/blob/fb79343b47d521effa9db738894e118cdfad80ea/resources/migrations/000_migrations.yaml#L8753


      
                    dbms: h2
                    sql: >-
                      UPDATE pulse p
                      SET collection_id = (
                        SELECT d.collection_id
                        FROM report_dashboard d
                        WHERE d.id = p.dashboard_id
                      )
                      WHERE dashboard_id IS NOT NULL;
          
          
- changeSet:
              id: 313
              author: jeff303
              comment: Added 0.42.0 - Secret domain object.
              validCheckSum: ANY
              changes:
                - createTable:
                    tableName: secret
                    remarks: Storage for managed secrets (passwords, binary data, etc.)
                    columns:
                      - column:

My assumption is that in core_user you have duplicate entries that are a must for that script to run no unique constraint matching

guillaume.cadoret · April 28, 2023, 8:18pm

Hello

Thanks very much for help.

Actually, you were right. the message meant that there were duplicate entries in my core_user table , I thought it did not , but I was probably wrong...
Anyway, now I have another problem...
Can you please help here?
I made a massive upgrade then from 0.41.4 to lastest version 0.46.2
Liquibase worked perfectly! and the app started without any issue.
But, I dont see my dashboard anymore ..

Here is my diagnostic infos

{
  "browser-info": {
    "language": "fr-FR",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.19+7",
    "java.vendor": "Eclipse Adoptium",
    "java.vendor.url": "https://adoptium.net/",
    "java.version": "11.0.19",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.19+7",
    "os.name": "Linux",
    "os.version": "4.14.311-233.529.amzn2.x86_64",
    "user.language": "en",
    "user.timezone": "Europe/Paris"
  },
  "metabase-info": {
    "databases": [
      "postgres"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "11.16"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.5.1"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2023-04-28",
      "tag": "v0.46.2",
      "branch": "release-x.46.x",
      "hash": "8967c94"
    },
    "settings": {
      "report-timezone": "Europe/Paris"
    }
  }
}

do you need another informations ? 

Thank you very much

Kindest regards, 

Guillaume

Luiggi · April 28, 2023, 8:21pm

what do you mean that you don't see the dashboard anymore?

guillaume.cadoret · April 28, 2023, 8:38pm

Thank for help

When I go onto the dashboard I see the message :

This dashboard is looking empty.

But before migration , it was not

Luiggi · April 28, 2023, 8:40pm

do you have a backup of the old instance? you should be restoring a backup and trying to see which upgrade broke the dashboard, this shouldn't be destructive

guillaume.cadoret · April 28, 2023, 8:42pm

Yes I have a backup of the old instance. Ok I will do that, thanks

guillaume.cadoret · April 28, 2023, 10:32pm

So I restore my dump , set version to 0.43.2, start metabse. Now I can login wtih sso google. but my dashboard are still gone..

Luiggi · May 1, 2023, 12:01pm

Have you checked the archive? Things that are saved to the db can’t dissapear.

Luiggi · May 3, 2023, 2:37pm

hi @guillaume.cadoret seems you deleted some posts, were you able to find the lost entities?

guillaume.cadoret · May 10, 2023, 1:10pm

yep, actually I did. But i lost some of my dashboard, very weird..

there is a way to change FQDM url in some dashboard if the app is move from one FQDM ton another one ?
Thanks

Luiggi · May 10, 2023, 11:17pm

Can you provide an example?