Problem in upgrade from 0.35.4 to 0.36.x, 0.37.x etc (MySQL) - group_table_access_policy already exists

Hello guys,

Today I went down to see what was going on. It first looked like it wanted to create a new table (group_table_access_policy) and that my user didn’t have CREATE rights.

Ok, easy: I gave them to him and tried again. Here, however, a strange problem appears. Although the table does not exist when the service starts, I get an error because the table would already exist.

I have checked this thoroughly several times. But nothing: it doesn’t work.

Any idea?

Here the log:

Best regards

Marcello

Hi @marciuz

So you’re saying that the table group_table_access_policy does not exist in your 0.35.4 application database?

Interesting, since that migration has been around since 0.30.0, so it seems like you have ended up with a corruption in your application database at some point:
https://github.com/metabase/metabase/blob/master/resources/migrations/000_migrations.yaml#L4192

Try starting with MB_DB_AUTOMIGRATE=false, so it shows all the migrations that it’s trying to do:
https://www.metabase.com/docs/latest/operations-guide/environment-variables.html#mb_db_automigrate

You don’t need to upgrade in steps - you can upgrade directly to the version you want.

Hi, this is the output with the MB_DB_AUTOMIGRATE=false

@marciuz

So you’re saying that the table group_table_access_policy does not exist in your 0.35.4 application database?

Now run the SQL from the output on your database directly - without Metabase running.
Once you have done that, then start 0.38.1 normally.

Thanks it works!

I report some errors: in the SQL there are some queries not correct for MySQL:

CREATE EXTENSION IF NOT EXISTS citext;
ALTER TABLE metabase.core_user MODIFY email CITEXT;
CREATE INDEX idx_lower_email ON metabase.core_user(lower(email));
ALTER TABLE metabase.core_user MODIFY email VARCHAR_IGNORECASE(254);

In any case don’t seems to be a mayor issue for the execution. But this would probably should blocked the automatic execution of updates.

Thanks for your help :slight_smile:

@flamber, is not finished :frowning:

The connection with a server MySQL (with SSH key and tunnel) goes down without apparent reason and if I restart the server works (for a while).

In addition, I have this page as administration

Any idea?

@marciuz Different problems. The migration has completed.

  1. Post “Diagnostic Info” from Admin > Troubleshooting.
  2. Are you starting with any environment variables besides MB_DB_* ?
  3. Check the server console (or Admin > Troubleshooting > Logs) for errors, and your browser console.
  4. I don’t know your setup, but you’re likely seeing this issue - see last comment for workaround:
    https://github.com/metabase/metabase/issues/6114 - upvote by clicking :+1: on the first post

Hi,

  1. here the the Diagnostic info:
{
  "browser-info": {
    "language": "en-US",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "14.0.2+12-Ubuntu-120.04",
    "java.vendor": "Private Build",
    "java.vendor.url": "Unknown",
    "java.version": "14.0.2",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "14.0.2+12-Ubuntu-120.04",
    "os.name": "Linux",
    "os.version": "5.4.0-65-generic",
    "user.language": "en",
    "user.timezone": "Europe/Berlin"
  },
  "metabase-info": {
    "databases": [
      "mysql",
      "googleanalytics"
    ],
    "hosting-env": "unknown",
    "application-database": "mysql",
    "application-database-details": {
      "database": {
        "name": "MySQL",
        "version": "5.7.32-35"
      },
      "jdbc-driver": {
        "name": "MariaDB Connector/J",
        "version": "2.6.2"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2021-03-03",
      "tag": "v0.38.1",
      "branch": "release-x.38.x",
      "hash": "79ef63a"
    },
    "settings": {
      "report-timezone": "Europe/Brussels"
    }
  }
}
  1. I think no… this is my metabase.service
[Unit]
Description=Metabase application service
Documentation=https://www.metabase.com/docs/latest

[Service]
WorkingDirectory=/opt/metabase

Environment=MB_DB_TYPE=mysql
Environment=MB_DB_DBNAME=metabase
Environment=MB_DB_PORT=3306
Environment=MB_DB_USER=metabase
Environment=MB_DB_PASS=###MY-PASSWORD###
Environment=MB_DB_HOST=127.0.0.1

#Environment=MB_DB_AUTOMIGRATE=false

Environment=MB_JETTY_PORT=3003
Environment=MB_ENCRYPTION_SECRET_KEY=###MY-SECRET_KEY###
ExecStart=/usr/bin/java -Xms128m -Xmx512m -jar metabase.jar
User=metabase
Type=simple
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target 
  1. I often see “Too many redirect” in the Browser Console.
    In the log I have a LOT of errors like this https://gist.github.com/marciuz/862850e1d228b3e641892259a32a7cff

:frowning:

@marciuz This is getting confusing. Let’s deal with one thing in each topic. Please open a topic for each problem and include “Diagnostic Info” and relevant logs for each.

Ok, sorry for this, but before you answer I thought was issues related to the upgrade (indeed happens only now…)

I will open the single topic.

Best,

M

@marciuz

But the upgrade was a success - even though I don’t understand why it was failing, but you didn’t comment on the table that was causing problems or why it was trying to run migrations from 0.30.0 even though you were using 0.35.4, which makes me think that you have (or had) a corruption in the application database.

If you start out upgrading and then hit some problem later on, then it’s very likely not related to the actual upgrade (migration procedure), but is probably something else.