User is getting 401 "did not match stored password" after upgrading to version 46

Hi There,

We recently upgraded our Beanstalk instance with a docker image of metabase version 0.46 and it all went well. We have an external DB to which I confirmed the instance is still pointing. However, all the users are now getting 401 "did not match stored password" error on login.

We have checked the log but there is nothing in there except the same error which is showing on UI. There is no way I found using which we can reset a user's password as well. There is only one error during a start-up which is as below. Although, I think it has nothing to do with the password not matching issue here. Can someone please help?

2023-04-18 14:08:02,491 ERROR jdbcjobstore.JobStoreTX :: MisfireHandler: Error handling misfires: Couldn't store trigger 'DEFAULT.metabase.task.abandonment-emails.trigger' for 'DEFAULT.metabase.task.abandonment-emails.job' job:Couldn't retrieve job because a required class was not found: metabase.task.follow_up_emails.AbandonmentEmail org.quartz.JobPersistenceException: Couldn't store trigger 'DEFAULT.metabase.task.abandonment-emails.trigger' for 'DEFAULT.metabase.task.abandonment-emails.job' job:Couldn't retrieve job because a required class was not found: metabase.task.follow_up_emails.AbandonmentEmail [See nested exception: org.quartz.JobPersistenceException: Couldn't retrieve job because a required class was not found: metabase.task.follow_up_emails.AbandonmentEmail [See nested exception: java.lang.ClassNotFoundException: metabase.task.follow_up_emails.AbandonmentEmail]] at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeTrigger( at org.quartz.impl.jdbcjobstore.JobStoreSupport.doUpdateOfMisfiredTrigger( at org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverMisfiredJobs( at org.quartz.impl.jdbcjobstore.JobStoreSupport.doRecoverMisfires( at org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.manage( at org.quartz.impl.jdbcjobstore.JobStoreSupport$ Caused by: org.quartz.JobPersistenceException: Couldn't retrieve job because a required class was not found: metabase.task.follow_up_emails.AbandonmentEmail [See nested exception: java.lang.ClassNotFoundException: metabase.task.follow_up_emails.AbandonmentEmail] at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob( at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeTrigger( ... 5 more Caused by: java.lang.ClassNotFoundException: metabase.task.follow_up_emails.AbandonmentEmail at java.base/ Source) at clojure.lang.DynamicClassLoader.findClass( at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at clojure.lang.DynamicClassLoader.loadClass( at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Unknown Source) at metabase.task$load_class.invokeStatic(task.clj:118) at metabase.task$load_class.invoke(task.clj:117) at metabase.task.ClassLoadHelper.loadClass(task.clj:128) at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectJobDetail( at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob( ... 6 more

All users get this message? Even Admins? ... What protocol do you use for Authentication? ... So you are currently locked out of Metabase?

Yes, all users are getting the same message including Admin. Not sure about the protocol used in Authentication. It's simple username password based authentication, and we don't allow users to change password by themselves, only the admin can reset it but since the admin also can't log in we are completely locked out of Metabase.

We have restored the old version on production and it helped to regain the access but we need to upgrade and have the staging instance still locked out so would be great if someone can give a pointer.

Can you share the Admin -> Troubleshooting -> Diagnostic Info ... Are you using H2 or some other Database as the application database?

Seems like something got corrupted

I am using external MySQL DB and not the H2 db. Can you please confirm where exactly I should be checking for Admin -> Troubleshooting -> Diagnostic Info

In that case check the core_user table of the Metabase Application Database in the case. Should give you a better clue of what is going on.

You will need to access the instance which you don't have unfortunately

Yeah, I figured the Diagnostic Info can be seen after login only which I can't. And yes I already check the core_user table and everything is just the same as of production db where we are able to login. The only diff is the production have older version and staging have latest one.

Then share the Diagnostic Info of Production since you are saying they used to be identical before the upgrade right?

Hi Tony,

I have downloaded the log from beanstalk and it looks like it is trying to connect to the H2 db instead of the MySQL. I have the environment variable set in staging instance just like production. Do you have any idea if the latest version has something changed around how to point to the external DB instance other than setting the environment variables?

Here is a Diagnostic Info from production

"browser-info": {
"language": "en-US",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36",
"vendor": "Google Inc."
"system-info": {
"file.encoding": "UTF-8",
"": "OpenJDK Runtime Environment",
"java.runtime.version": "11.0.15+10",
"java.vendor": "Eclipse Adoptium",
"java.vendor.url": "",
"java.version": "11.0.15",
"": "OpenJDK 64-Bit Server VM",
"java.vm.version": "11.0.15+10",
"": "Linux",
"os.version": "4.14.281-212.502.amzn2.x86_64",
"user.language": "en",
"user.timezone": "GMT"
"metabase-info": {
"databases": [
"hosting-env": "elastic-beanstalk",
"application-database": "mysql",
"application-database-details": {
"database": {
"name": "MySQL",
"version": "8.0.28"
"jdbc-driver": {
"name": "MariaDB Connector/J",
"version": "2.7.5"
"run-mode": "prod",
"version": {
"date": "2022-06-27",
"tag": "v0.43.4",
"branch": "release-x.43.x",
"hash": "61cc28e"
"settings": {
"report-timezone": "Australia/Melbourne"

You upgraded from 0.43 directly to 0.46 or you started to upgrade 0.43 -> 0.44 -> 0.45 -> 0.46?

I upgraded directly to 0.46. And ignore my last comment on the app is using H2 Db, I can see it is connecting to MySQL too just like prod.

So should I try to upgrade to 0.44 -> 0.45 -> 0.46? I think we first tried to upgrade to 0.46 but since it was't working we tried to downgraded to 45 -> 44 but since nothing worked we moved back to 43.

Okay so jackpot :stuck_out_tongue: Metabase doesn't really support downgrades so make sure to load the old backup back in the staging environment ... start upgrading slowly major version after major version and make sure if works after every upgrade

Thanks Tony,

I will check and update how it goes.

Hi Tony,

This didn't work. It still shows the same error and the start-up log doesn't have a single error in it. Also, I found this sequential update only requires if we are upgrading from prior to 0.40 version. Is there any other work around you know that we can try? Thanks in advance.

Which version caused you issues? So you upgraded from some version to another. I am assuming you went from 43 to 44 and experienced the issue right?

Also a long shot but have you tried logging in using incognito?

Yes, we tried to upgrade from 43 to 44. And same issue in incognito as well.

I am checking table DATABASECHANGELOG and finding it only has entries till version 0.43. There are no new entries added in there for 0.44. I am not sure but would expect the migration should add some new entries for the new version in there. No?

You might be onto something here. There should be a number of schema updates:

is it possible that the user doesn't have rights to update or something?

Not sure how I can check if it's a rights issue. Won't it give at least an error in the log if that is the case?

I can see no table named persisted_info is there so definitely the DB migration is not working properly but wondering why the application is running fine if migration is failing and showing a login page

I just tried to upgrade from 43.4 to 43.7 and got this error. Any suggestion?

2023-05-02 11:04:06,672 ERROR metabase.core :: Metabase Initialization FAILED
liquibase.exception.ValidationFailedException: Validation Failed:
     1 change sets check sum
          migrations/000_migrations.yaml::268::rlotun was: 8:4ccc4d50f9b233bc6515780d9cae360b but is now: 8:9da2f706a7cd42b5101601e0106fa929

	at liquibase.changelog.DatabaseChangeLog.validate(
	at liquibase.Liquibase$
	at liquibase.Scope.lambda$child$0(
	at liquibase.Scope.child(
	at liquibase.Scope.child(
	at liquibase.Scope.child(
	at liquibase.Liquibase.runInScope(
	at liquibase.Liquibase.listUnrunChangeSets(
	at liquibase.Liquibase.listUnrunChangeSets(
	at metabase.db.liquibase$has_unrun_migrations_QMARK_.invokeStatic(liquibase.clj:109)
	at metabase.db.liquibase$has_unrun_migrations_QMARK_.invoke(liquibase.clj:99)
	at metabase.db.liquibase$migrate_up_if_needed_BANG_.invokeStatic(liquibase.clj:134)
	at metabase.db.liquibase$migrate_up_if_needed_BANG_.invoke(liquibase.clj:130)
	at metabase.db.setup$fn__34835$migrate_BANG___34840$fn__34841$fn__34842.invoke(setup.clj:68)
	at metabase.db.liquibase$fn__30492$do_with_liquibase__30497$fn__30498.invoke(liquibase.clj:59)
	at metabase.db.liquibase$fn__30492$do_with_liquibase__30497.invoke(liquibase.clj:51)
	at metabase.db.setup$fn__34835$migrate_BANG___34840$fn__34841.invoke(setup.clj:63)
	at metabase.db.setup$fn__34835$migrate_BANG___34840.invoke(setup.clj:40)
	at metabase.db.setup$fn__34894$run_schema_migrations_BANG___34899$fn__34900.invoke(setup.clj:121)
	at metabase.db.setup$fn__34894$run_schema_migrations_BANG___34899.invoke(setup.clj:115)
	at metabase.db.setup$fn__34946$setup_db_BANG___34951$fn__34952$fn__34955$fn__34956.invoke(setup.clj:147)
	at metabase.util$do_with_us_locale.invokeStatic(util.clj:724)
	at metabase.util$do_with_us_locale.invoke(util.clj:710)
	at metabase.db.setup$fn__34946$setup_db_BANG___34951$fn__34952$fn__34955.invoke(setup.clj:145)
	at metabase.db.setup$fn__34946$setup_db_BANG___34951$fn__34952.invoke(setup.clj:144)
	at metabase.db.setup$fn__34946$setup_db_BANG___34951.invoke(setup.clj:138)
	at metabase.db$setup_db_BANG_$fn__34981.invoke(db.clj:65)
	at metabase.db$setup_db_BANG_.invokeStatic(db.clj:60)
	at metabase.db$setup_db_BANG_.invoke(db.clj:51)
	at metabase.core$init_BANG__STAR_.invokeStatic(core.clj:98)
	at metabase.core$init_BANG__STAR_.invoke(core.clj:81)
	at metabase.core$init_BANG_.invokeStatic(core.clj:138)
	at metabase.core$init_BANG_.invoke(core.clj:133)
	at metabase.core$start_normally.invokeStatic(core.clj:150)
	at metabase.core$start_normally.invoke(core.clj:144)
	at metabase.core$_main.invokeStatic(core.clj:183)
	at metabase.core$_main.doInvoke(core.clj:177)
	at clojure.lang.RestFn.invoke(
	at clojure.lang.AFn.applyToHelper(
	at clojure.lang.RestFn.applyTo(
	at metabase.core.main(Unknown Source)