Fails to start after Google OIDC configuration

1

Metabase has gone into a crash loop after me trying to enable Google OIDC for logins. It fails with these logs (no ERROR or WARN levels at all)

metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,472 INFO core.QuartzScheduler :: Scheduler meta-data: Quartz Scheduler (v2.3.2) 'MetabaseScheduler' with instanceId 'metabase-d695cdcf9-84c8w1669407890434'
metabase-d695cdcf9-84c8w metabase   Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
metabase-d695cdcf9-84c8w metabase   NOT STARTED.
metabase-d695cdcf9-84c8w metabase   Currently in standby mode.
metabase-d695cdcf9-84c8w metabase   Number of jobs executed: 0
metabase-d695cdcf9-84c8w metabase   Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
metabase-d695cdcf9-84c8w metabase   Using job-store 'org.quartz.impl.jdbcjobstore.JobStoreTX' - which supports persistence. and is clustered.
metabase-d695cdcf9-84c8w metabase
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,472 INFO impl.StdSchedulerFactory :: Quartz scheduler 'MetabaseScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,472 INFO impl.StdSchedulerFactory :: Quartz scheduler version: 2.3.2
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,603 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-d695cdcf9-84c8w1669407890434 paused.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,604 INFO metabase.task :: Task scheduler initialized into standby mode.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,606 INFO metabase.task :: Initializing task SyncDatabases 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,719 INFO task.sync-databases :: Updated default schedules for 0 databases
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,724 INFO metabase.task :: Initializing task PersistRefresh 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,805 INFO driver.impl :: Initializing driver :sql...
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,805 INFO driver.impl :: Initializing driver :sql-jdbc...
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,806 INFO driver.impl :: Initializing driver :postgres...
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,841 INFO metabase.task :: Initializing task CheckForNewVersions 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,911 INFO metabase.task :: Initializing task PersistPrune 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,921 INFO metabase.task :: Initializing task SendAnonymousUsageStats 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:50,968 INFO metabase.task :: Initializing task RefreshSlackChannelsAndUsers 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,173 INFO metabase.task :: Initializing task SendAbandomentEmails 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,249 INFO metabase.task :: Initializing task SendPulses 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,296 INFO metabase.task :: Initializing task SendFollowUpEmails 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,369 INFO metabase.task :: Initializing task TaskHistoryCleanup 📆
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,479 INFO jdbcjobstore.JobStoreTX :: ClusterManager: detected 1 failed or restarted instances.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,480 INFO jdbcjobstore.JobStoreTX :: ClusterManager: Scanning for instance "metabase-d695cdcf9-84c8w1669407605110"'s failed in-progress jobs.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,494 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-d695cdcf9-84c8w1669407890434 started.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,495 INFO metabase.task :: Task scheduler started
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,496 INFO metabase.core :: Metabase Initialization COMPLETE
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,602 INFO task.refresh-slack-channel-user-cache :: Starting Slack user/channel startup cache refresh...
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:51,604 INFO integrations.slack :: Refreshing slack channels and usernames.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:24:52,245 INFO task.refresh-slack-channel-user-cache :: Slack user/channel startup cache refreshed with 118 entries, took 642ms.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,586 INFO metabase.core :: Metabase Shutting Down ...
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,586 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-d695cdcf9-84c8w1669407890434 shutting down.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,587 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-d695cdcf9-84c8w1669407890434 paused.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,590 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-d695cdcf9-84c8w1669407890434 shutdown complete.
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,591 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
metabase-d695cdcf9-84c8w metabase 2022-11-25 21:26:21,605 INFO metabase.core :: Metabase Shutdown COMPLETE
- metabase-d695cdcf9-84c8w › metabase

How do I get Metabase back up?

2

Hi @haf
Everything is indicating that the container is being shutdown (not crashed) by the host. I don't see why this should have anything to do with OIDC, but perhaps it's failing health-check or something like that. Try looking in your host logs for more details.

3

Hmm what host logs? This is k8s on GKE. Can I make metabase print if it gets a signal SIGTERM/SIGINT/SIGHUP?

4

Or maybe this is about how long it takes for the health check to become OK? What's the endpoint I should let k8s poll and how long is it supposed to take?

5

@haf
When using Docker, then you're referencing a bash script in ID 1, not Metabase itself.
https://github.com/metabase/metabase/tree/master/bin/docker

The health endpoint is /api/health

Your Kubernetes logs should tell you if it is there's unhealthy checks and then adjust your timing.

6

The API endpoint was changed to /api/health and this was the reason that it was not working (was using / before this version).