Initial install failing after DB build

Hello all

I have just stood up metabase-0.64 in a k8s cluster with an external postgres instance. After finally assigning 2Gi RAM (!) it was able to start up and connected correctly and built the schema - it then failed for no reason I can figure out from the incredibly dense stacktrace - I have put the initial failure below - after that is a huge sequence of log lines, but maybe this is enough for some suggestions for approaches to debug?

We found the test install of Metabase on docker really nice and wanted to set it up properly…

026-10-29 15:45:01,448 INFO task.impl :: Initializing task SendFollowUpEmails πŸ“†
2026-10-29 15:45:01,484 INFO core.core :: Looks like this is a new installation ... preparing setup wizard
2026-10-29 15:45:01,557 INFO core.core :: Please use the following URL to setup your Metabase installation:

http://0.0.0.0:3000/setup/


2026-10-29 15:45:01,589 INFO util.files :: File system at #object[java.net.URI 0x7b70662 jar:file:/app/metabase.jar!/sample-database.sqlite] already exists
2026-10-29 15:45:01,590 INFO util.files :: Extract file /sample-database.sqlite -> /plugins/sample-database.sqlite
2026-10-29 15:45:01,723 INFO task.sync-databases-trigger :: A trigger for "Sync and Analyze" of Database 1 has been enabled with schedule: "0 48 * * * ? *"
2026-10-29 15:45:01,786 INFO task.sync-databases-trigger :: A trigger for "Scan Field Values" of Database 1 has been enabled with schedule: "0 0 19 * * ? *"
2026-10-29 15:45:02,160 WARN core.core :: Received system signal: SIGTERM
2026-10-29 15:45:02,165 INFO core.core :: Metabase Shutting Down ...
2026-10-29 15:45:02,169 INFO startup.core :: Running shutdown logic MqShutdown
2026-10-29 15:45:02,171 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793288700377 shutting down.
2026-10-29 15:45:02,171 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793288700377 paused.
2026-10-29 15:45:02,173 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793288700377 shutdown complete.
2026-10-29 15:45:02,176 INFO server.instance :: Shutting Down Embedded Jetty Webserver
2026-10-29 15:45:02,193 INFO analytics.prometheus :: Prometheus web-server shut down
2026-10-29 15:45:02,207 INFO notification.send :: Shutting down notification dispatchers... {mb-dispatcher-count=2}
2026-10-29 15:45:02,215 INFO notification.send :: Starting notification thread pool with 3 threads {mb-dispatcher-count=2}
2026-10-29 15:45:02,218 INFO notification.send :: Gracefully shutting down notification dispatcher with 0 pending notifications to process {mb-dispatcher-count=2}
2026-10-29 15:45:02,919 WARN metrics.core :: Failed to sync dimensions for metric card 9
clojure.lang.ExceptionInfo: error when updating metric: :metabase-search/index-update-ms {:toucan2/context-trace [["Apply after row fn" {:toucan2.tools.after/query-type :toucan.query-type/update.instances, :toucan2.tools.after/model :model/Card}] ["execute SQL with class com.mchange.v2.c3p0.impl.NewProxyConnectionJdbc43Full" {:toucan2.jdbc.query/sql-args []}
	at iapetos.core$inc$fn__79596.invoke(core.clj:231)
	at iapetos.core$inc.invokeStatic(core.clj:231)
	at iapetos.core$inc.invoke(core.clj:217)
	at metabase.analytics.prometheus$inc_BANG_.invokeStatic(prometheus.clj:1065)

Happy to post more - I’m not really sure where to start looking - the final part of the stack trace is as follows:

	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:135)
	at metabase.core.bootstrap.main(Unknown Source)
Caused by: java.lang.RuntimeException: error when updating metric: :metabase-notification/template-create
	... 366 more
Caused by: java.lang.IllegalArgumentException: No implementation of method: :get of protocol: #'iapetos.registry/Registry found for class: nil
	at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:585)
	at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:577)
	at iapetos.registry$fn__79438$G__79416__79447.invoke(registry.clj:10)
	at iapetos.core$inc$fn__79596.invoke(core.clj:232)
	... 365 more
2026-10-29 15:45:04,251 INFO notification.send :: Notification worker shut down successfully {mb-dispatcher-count=2}
2026-10-29 15:45:04,251 INFO notification.send :: All notification workers shut down successfully {mb-dispatcher-count=2}
2026-10-29 15:45:04,252 INFO core.core :: Metabase Shutdown COMPLETE

Thx!

cam

Weird.. I just went and tailed the pod log and realised it seems to be starting up and failing again regularly - possibly k8s restarts? But the error is now different:

2026-10-29 19:41:58,209 INFO driver.impl :: Initializing driver :sqlite...
2026-10-29 19:41:58,254 INFO task.impl :: Initializing task SendNotifications πŸ“†
2026-10-29 19:41:58,299 INFO task.impl :: Initializing task RefreshSlackChannelsAndUsers πŸ“†
2026-10-29 19:41:58,370 INFO task.impl :: Initializing task TransformDagRunHeartbeat πŸ“†
2026-10-29 19:41:58,371 INFO task.impl :: Initializing task TransformRunHeartbeat πŸ“†
2026-10-29 19:41:58,371 INFO task.impl :: Initializing task SendWarnPulseRemovalEmail πŸ“†
2026-10-29 19:41:58,402 INFO task.impl :: Initializing task SearchIndexInit πŸ“†
2026-10-29 19:41:58,415 INFO task.impl :: Initializing task CancelRuns πŸ“†
2026-10-29 19:41:58,417 INFO transforms.canceling :: Scheduling the cancelation background task.
2026-10-29 19:41:58,418 INFO task.impl :: Initializing task SendAnonymousUsageStats πŸ“†
2026-10-29 19:41:58,483 INFO task.impl :: Initializing task SessionCleanup πŸ“†
2026-10-29 19:41:58,532 INFO task.impl :: Initializing task QuartzTracing πŸ“†
2026-10-29 19:41:58,533 INFO tracing.quartz :: Initializing Quartz tracing instrumentation
2026-10-29 19:41:58,535 INFO task.impl :: Initializing task FlushMessageOutbox πŸ“†
2026-10-29 19:41:58,590 INFO task.impl :: Initializing task RunTransform πŸ“†
2026-10-29 19:41:58,591 INFO transforms.schedule :: Initializing transform job execution jobs
2026-10-29 19:41:58,613 INFO transforms.schedule :: Initializing schedule for transform job 1
2026-10-29 19:41:58,667 INFO transforms.schedule :: Initializing schedule for transform job 2
2026-10-29 19:41:58,719 INFO transforms.schedule :: Initializing schedule for transform job 3
2026-10-29 19:41:58,774 INFO transforms.schedule :: Initializing schedule for transform job 4
2026-10-29 19:41:58,830 INFO task.impl :: Initializing task UsageMetadataProcess πŸ“†
2026-10-29 19:41:58,878 INFO task.impl :: Initializing task SaveReport πŸ“†
2026-10-29 19:41:58,927 INFO task.impl :: Initializing task TransformRunReaper πŸ“†
2026-10-29 19:41:58,932 INFO task.impl :: Initializing task TaskRunReaper πŸ“†
2026-10-29 19:41:58,949 INFO task.impl :: Initializing task SendTransformFailureDigest πŸ“†
2026-10-29 19:41:59,003 INFO task.impl :: Initializing task SendFollowUpEmails πŸ“†
2026-10-29 19:41:59,072 INFO core.core :: Looks like this is a new installation ... preparing setup wizard
2026-10-29 19:41:59,077 INFO core.core :: Please use the following URL to setup your Metabase installation:

http://0.0.0.0:3000/setup/


2026-10-29 19:41:59,119 INFO util.files :: File system at #object[java.net.URI 0x169dbbfe jar:file:/app/metabase.jar!/sample-database.sqlite] already exists
2026-10-29 19:41:59,120 INFO util.files :: Extract file /sample-database.sqlite -> /plugins/sample-database.sqlite
2026-10-29 19:41:59,205 INFO core.core :: Not installing EE audit app DB
2026-10-29 19:41:59,206 INFO notification.seed :: Seeding default notifications
2026-10-29 19:41:59,530 INFO notification.seed :: Seeded notifications: {:skip 8}
2026-10-29 19:41:59,542 INFO startup.core :: Running startup validation map-tile-server-allowed-networks
2026-10-29 19:41:59,543 INFO startup.core :: Running startup validation check-for-removed-env-vars
2026-10-29 19:41:59,543 INFO startup.core :: Running startup validation warehouse-allowed-networks
2026-10-29 19:41:59,544 INFO startup.core :: Running startup validation llm-allowed-networks
2026-10-29 19:41:59,544 INFO startup.core :: Running startup validation http-channel-allowed-networks
2026-10-29 19:41:59,544 INFO startup.core :: Running startup validation MqBackendValidation
2026-10-29 19:41:59,545 INFO startup.core :: Running startup validation oidc-allowed-networks
2026-10-29 19:41:59,546 INFO startup.core :: Running setup logic SearchIndexInit β˜‘οΈ
2026-10-29 19:41:59,548 INFO startup.core :: Running setup logic clear-token-cache β˜‘οΈ
2026-10-29 19:41:59,582 INFO startup.core :: Running setup logic MqStart β˜‘οΈ
2026-10-29 19:41:59,586 INFO mq.init :: Queue backend: :queue.backend/quartz
2026-10-29 19:41:59,596 INFO search.engine :: Search will be served by the :search.engine/appdb engine
2026-10-29 19:41:59,599 INFO search.core :: Initializing search indexes
2026-10-29 19:41:59,618 INFO jdbcjobstore.JobStoreTX :: ClusterManager: detected 1 failed or restarted instances.
2026-10-29 19:41:59,619 INFO jdbcjobstore.JobStoreTX :: ClusterManager: Scanning for instance "metabase-74864948cc-lpd8t1793302827960"'s failed in-progress jobs.
2026-10-29 19:41:59,644 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793302916485 started.
2026-10-29 19:41:59,645 INFO task.impl :: Task scheduler started
2026-10-29 19:41:59,650 INFO util.queue :: Starting listener search-index-update with 1 threads 🎧
2026-10-29 19:41:59,652 INFO core.core :: Metabase Initialization COMPLETE
2026-10-29 19:41:59,653 INFO core.core :: Metabase Initialization COMPLETE in 14.5 s (JVM uptime: 49.3 s)
2026-10-29 19:41:59,703 INFO search.core :: Found existing search index, and using it.
2026-10-29 19:41:59,799 INFO task.send-pulses :: Initializing SendPulse triggers for dashboard subscriptions {mb-quartz-job-type=InitSendPulseTriggers}
2026-10-29 19:41:59,867 INFO task.send :: Initializing SendNotification triggers {mb-quartz-job-type=InitNotificationTriggers}
2026-10-29 19:41:59,904 INFO task.refresh-slack-channel-user-cache :: Slack is not configured, not refreshing slack user/channel cache. {mb-quartz-job-type=RefreshCacheOnStartup}
2026-10-29 19:42:07,591 INFO task.suggested-prompts-generator :: No suggested prompts found for b5716059-ad40-4d83-a4e1-673af020b2d8. Generating suggested prompts. {mb-quartz-job-type=SuggestedPromptsGenerator}
2026-10-29 19:42:08,556 INFO metabot.example-question-generator :: Generating example questions natively {:table-count 1, :metric-count 2} {mb-quartz-job-type=SuggestedPromptsGenerator}
2026-10-29 19:42:08,576 WARN metabot.example-question-generator :: Example question generation failed for one item: No LLM provider connection named "anthropic" is configured.
2026-10-29 19:42:08,591 WARN metabot.example-question-generator :: Example question generation failed for one item: No LLM provider connection named "anthropic" is configured.
2026-10-29 19:42:08,591 WARN metabot.example-question-generator :: Example question generation failed for one item: No LLM provider connection named "anthropic" is configured.
2026-10-29 19:42:08,594 ERROR task.suggested-prompts-generator :: Suggested prompts generation failed: No LLM provider connection named "anthropic" is configured. {mb-quartz-job-type=SuggestedPromptsGenerator}
2026-10-29 19:42:32,148 WARN core.core :: Received system signal: SIGTERM
2026-10-29 19:42:32,150 INFO core.core :: Metabase Shutting Down ...
2026-10-29 19:42:32,151 INFO util.queue :: Stopping listener search-index-update...
2026-10-29 19:42:32,152 INFO util.queue :: Stopping listener search-index-update...done
2026-10-29 19:42:32,153 INFO startup.core :: Running shutdown logic MqShutdown
2026-10-29 19:42:32,154 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793302916485 shutting down.
2026-10-29 19:42:32,154 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793302916485 paused.
2026-10-29 19:42:32,155 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-74864948cc-lpd8t1793302916485 shutdown complete.
2026-10-29 19:42:32,156 INFO server.instance :: Shutting Down Embedded Jetty Webserver
2026-10-29 19:42:32,168 INFO analytics.prometheus :: Prometheus web-server shut down
2026-10-29 19:42:32,170 INFO notification.send :: Shutting down notification dispatchers... {mb-dispatcher-count=2}
2026-10-29 19:42:32,173 INFO notification.send :: Starting notification thread pool with 3 threads {mb-dispatcher-count=2}
2026-10-29 19:42:32,176 INFO notification.send :: Gracefully shutting down notification dispatcher with 0 pending notifications to process {mb-dispatcher-count=2}
2026-10-29 19:42:33,176 INFO notification.send :: Notification worker shut down successfully {mb-dispatcher-count=2}
2026-10-29 19:42:33,177 INFO notification.send :: Starting notification thread pool with 5 threads {mb-dispatcher-count=2}
2026-10-29 19:42:33,179 INFO notification.send :: Gracefully shutting down notification dispatcher with 0 pending notifications to process {mb-dispatcher-count=2}
2026-10-29 19:42:34,179 INFO notification.send :: Notification worker shut down successfully {mb-dispatcher-count=2}
2026-10-29 19:42:34,180 INFO notification.send :: All notification workers shut down successfully {mb-dispatcher-count=2}
2026-10-29 19:42:34,180 INFO core.core :: Metabase Shutdown COMPLETE

I can also see a bunch of success entries in the task_history table about once per hour… anyone got a clue what is going on?

Meh.. I thought I was being clever by rebuilding my pod with env variable MB_AI_FEATURES_ENABLED set to false.. I still got a shutdown straight after startup:

2026-10-29 21:53:21,691 INFO task.impl :: Initializing task UsageMetadataProcess πŸ“†
2026-10-29 21:53:21,743 INFO task.impl :: Initializing task SaveReport πŸ“†
2026-10-29 21:53:21,798 INFO task.impl :: Initializing task TransformRunReaper πŸ“†
2026-10-29 21:53:21,806 INFO task.impl :: Initializing task TaskRunReaper πŸ“†
2026-10-29 21:53:21,813 INFO task.impl :: Initializing task SendTransformFailureDigest πŸ“†
2026-10-29 21:53:21,872 INFO task.impl :: Initializing task SendFollowUpEmails πŸ“†
2026-10-29 21:53:21,934 INFO core.core :: Looks like this is a new installation ... preparing setup wizard
2026-10-29 21:53:21,940 INFO core.core :: Please use the following URL to setup your Metabase installation:

http://0.0.0.0:3000/setup/


2026-10-29 21:53:21,984 INFO util.files :: File system at #object[java.net.URI 0x771976e2 jar:file:/app/metabase.jar!/sample-database.sqlite] already exists
2026-10-29 21:53:21,985 INFO util.files :: Extract file /sample-database.sqlite -> /plugins/sample-database.sqlite
2026-10-29 21:53:22,066 INFO core.core :: Not installing EE audit app DB
2026-10-29 21:53:22,067 INFO notification.seed :: Seeding default notifications
2026-10-29 21:53:22,474 INFO notification.seed :: Seeded notifications: {:skip 8}
2026-10-29 21:53:22,485 INFO startup.core :: Running startup validation map-tile-server-allowed-networks
2026-10-29 21:53:22,486 INFO startup.core :: Running startup validation check-for-removed-env-vars
2026-10-29 21:53:22,486 INFO startup.core :: Running startup validation warehouse-allowed-networks
2026-10-29 21:53:22,486 INFO startup.core :: Running startup validation llm-allowed-networks
2026-10-29 21:53:22,486 INFO startup.core :: Running startup validation http-channel-allowed-networks
2026-10-29 21:53:22,487 INFO startup.core :: Running startup validation MqBackendValidation
2026-10-29 21:53:22,487 INFO startup.core :: Running startup validation oidc-allowed-networks
2026-10-29 21:53:22,488 INFO startup.core :: Running setup logic SearchIndexInit β˜‘οΈ
2026-10-29 21:53:22,489 INFO startup.core :: Running setup logic clear-token-cache β˜‘οΈ
2026-10-29 21:53:22,516 INFO startup.core :: Running setup logic MqStart β˜‘οΈ
2026-10-29 21:53:22,517 INFO mq.init :: Queue backend: :queue.backend/quartz
2026-10-29 21:53:22,518 INFO search.engine :: Search will be served by the :search.engine/appdb engine
2026-10-29 21:53:22,520 INFO search.core :: Initializing search indexes
2026-10-29 21:53:22,541 INFO jdbcjobstore.JobStoreTX :: ClusterManager: detected 1 failed or restarted instances.
2026-10-29 21:53:22,542 INFO jdbcjobstore.JobStoreTX :: ClusterManager: Scanning for instance "metabase-74864948cc-lpd8t1793310625105"'s failed in-progress jobs.
2026-10-29 21:53:22,556 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-6475c5899c-bp4861793310799889 started.
2026-10-29 21:53:22,556 INFO task.impl :: Task scheduler started
2026-10-29 21:53:22,561 INFO util.queue :: Starting listener search-index-update with 1 threads 🎧
2026-10-29 21:53:22,564 INFO core.core :: Metabase Initialization COMPLETE
2026-10-29 21:53:22,567 INFO core.core :: Metabase Initialization COMPLETE in 13.1 s (JVM uptime: 47.1 s)
2026-10-29 21:53:22,616 INFO search.core :: Found existing search index, and using it.
2026-10-29 21:53:23,046 INFO task.send-pulses :: Initializing SendPulse triggers for dashboard subscriptions {mb-quartz-job-type=InitSendPulseTriggers}
2026-10-29 21:53:23,085 INFO task.send :: Initializing SendNotification triggers {mb-quartz-job-type=InitNotificationTriggers}
2026-10-29 21:53:23,144 INFO task.refresh-slack-channel-user-cache :: Slack is not configured, not refreshing slack user/channel cache. {mb-quartz-job-type=RefreshCacheOnStartup}
2026-10-29 21:53:30,760 INFO task.suggested-prompts-generator :: Metabot is disabled. Skipping suggested prompt generation. {mb-quartz-job-type=SuggestedPromptsGenerator}
2026-10-29 21:53:58,421 WARN core.core :: Received system signal: SIGTERM
2026-10-29 21:53:58,423 INFO core.core :: Metabase Shutting Down ...
2026-10-29 21:53:58,424 INFO util.queue :: Stopping listener search-index-update...
2026-10-29 21:53:58,426 INFO util.queue :: Stopping listener search-index-update...done
2026-10-29 21:53:58,427 INFO startup.core :: Running shutdown logic MqShutdown
2026-10-29 21:53:58,428 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-6475c5899c-bp4861793310799889 shutting down.
2026-10-29 21:53:58,428 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-6475c5899c-bp4861793310799889 paused.
2026-10-29 21:53:58,430 INFO core.QuartzScheduler :: Scheduler MetabaseScheduler_$_metabase-6475c5899c-bp4861793310799889 shutdown complete.
2026-10-29 21:53:58,431 INFO server.instance :: Shutting Down Embedded Jetty Webserver
2026-10-29 21:53:58,428 ERROR core.QuartzSchedulerThread :: Runtime error occurred in main trigger firing loop.
java.lang.IllegalStateException: JobStore is shutdown - aborting retry
        at org.quartz.impl.jdbcjobstore.JobStoreSupport.retryExecuteInNonManagedTXLock(JobStoreSupport.java:3833)
        at org.quartz.impl.jdbcjobstore.JobStoreSupport.releaseAcquiredTrigger(JobStoreSupport.java:2940)
        at org.quartz.core.QuartzSchedulerThread.releaseIfScheduleChangedSignificantly(QuartzSchedulerThread.java:472)
        at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:333)
2026-10-29 21:53:58,454 INFO analytics.prometheus :: Prometheus web-server shut down
2026-10-29 21:53:58,457 INFO notification.send :: Shutting down notification dispatchers... {mb-dispatcher-count=2}
2026-10-29 21:53:58,461 INFO notification.send :: Starting notification thread pool with 3 threads {mb-dispatcher-count=2}
2026-10-29 21:53:58,465 INFO notification.send :: Gracefully shutting down notification dispatcher with 0 pending notifications to process {mb-dispatcher-count=2}
2026-10-29 21:53:59,465 INFO notification.send :: Notification worker shut down successfully {mb-dispatcher-count=2}
2026-10-29 21:53:59,466 INFO notification.send :: Starting notification thread pool with 5 threads {mb-dispatcher-count=2}
2026-10-29 21:53:59,467 INFO notification.send :: Gracefully shutting down notification dispatcher with 0 pending notifications to process {mb-dispatcher-count=2}
2026-10-29 21:54:00,467 INFO notification.send :: Notification worker shut down successfully {mb-dispatcher-count=2}
2026-10-29 21:54:00,468 INFO notification.send :: All notification workers shut down successfully {mb-dispatcher-count=2}
2026-10-29 21:54:00,469 INFO core.core :: Metabase Shutdown COMPLETE

Doh! This was because I did not have my probes setot correct endpoints as described clearly here..

k8s was correctly restarting my container - fixed now!