Metabase does not sync all tables

Hello, we have multiple connections to Redshift Serverless, but sync does not get all schemas. It gets the first 22 schemas by alphabetic order but not the other ones (lots of schemas are missing).
We tried to check the queries on Redshift Serverless but nothing special
However we get this error:

2023-11-16 22:21:04,748 ERROR driver.redshift :: Error checking schema permissions
com.amazon.redshift.util.RedshiftException: This statement has been closed.
at com.amazon.redshift.jdbc.RedshiftStatementImpl.checkClosed(RedshiftStatementImpl.java:827)
at com.amazon.redshift.jdbc.RedshiftPreparedStatement.setString(RedshiftPreparedStatement.java:419)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.setString(NewProxyPreparedStatement.java:431)
at metabase.driver.redshift$reducible_schemas_with_usage_permissions$reify__126794$fn__126795.invoke(redshift.clj:346)
at clojure.core$filter$fn__5958$fn__5959.invoke(core.clj:2822)
at clojure.core.Eduction.reduce(core.clj:7750)
at clojure.core$transduce.invokeStatic(core.clj:6946)
at clojure.core.Eduction.reduce(core.clj:7750)
at clojure.core$reduce.invokeStatic(core.clj:6885)
at clojure.core$reduce.invoke(core.clj:6868)
at metabase.driver.redshift$reducible_schemas_with_usage_permissions$reify__126794.reduce(redshift.clj:340)
at clojure.core$transduce.invokeStatic(core.clj:6946)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at metabase.task.sync_databases.SyncAndAnalyzeDatabase.execute(sync_databases.clj:85)

Here is the diagnostic info:

{
  "browser-info": {
    "language": "en-GB",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0",
    "vendor": ""
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.21+9",
    "java.vendor": "Eclipse Adoptium",
    "java.vendor.url": "https://adoptium.net/",
    "java.version": "11.0.21",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.21+9",
    "os.name": "Linux",
    "os.version": "5.10.165-143.735.amzn2.x86_64",
    "user.language": "en",
    "user.timezone": "GMT"
  },
  "metabase-info": {
    "databases": [
      "postgres",
      "redshift",
      "athena"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "13.8"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.5.4"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "?",
      "tag": "vUNKNOWN",
      "branch": "?",
      "hash": "?"
    },
    "settings": {
      "report-timezone": null
    }
  }
}

Here is my understanding of metabase sync behaviour and our issue :

  1. Metabase creates a connection (connection A), and uses this connection to fetch the list of schemas.
  2. For each schema :
    a. Metabase uses connection A to run the following query : SELECT HAS_SCHEMA_PRIVILEGE(?, 'USAGE');
    b. Metabase syncs all tables and fields for that schema, not using connection A, but creating new connections from a pool.

My understanding of our issue is that for some of our schemas, step 2b is very long (we have thousands of tables). When this step finishes and step 2a for the next schema is up, connection A has timed out, hence the RedshiftException: This statement has been closed error.

From our stack trace, is my understanding of our issue correct?

What we have tried so far :

  • We have added tcpKeepAlive=true to Additional JDBC connection string options in the connection.
  • We are currently disabling foreign key syncing to see if that speeds up step 2a enough for connection A to still be alive.