Hi Metabase team,
We are using Metabase with a PostgreSQL database that contains well over 1000 tables.
We’ve noticed the following behavior:
After syncing the database schema, some tables appear and others disappear from both the Data Model view and SQL editor autocomplete.
The Metabase sidebar consistently shows exactly 1000 tables, even though the database contains many more.
When we force a resync, different tables show up — but others vanish, even though they still exist in the PostgreSQL database.
Some "phantom tables" appear in the search or browse UI, but querying them returns errors or empty results because they don’t exist in the database.
Using direct SQL queries works only if we know the exact table name, but autocomplete/search is unreliable due to this issue.
We understand this might be linked to the hardcoded 1000-table metadata scan limit, as referenced in some GitHub issues (e.g., #12383).
Is there a recommended way to:
Increase or remove the table scan limit?
Prioritize scanning specific schemas or table patterns?
Avoid phantom tables appearing in the Data Model?
This issue makes it hard to explore or reliably query large schemas.
We’d appreciate any guidance on best practices or workarounds for handling large PostgreSQL databases in Metabase.
Absent any in-Metabase hacks, you could get around that limit by using multiple database connections and different usernames with each user having visibility into a limited set of schemas. This will work if you don't need to do any cross-schema joins or the like in the database (I think Models can do cross-database stuff, but that pushes more compute work into Metabase).
Watch out for any database connection limits as having separate users will balloon the number of database connections.
If the scan is failing due to timeouts it is possible to adjust the timeouts. This also changes timeouts for questions so there are side effects.
GitHub Issue #12383 doesn't have anything to do with scan limits (it discusses bot deprecation in Slack), but you can search "database scan limit" in GH to find a few related tickets.
While the hard limits can be upped, it's a bit of an arms race as there will always be large enough databases to explode any default JVM config.
+1 to what Doug said. The issue should be visible in debug logs during the sync (if you're on 55 you can enable debug logs from Admin > Troubleshooting > Logs > Customize log levels > Load preset > Sync issue troubleshooting to get a full detail of what it's doing).
There's a possibility that it's finishing the sync early due to a query timeout before running through all the tables in the list. You can try increasing DB_QUERY_TIMEOUT_MINUTES from the default 20 minutes and see if that changes anything. When you have a lot of tables, the "get all tables" query is kept open for a long time while Metabase loops through the results, and this could be reaching a timeout.