Hi,
Metabase is a cool thing!
We have a problem that metabase use big number(we have tight resource limitations) of connections to store metabase’s data.
Is there any way to control max number of connections for docker setup?
Hi there!
Metabase by default has a max connection limit of 15.
There’s an open issue for being able to modify this in the Admin settings ( https://github.com/metabase/metabase/issues/2172 ), but for now you can tweak it with java command-line arguments: https://github.com/metabase/metabase/issues/2172#issuecomment-559638836
If you’re running on Docker, you can pass in custom java options like this:
docker run -d -p 3000:3000 -e "JAVA_OPTS=-Dc3p0.maxPoolSize=0" --name metabase metabase/metabase
Damon
Whatever I set the maxPoolSize to, Metabase keeps saying:
mysql DB 2 connections: 4/6 (0 threads blocked)
And I see only 6 cores out of 8 engaged. At a certain setting (I think 8) it started using 7 out of 8 and docker stats showed 700% usage. But I could never get it to 8. Any ideas?
@andreychirikba Use the built-in variable instead:
https://www.metabase.com/docs/latest/operations-guide/environment-variables.html#mb_jdbc_data_warehouse_max_connection_pool_size
It's not "cores", it's connections. I wrote a more detailed guide to the logs:
https://github.com/metabase/metabase/blob/master/docs/troubleshooting-guide/server-logs.md
@flamber, I tried setting the MB_JDBC_DATA_WAREHOUSE_MAX_CONNECTION_POOL_SIZE setting:
root@d22154f20b7d:/# env | grep POOL
MB_JDBC_DATA_WAREHOUSE_MAX_CONNECTION_POOL_SIZE=15
but the log still says:
mysql DB 2 connections: 5/7 (0 threads blocked)
PS. I am aware it's connections, not cores, but I would expect so many database server cores to be engaged for so many connections.
@andreychirikba 15 connections is the default. Metabase does not open 15 connections initially. It only opens connections when needed (up to the max), so that's why you're only seeing a pool of 7.
I'm not sure what you mean by "cores". But if it is on the database side, then you need to check the manual there.
@flamber, I have a dashboard with ~50 questions. MySQL only serves 1 query per 1 core, and the queries take up 100% of each core, so we added 2 cores for it to be able to process more queries simultaneously. I thought I could tweak the setting in Metabase, so it would issue more queries at once. Shouldn't it be doing that for a dashboard with a lot of questions?
@andreychirikba That's a limitation in MySQL:
https://dba.stackexchange.com/questions/5666/possible-to-make-mysql-use-more-than-one-core
Metabase has a connection pool for each database, which is defaulted to max 15 connections, and sends queries on available connections.
But a dashboard is handled by the browser, where you are limited to 6 connections unless you are using HTTP/2, so that's likely your problem. Read this:
https://www.metabase.com/learn/administration/metabase-at-scale
@flamber, indeed, switching to HTTP/2 did the trick, thanks for the info!
Can I set a maximum number of connections for a single database? That is, different databases set different connection numbers, and the current variable(MB_JDBC_DATA_WAREHOUSE_MAX_CONNECTION_POOL_SIZE) seems to set the same maximum connection number for all databases.How do I set this up if it is possible?
It’s for all dbs
Can you tell us the step how to switch to http/2