Slow API calls resulting in very poor user experience

Hi there,

We love metabase. But sometimes the UI is mysteriously and frustratingly slow. As far as we know/see there isn’t obvious resourcing problems (our DB is cloud hosted and well resourced), and our instance hosting metabase doesn’t appear to CPU bound in any way.

So how do we figure out why it’s so slow!?

We’re currently using 0.17.2, using the official docker image.

Here’s a sample of APIs that see this slowness (by watching the Chrome inspector’s network view)

/api/dashboard?f=all - consistently ~40 secs
/api/activity - consistently ~30 secs
/api/database?include_tables=true&include_cards=true - usually ~1-2secs, sometimes 40secs.

These make the UI horribly unresponsive. The dashboard api seems to be called quite regularly.

I’m happy to get my hands dirty and help diagnose this … it’s gonna make life MUCH better for our users and hopefully make some significant improvement to metabase product.

Thanks,
Matt

I presume you’re running 0.27.2 (otherwise I think you should upgrade)

For comparison I just ran /api/dashboard?f=all and /api/activity and they were both well below 1 sec. That’s with the H2 default Metabase internal db and less than 10 dashboards (what db do you use? - and how many dashboards do you have?)

The core devs are currently looking perf of another api:

oops, yeah I’m running 0.27.2!! :slight_smile:

We’re running MySQL 5.7 on Google Cloud SQL, plenty of resources.

24 dashboards. It just doesn’t make sense. That’s why I’m looking for some assistance in pinning it down.

The activity table has 1500 entries, and it takes 30 seconds to load the list. Something is very fishy.

We have 573 questions.

This one probably more related to dashboards:

It has mention of a debug line in the log with db calls. Maybe useful for you?

Other than that in your shoes I think I would try to spin up a local Metabase + MySQL clone to try to isolate if it’s an environment issue and it would also enable you to test with a 0.28.0-RC version.

I’m doing a bit of that. If I pull the DB locally and run metabase and DB locally I don’t see the delays.

I also can’t see any long-running queries running on our Google hosted MySQL whilst these long delays are occurring. There is something fundamentally wrong here, but need to isolate it.

How do you get debug logging enabled in metabase using the docker container? I only see INFO and WARN messages. It might help me understand where delays coming from

I’ve got it running in dev environment, debug on high. There are some pretty horrible things going on here. It’s time to raise a ticket I think.

Here’s a /api/database log that took 3 minutes!! and made 915 database calls. Yowsers! (time is exacerbated as I’m running DB in cloud (north america) and I’m in Australia, so latency is high).

02-07 17:27:17 DEBUG metabase.middleware :: GET /api/database 200 (3 mins) (915 DB calls) [qtp1083411476-29]

and this one for /api/dashboard

02-07 17:31:06 DEBUG metabase.middleware :: GET /api/dashboard 200 (49 s) (261 DB calls) [qtp1083411476-28]

1 Like

Raised issue: