Optimizing a Dashboard - Query loads 30s individually, takes 6 minutes in dashboard

Hello, I’m looking for some help on speeding up a dashboard load time. I’ve reviewed the help article and don’t see how the listed steps help too much (already working with transformed data, can’t cache this data as frequently applying new filters)

These queries take about 30s to run when run individually:

  • Provider Bookings Change YOY
  • Edit: Cannot link the other query, as I’m restricted to only two links, but it’s in the dashboard below and the same exact SQL as the above query, just graph vs table view. Code block below…
-- Provider Bookings Change YOY
with date_spine as (

 select 
   month_of_year_int as month, 
   year_int as year 
 from utils.dates where year_int in (2020, 2019)

)

select
   date_spine.month,
   coalesce(count (distinct case when 
       extract(year from bookings.created_at_et) = '2019' and is_online is true
       then bookings.booking_id else null end), 0) as total_online_bookings_2019,
   coalesce(count (distinct case when 
       extract(year from bookings.created_at_et) = '2020' and is_online is true
       then bookings.booking_id else null end), 0) as total_online_bookings_2020,
   coalesce(count (distinct case when 
       extract(year from bookings.created_at_et) = '2019' and is_online is false
       then bookings.booking_id else null end), 0) as total_in_person_bookings_2019,
   coalesce(count (distinct case when 
       extract(year from bookings.created_at_et) = '2020' and is_online is false
       then bookings.booking_id else null end), 0) as total_in_person_bookings_2020
from date_spine
left join analytics.bookings on date_spine.year = extract(year from bookings.created_at_et)
   and date_spine.month = extract(month from bookings.created_at_et)
   and bookings.created_at_et > '2019-01-01'
left join analytics.order_items on bookings.booking_id = order_items.booking_id
left join analytics.providers on bookings.provider_id = providers.provider_id
   and providers.tools_live_date < (current_timestamp -  interval '1 year')
   and providers.provider_status = 'live'
   and providers.provider_id = {{provider_id}}
group by 1
order by 1 asc
;

When I add these queries to a dashboard, they take 5-6 minutes to run. See the dashboard here.

  • Looking at the (/admin/troubleshooting/logs) (cmd+f “dashboard/185”) after running, Metabase consistently fails to load the two queries on the righthand side of the dashboard. Redshift seems to be handling the queries normally, though. As noted, the data loads eventually on Metabase after about 5-6 mins of waiting.

Is there something I’m missing here on how to optimize this query?

**Using Chrome, and Metabase v0.37.4.

Please let me know if I can provide any additional information, this is my first time posting here :slight_smile:

Hi @viborja
I’m guessing that you have already read these:
https://www.metabase.com/learn/data-diet/analytics/making-dashboards-faster.html
https://www.metabase.com/learn/data-diet/analytics/metabase-at-scale.html
https://www.metabase.com/learn/building-analytics/sql-templates/sql-best-practices.html

Without seeing the logs from when you view the dashboard, then it’s difficult to fully understand where the slow-down might occur.

Hello @flamber!

Yes, I took a look at those. I thought the first one would help but I’m hoping to be able to accomplish this without having to make a materialized view or something (the last thing I can think to do). I pasted below some of the logs I’ve gotten the past couple times I’ve pulled up the dashboard yesterday. What do you think?

Did not finish loading:

2021-01-04T16:46:18-05:00 DEBUG metabase.middleware.log GET /api/dashboard/185 200 400.4 ms (12 DB calls) App DB connections: 1/15 Jetty threads: 5/50 (1 idle, 0 queued) (137 total active threads) Queries in flight: 0 (0 queued)

2021-01-04T16:52:57-05:00 DEBUG metabase.middleware.log GET /api/dashboard/185 200 197.8 ms (12 DB calls) App DB connections: 1/10 Jetty threads: 5/50 (4 idle, 0 queued) (132 total active threads) Queries in flight: 0 (0 queued)

2021-01-04T16:53:44-05:00 DEBUG metabase.middleware.log GET /api/dashboard/185 200 128.6 ms (12 DB calls) App DB connections: 1/15 Jetty threads: 5/50 (1 idle, 0 queued) (140 total active threads) Queries in flight: 1 (0 queued)

Here are some of the logs I’m getting as I’m trying to run it now and it’s timing out. But again, this loads in 30s-1 min as long as I’m not in a dashboard view, or when I’m doing nothing on the computer and it runs in a few minutes. It keeps re-doing this over and over again:

2021-01-05T09:51:17-05:00 ERROR metabase.query-processor.middleware.catch-exceptions Error processing query: null
{:database_id XX,
 :started_at #t "2021-01-05T14:49:54.140Z[Etc/UTC]",
 :json_query
 {:constraints {:max-results 10000, :max-results-bare-rows 2000},
  :type :native,
  :middleware {:js-int-to-string? true},
  :native
  {:query
   ...[truncated]...
 :context :question,
 :error nil,
 :row_count 0,
 :running_time 0,
 :data {:rows [], :cols []}}
2021-01-05T09:51:18-05:00 DEBUG metabase.middleware.log POST /api/card/2003/query 202 

[ASYNC: canceled] 31.1 s (5 DB calls) App DB connections: 0/15 Jetty threads: 4/50 (2 idle, 0 queued) (136 total active threads) Queries in flight: 1 (0 queued)

(after this it repeats)

I also got this warning a bunch of times. Maybe I need to adjust how I’m using the filters?

[d041eda3-992c-4c58-8611-73f0a521d078] 2021-01-05T10:06:34-05:00 WARN metabase.driver.sql-jdbc.sync.describe-table Don't know how to map column type 'string' to a Field base_type, falling back to :type/*.

@viborja You need to provide the full logs - too much is being snipped, but I’m fairly sure that you are hitting this issue:
https://github.com/metabase/metabase/issues/12423 - upvote by clicking :+1: on the first post
Which has no-workaround on Heroku, because of their known timeouts:
https://github.com/metabase/metabase/issues/11463

1 Like

Hi @flamber I just finished going over the issue with my team and we believe we are hitting that issue as well. When we searched for the error code in one of the issues you linked to, it seems like that’s firing for these queries. I’ve upvoted the thread. Thanks for your help in identifying the issue!

1 Like