Dashboard load slowly

Get Help
1

I'm experiencing a performance issue with dashboards loading, while my database and backend data warehouse queries executing quickly. Hopefully someone could give me some troubleshooting advice.

Information:

  1. One Dashboards with 3 tabs, each tab contains about 15 cards.
  2. Loading a tab takes 3~6 seconds.
  3. backend db: Amazon RDS (db.t4g.medium) MySQL 8.0
  4. backend data warehouse: StarRocks v3.4.0
  5. Metabase version：v0.47.11

Diagnostic Info:

{
  "browser-info": {
    "language": "zh-CN",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.21+9",
    "java.vendor": "Eclipse Adoptium",
    "java.vendor.url": "https://adoptium.net/",
    "java.version": "11.0.21",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.21+9",
    "os.name": "Linux",
    "os.version": "5.10.234-225.895.amzn2.x86_64",
    "user.language": "en",
    "user.timezone": "GMT"
  },
  "metabase-info": {
    "databases": [
      "mysql",
      "redshift"
    ],
    "hosting-env": "unknown",
    "application-database": "mysql",
    "application-database-details": {
      "database": {
        "name": "MySQL",
        "version": "8.0.28"
      },
      "jdbc-driver": {
        "name": "MariaDB Connector/J",
        "version": "2.7.6"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2024-01-05",
      "tag": "v0.47.11",
      "branch": "?",
      "hash": "51935b1"
    },
    "settings": {
      "report-timezone": "UTC"
    }
  }
}

What I have checked:

  1. From Metabase log, I'm sure that the time is spent inside the metabase, not the browser loading time.
    such as:
2025-04-11 02:31:15,205 DEBUG middleware.log :: <mark>POST /api/dashboard/685</mark>/dashcard/9522/card/5498/query 202 [ASYNC: completed] 3.4 s (38 DB calls) App DB connections: 5/50 Jetty threads: 3/50 (38 idle, 0 queued) (217 total active threads) Queries in flight: 11 (0 queued); mysql DB 24 connections: 3/13 (0 threads blocked)
  1. From prometheus metrics, I have checked the cpu/memory usage of Metabase and RDS, they're all lower. Resources aren't tight.
  2. From the metrics provided by AWS, All RDS operations are completed in milliseconds.
  3. I have enabled the slow query logs about RDS, and the threshold is 500ms, there is nothing output, it means that the bottleneck is not in RDS.
  4. The StarRocks query history list shows that all the queries completed within 300ms, and I have executed queries in StarRocks from other clients, it takes about the same time, meaning the bottleneck isn't in StarRocks either.
  5. when I reduce the number of cards in one tab(15->3), it load fastly(within 1s), it seems to have something to do with concurrency, but I've looked through some metrics and logs, and the thread number, and database connections aren't at the top line, so I'm not sure what the problem is.

Since the metabase logs don't include the time spent during a card load process, I have enabled the audit log of RDS and tried to keep track of some key time points, and one of the cards involves the following database accesses like this:

2025-04-11 10:31:11.774643,1744338671774643,<db_name>,<username>,10.11.16.170,41,9988,QUERY,metabase_0_47_11,'SELECT EXISTS (SELECT 1 FROM `report_dashboardcard` WHERE (`id` = 9522) AND (`dashboard_id` = 685) AND (`card_id` = 5498)) AS `exists`',0
2025-04-11 10:31:11.911722,1744338671911722,<db_name>,<username>,10.11.16.170,48,10017,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.914783,1744338671914783,<db_name>,<username>,10.11.16.170,35,10018,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.918572,1744338671918572,<db_name>,<username>,10.11.16.170,26,10020,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.918601,1744338671918601,<db_name>,<username>,10.11.16.170,44,10016,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.922524,1744338671922524,<db_name>,<username>,10.11.16.170,12,10019,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.922974,1744338671922974,<db_name>,<username>,10.11.16.170,46,10021,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.925075,1744338671925075,<db_name>,<username>,10.11.16.170,54,10023,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.925473,1744338671925473,<db_name>,<username>,10.11.16.170,67,10022,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:11.999771,1744338671999771,<db_name>,<username>,10.11.16.170,41,10024,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:12.112589,1744338672112589,<db_name>,<username>,10.11.16.170,9,10028,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:12.114617,1744338672114617,<db_name>,<username>,10.11.16.170,69,10029,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:12.116907,1744338672116907,<db_name>,<username>,10.11.16.170,60,10030,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:12.134424,1744338672134424,<db_name>,<username>,10.11.16.170,55,10031,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0
2025-04-11 10:31:12.400840,1744338672400840,<db_name>,<username>,10.11.16.170,57,10038,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` IN (5480, 5476, 5459, 5500, 5503, 5475, 5464, 5457, 5498, 5478, 5501, 5469, 5490, 5486, 5460, 5456, 5493, 5502, 5494, 5477, 5471, 5497, 5488, 5484, 5489, 5472, 5495, 5499, 5458, 5491, 5468, 5470, 5487, 5481, 5492, 5455, 5461, 5467, 5473, 5496, 5482, 5485, 5463, 5483, 5474, 5466, 5465, 5479, 5462)',0





2025-04-11 10:31:14.201507,1744338674201507,<db_name>,<username>,10.11.16.170,48,10083,QUERY,metabase_0_47_11,'SELECT * FROM `report_dashboardcard` WHERE `id` = 9522',0
2025-04-11 10:31:14.208823,1744338674208823,<db_name>,<username>,10.11.16.170,12,10087,QUERY,metabase_0_47_11,'SELECT `report_card`.`id`, `report_card`.`name`, `report_card`.`dataset_query`, `report_card`.`database_id`, `report_card`.`cache_ttl`, `report_card`.`collection_id`, `report_card`.`dataset`, `report_card`.`result_metadata`, `report_card`.`visualization_settings` FROM `report_card` WHERE `id` = 5498',0
2025-04-11 10:31:14.829682,1744338674829682,<db_name>,<username>,10.11.16.170,12,10192,QUERY,metabase_0_47_11,'SELECT `report_card`.`collection_id` FROM `report_card` WHERE `id` = 5498',0
2025-04-11 10:31:15.054712,1744338675054712,<db_name>,<username>,10.11.16.170,55,10258,QUERY,metabase_0_47_11,'SELECT * FROM `report_card` WHERE `id` = 5498',0
2025-04-11 10:31:15.106742,1744338675106742,<db_name>,<username>,10.11.16.170,55,10276,QUERY,metabase_0_47_11,'SELECT * FROM `parameter_card` WHERE `card_id` = 5498',0
2025-04-11 10:31:15.114267,1744338675114267,<db_name>,<username>,10.11.16.170,55,10296,QUERY,metabase_0_47_11,'DELETE FROM `parameter_card` WHERE (`parameterized_object_type` = \'card\') AND (`parameterized_object_id` = 5498)',0
2025-04-11 10:31:15.124735,1744338675124735,<db_name>,<username>,10.11.16.170,55,10322,QUERY,metabase_0_47_11,'UPDATE `report_card` SET `id` = 5498, `result_metadata` = \'[{\\"display_name\\":\\"date\\",\\"field_ref\\":[\\"field\\",\\"date\\",{\\"base-type\\":\\"type/Date\\"}],\\"name\\":\\"date\\",\\"base_type\\":\\"type/Date\\",\\"effective_type\\":\\"type/Date\\",\\"semantic_type\\":null,\\"fingerprint\\":{\\"global\\":{\\"distinct-count\\":29,\\"nil%\\":0.0},\\"type\\":{\\"type/DateTime\\":{\\"earliest\\":\\"2025-03-12T00:00:00Z\\",\\"latest\\":\\"2025-04-09T00:00:00Z\\"}}}},{\\"display_name\\":\\"account_name\\",\\"field_ref\\":[\\"field\\",\\"account_name\\",{\\"base-type\\":\\"type/Text\\"}],\\"name\\":\\"account_name\\",\\"base_type\\":\\"type/Text\\",\\"effective_type\\":\\"type/Text\\",\\"semantic_type\\":null,\\"fingerprint\\":{\\"global\\":{\\"distinct-count\\":1,\\"nil%\\":0.0},\\"type\\":{\\"type/Text\\":{\\"percent-json\\":0.0,\\"percent-url\\":0.0,\\"percent-email\\":0.0,\\"percent-state\\":0.0,\\"average-length\\":23.0}}}},{\\"display_name\\":\\"on_demand_cost_usd\\",\\"field_ref\\":[\\"field\\",\\"on_demand_cost_usd\\",{\\"base-type\\":\\"type/Decimal\\"}],\\"name\\":\\"on_demand_cost_usd\\",\\"base_type\\":\\"type/Decimal\\",\\"effective_type\\":\\"type/Decimal\\",\\"semantic_type\\":\\"type/Cost\\",\\"fingerprint\\":{\\"global\\":{\\"distinct-count\\":29,\\"nil%\\":0.0},\\"type\\":{\\"type/Number\\":{\\"min\\":1827.0183032544,\\"q1\\":2164.754354249075,\\"q3\\":2197.988125636975,\\"max\\":2272.8175906986,\\"sd\\":75.39588416436382,\\"avg\\":2173.78821881071}}}}]\', `updated_at` = NOW(6) WHERE `id` = 5498',0
2025-04-11 10:31:15.207772,1744338675207772,<db_name>,<username>,10.11.16.170,41,10358,QUERY,metabase_0_47_11,'INSERT INTO `view_log` (`timestamp`, `user_id`, `model`, `model_id`, `metadata`) VALUES (NOW(), 144, \'card\', 5498, \'{\\"cached\\":null,\\"ignore_cache\\":false,\\"context\\":\\"dashboard\\"}\')',0
2025-04-11 10:31:15.302670,1744338675302670,<db_name>,<username>,10.11.16.170,54,10415,QUERY,metabase_0_47_11,'INSERT INTO `query_execution` (`hash`, `database_id`, `result_rows`, `started_at`, `executor_id`, `native`, `pulse_id`, `card_id`, `context`, `cache_hit`, `running_time`, `dashboard_id`) VALUES (_binary \'8����QJ���T��.pb�܋y/�{l��V\', 24, 29, \'2025-04-11 02:31:14.403194\', 144, TRUE, NULL, 5498, \'dashboard\', FALSE, 800, 685)',0

It seems to have taken 2s between SELECT * FROM report_cardWHEREid IN xxx and SELECT * FROM report_dashboardcardWHEREid = 9522', I am unsure whether this is related to the issue, and I would like to understand which operations between the two SQL queries were more time-consuming.
Any insights or recommendations regarding this matter would be greatly appreciated!

2

v47 is so old that we can't help here. Please upgrade and report back