Metabase super slow and faulty after H2 -> MySQL Migration

timsiegelkow · March 23, 2023, 5:56pm

Heyo,
I migrated from H2 to MySQL (Google CloudSQL) following the documentation (Migrating to a production application database).

Ever since, Metabase has become 50% slower in everything. Additionally, when editing Dashboards, it always fails to save or messes up parts of the Dashboard.

Any idea where this might come from and how to fix it?

Diagnostic Info:
{
"browser-info": {
"language": "de-DE",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36",
"vendor": "Google Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "OpenJDK Runtime Environment",
"java.runtime.version": "11.0.16.1+1",
"java.vendor": "Eclipse Adoptium",
"java.vendor.url": "https://adoptium.net/",
"java.version": "11.0.16.1",
"java.vm.name": "OpenJDK 64-Bit Server VM",
"java.vm.version": "11.0.16.1+1",
"os.name": "Linux",
"os.version": "5.15.0-1027-gcp",
"user.language": "en",
"user.timezone": "GMT"
},
"metabase-info": {
"databases": [
"h2",
"mysql"
],
"hosting-env": "unknown",
"application-database": "mysql",
"application-database-details": {
"database": {
"name": "MySQL",
"version": "8.0.26-google"
},
"jdbc-driver": {
"name": "MariaDB Connector/J",
"version": "2.7.6"
}
},
"run-mode": "prod",
"version": {
"date": "2022-09-29",
"tag": "v0.44.4",
"branch": "release-x.44.x",
"hash": "382d728"
},
"settings": {
"report-timezone": null
}
}
}

Luiggi · March 23, 2023, 6:31pm

Hi, where's the MB server located? where's the MySQL located? what's the sizing of both?

timsiegelkow · March 24, 2023, 9:21am

Hi Luiggi,
the MB server is running on a docker image on a Virtual Machine instance in a Google Cloud Project.
The MySQL is a CloudSQL database instance in the same GCP.
Size of the VM: "e2-standard-2" with 2 CPU, 8GB RAM, 40GB disk
Size of CloudSQL: 2 CPU, 8GB RAM

Luiggi · March 24, 2023, 12:39pm

Thanks, are those 2 in the same network? Even though, it would be great if you could:

upgrade to the latest version
send us the logs of the frontend and backend when you see the errors

timsiegelkow · April 6, 2023, 11:37am

Hi Luiggi,
Yes, they are in the same network.
I upgraded to the latest version - errors still happening.
I don't really know which logs to send. Here are the logs that happened right after a dashboard failed to save again:

[6804fccf-4559-4111-9fff-a09cf969a9b3] 2023-04-06T13:35:47+02:00 ERROR metabase.async.streaming-response Caught unexpected Exception in streaming response body

java.lang.NullPointerException: Deflater has been closed

at java.base/java.util.zip.Deflater.ensureOpen(Unknown Source)

at java.base/java.util.zip.Deflater.deflate(Unknown Source)

at java.base/java.util.zip.GZIPOutputStream.finish(Unknown Source)

at java.base/java.util.zip.DeflaterOutputStream.close(Unknown Source)

at metabase.async.streaming_response$delay_output_stream$fn__36937.invoke(streaming_response.clj:121)

at metabase.async.streaming_response.proxy$java.io.OutputStream$ff19274a.close(Unknown Source)

at metabase.async.streaming_response$write_error_BANG_.invokeStatic(streaming_response.clj:58)

at metabase.async.streaming_response$write_error_BANG_.invoke(streaming_response.clj:46)

at metabase.query_processor.streaming$streaming_response_STAR_$fn__54305.invoke(streaming.clj:174)

at clojure.lang.AFn.applyToHelper(AFn.java:156)

at clojure.lang.AFn.applyTo(AFn.java:144)

at clojure.core$apply.invokeStatic(core.clj:667)

at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)

at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)

at clojure.lang.RestFn.applyTo(RestFn.java:142)

at clojure.core$apply.invokeStatic(core.clj:671)

at clojure.core$bound_fn_STAR_$fn__5818.doInvoke(core.clj:2020)

at clojure.lang.RestFn.invoke(RestFn.java:421)

at metabase.async.streaming_response$do_f_STAR_.invokeStatic(streaming_response.clj:69)

at metabase.async.streaming_response$do_f_STAR_.invoke(streaming_response.clj:67)

at metabase.async.streaming_response$do_f_async$task__36922.invoke(streaming_response.clj:88)

at clojure.lang.AFn.run(AFn.java:22)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

[6804fccf-4559-4111-9fff-a09cf969a9b3] 2023-04-06T13:35:47+02:00 ERROR metabase.async.streaming-response Error writing error to output stream {:via [{:type java.lang.NullPointerException, :message Deflater has been closed, :at [java.util.zip.Deflater ensureOpen nil -1]}], :trace [[java.util.zip.Deflater ensureOpen nil -1] [java.util.zip.Deflater deflate nil -1] [java.util.zip.Deflater deflate nil -1] [java.util.zip.GZIPOutputStream finish nil -1] [java.util.zip.DeflaterOutputStream close nil -1] [metabase.async.streaming_response$delay_output_stream$fn__36937 invoke streaming_response.clj 121] [metabase.async.streaming_response.proxy$java.io.OutputStream$ff19274a close nil -1] [metabase.async.streaming_response$write_error_BANG_ invokeStatic streaming_response.clj 58] [metabase.async.streaming_response$write_error_BANG_ invoke streaming_response.clj 46] [metabase.query_processor.streaming$streaming_response_STAR_$fn__54305 invoke streaming.clj 174] [clojure.lang.AFn applyToHelper AFn.java 156] [clojure.lang.AFn applyTo AFn.java 144] [clojure.core$apply invokeStatic core.clj 667] [clojure.core$with_bindings_STAR_ invokeStatic core.clj 1990] [clojure.core$with_bindings_STAR_ doInvoke core.clj 1990] [clojure.lang.RestFn applyTo RestFn.java 142] [clojure.core$apply invokeStatic core.clj 671] [clojure.core$bound_fn_STAR_$fn__5818 doInvoke core.clj 2020] [clojure.lang.RestFn invoke RestFn.java 421] [metabase.async.streaming_response$do_f_STAR_ invokeStatic streaming_response.clj 69] [metabase.async.streaming_response$do_f_STAR_ invoke streaming_response.clj 67] [metabase.async.streaming_response$do_f_async$task__36922 invoke streaming_response.clj 88] [clojure.lang.AFn run AFn.java 22] [java.util.concurrent.Executors$RunnableAdapter call nil -1] [java.util.concurrent.FutureTask run nil -1] [java.util.concurrent.ThreadPoolExecutor runWorker nil -1] [java.util.concurrent.ThreadPoolExecutor$Worker run nil -1] [java.lang.Thread run nil -1]], :cause Deflater has been closed, :_status 500}

java.lang.NullPointerException: Deflater has been closed

at java.base/java.util.zip.Deflater.ensureOpen(Unknown Source)

at java.base/java.util.zip.Deflater.deflate(Unknown Source)

at java.base/java.util.zip.GZIPOutputStream.finish(Unknown Source)

at java.base/java.util.zip.DeflaterOutputStream.close(Unknown Source)

at metabase.async.streaming_response$delay_output_stream$fn__36937.invoke(streaming_response.clj:121)

at metabase.async.streaming_response.proxy$java.io.OutputStream$ff19274a.close(Unknown Source)

at java.base/sun.nio.cs.StreamEncoder.implClose(Unknown Source)

at java.base/sun.nio.cs.StreamEncoder.close(Unknown Source)

at java.base/java.io.OutputStreamWriter.close(Unknown Source)

at java.base/java.io.BufferedWriter.close(Unknown Source)

at metabase.async.streaming_response$write_error_BANG_.invokeStatic(streaming_response.clj:61)