We have long queries (MongoDB aggregation ones) that usually take 12/15 min to load, but recently we had some problems loading them, sometimes the question simply stops. After reading through the logs I’ve had found this:
Exception in thread "async-dispatch-429" java.lang.IllegalStateException: AsyncContext completed and/or Request lifecycle recycled
at org.eclipse.jetty.server.AsyncContextState.state(AsyncContextState.java:54)
at org.eclipse.jetty.server.AsyncContextState.complete(AsyncContextState.java:74)
at ring.util.servlet$make_output_stream$fn__46932.invoke(servlet.clj:89)
at ring.util.servlet.proxy$java.io.FilterOutputStream$ff19274a.close(Unknown Source)
at java.base/sun.nio.cs.StreamEncoder.implClose(Unknown Source)
at java.base/sun.nio.cs.StreamEncoder.close(Unknown Source)
at java.base/java.io.OutputStreamWriter.close(Unknown Source)
at java.base/java.io.BufferedWriter.close(Unknown Source)
at metabase.async.api_response$write_chan_vals_to_writer_BANG_$fn__67375$state_machine__6775__auto____67380$fn__67382.invoke(api_response.clj:87)
at metabase.async.api_response$write_chan_vals_to_writer_BANG_$fn__67375$state_machine__6775__auto____67380.invoke(api_response.clj:79)
at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:973)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:972)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:977)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
at clojure.core.async.impl.ioc_macros$take_BANG_$fn__6793.invoke(ioc_macros.clj:986)
at clojure.core.async.impl.channels.ManyToManyChannel$fn__1679$fn__1680.invoke(channels.clj:95)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
11-25 20:14:27 e[1mDEBUG middleware.loge[0m :: e[32mPOST /api/card/801/query 200 [ASYNC: canceled] 10.0 mins (7 DB calls) Jetty threads: 2/50 (4 idle, 0 queued) (119 total active threads) Queries in flight: 10e[0m
And this happens to every question that take more than 10 min to load. What seems strange to me is if I try to run the aggregations direct on DB (using Robo 3T) the query run normally and sometimes even faster.
We already have the cache enabled and that don’t helped a lot, and our Metabase version is the 0.33.6.
Also, what does that Request lifecycle recycled means?