Metabase failing to fetch large dataset

Get Help
1

Hello Team,

for version: 31.2

We are having a big bottleneck issue in fetching large dataset say 2GB. The error I am getting is “query took too long”. And the Metabase UI is frozen.

And in background of metabase container stat, i could see the CPU usage reaching more than 200 and then flatten to 0.

I tried couple of work around like passing the parameter
-e “JAVA_TOOL_OPTIONS=-Xmx16g”, but still facing the same issue.

The container logs is below:
Mar 28, 2019 3:46:22 PM net.snowflake.client.jdbc.internal.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (net.snowflake.client.jdbc.internal.apache.http.NoHttpResponseException) caught when processing request to {s}->https://sfc-va-ds1-customer-stage.s3.amazonaws.com:443: The target server failed to respond
Mar 28, 2019 3:46:22 PM net.snowflake.client.jdbc.internal.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://sfc-va-ds1-customer-stage.s3.amazonaws.com:443

i have a feeling it s out of memory issue, however still can not find a workaround.

2

This Issue has been solved.
The problem was Docker not Metabase. I had to increase the docker ram to 4GB, then ran the same command .

And voila it worked( little slow though but yes it worked)

3

How did you increase the RAM? We are having a similar issue - people trying to download large datasets frequently get errors.

4

I increased the Docker memory for docker UI. I have researched but could not find any command to increase the docker memory from command prompt.

However from docker UI you can go to preference --> advance setting and change the memory size.

Hope this help

5

And could you please state the exact issue you are facing.
Like if the query taking to long is getting displayed on Metabase UI or the container is getting Killed all together.

6

Since we are running Metabase on Docker on a server, we don’t have a UI.

7

The query takes too long and eventually returns an error in the UI. Luckily the Metabase container doesn’t crash.

8

@kgl_m
Which version on Metabase? If you’re using 0.32.4, then you might be see this issue:
https://github.com/metabase/metabase/issues/9693
Have a look in your Metabase logs. They will give a lot more details.

9

Please share the metabase log.
go to shell and write docker logs <container_name>

10

If you have a report that’s pulling a 2Gb data set, it should be simplified or the underlying data should be aggregated.