Metabase on Elastic Beanstalk, Can't Connect to Snowflake

Get Help
1

We followed the procedure inthis documentation to set up our metabase application on an Elastic Beanstalk instance.
We aren't able to establish a connection to a snowflake database. We receive a timeout notice after 10 seconds. Here is the output from Metabase Admin > Troubleshooting > Logs:

[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:56-07:00 INFO metabase.driver.impl Initializing driver :snowflake...
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:56-07:00 INFO metabase.plugins.classloader Added URL file:/plugins/snowflake.metabase-driver.jar to classpath
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:56-07:00 DEBUG metabase.plugins.init-steps Loading plugin namespace metabase.driver.snowflake...
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:57-07:00 INFO metabase.driver.impl Registered abstract driver :metabase.driver.sql-jdbc.execute.legacy-impl/use-legacy-classes-for-read-and-set :truck:
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:57-07:00 INFO metabase.driver.impl Registered driver :snowflake (parents: [:sql-jdbc :metabase.driver.sql-jdbc.execute.legacy-impl/use-legacy-classes-for-read-and-set]) :truck:
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:57-07:00 DEBUG metabase.plugins.jdbc-proxy Registering JDBC proxy driver for net.snowflake.client.jdbc.SnowflakeDriver...
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:57-07:00 INFO metabase.util Load lazy loading driver :snowflake took 229.2 ms
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:55:57-07:00 INFO metabase.driver.impl Initializing driver :metabase.driver.sql-jdbc.execute.legacy-impl/use-legacy-classes-for-read-and-set...
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:56:06-07:00 ERROR metabase.driver.snowflake (type message)
=> java.lang.String
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:56:06-07:00 ERROR metabase.api.database Cannot connect to Database
java.lang.Exception: Timed out after 10.0 s
at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:55)
at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:39)
at clojure.lang.RestFn.invoke(RestFn.java:442)
at metabase.api.database$test_database_connection.invokeStatic(database.clj:514)
at metabase.api.database$test_database_connection.doInvoke(database.clj:503)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at metabase.api.database$fn__74755$test_connection_details__74760$fn__74761.invoke(database.clj:563)
at metabase.api.database$fn__74755$test_connection_details__74760.invoke(database.clj:544)
at metabase.api.database$fn__74787.invokeStatic(database.clj:580)
at metabase.api.database$fn__74787.invoke(database.clj:566)
at compojure.core$wrap_response$fn__28036.invoke(core.clj:160)
at compojure.core$wrap_route_middleware$fn__28020.invoke(core.clj:132)
at compojure.core$wrap_route_info$fn__28025.invoke(core.clj:139)
at compojure.core$wrap_route_matches$fn__28029.invoke(core.clj:151)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048.invoke(core.clj:200)
at metabase.server.middleware.auth$enforce_authentication$fn__66749.invoke(auth.clj:14)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048.invoke(core.clj:200)
at compojure.core$make_context$handler__28076.invoke(core.clj:289)
at compojure.core$make_context$fn__28080.invoke(core.clj:299)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$make_context$fn__28080.invoke(core.clj:300)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at metabase.api.routes$fn__78112$fn__78115.invoke(routes.clj:59)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048.invoke(core.clj:200)
at clojure.lang.AFn.applyToHelper(AFn.java:160)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
at metabase.server.routes$fn__78262$fn__78263.doInvoke(routes.clj:67)
at clojure.lang.RestFn.invoke(RestFn.java:436)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048.invoke(core.clj:200)
at compojure.core$make_context$handler__28076.invoke(core.clj:289)
at compojure.core$make_context$fn__28080.invoke(core.clj:299)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$wrap_route_matches$fn__28029.invoke(core.clj:153)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$wrap_route_matches$fn__28029.invoke(core.clj:153)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at compojure.core$wrap_route_matches$fn__28029.invoke(core.clj:153)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048$f__28049$respond_SINGLEQUOTE___28050.invoke(core.clj:197)
at metabase.server.routes$fn__78248$fn__78250.invoke(routes.clj:45)
at compojure.core$routes$fn__28048$f__28049.invoke(core.clj:198)
at compojure.core$routes$fn__28048.invoke(core.clj:200)
at metabase.server.middleware.exceptions$catch_uncaught_exceptions$fn__75074.invoke(exceptions.clj:98)
at metabase.server.middleware.exceptions$catch_api_exceptions$fn__75071.invoke(exceptions.clj:86)
at metabase.server.middleware.log$log_api_call$fn__78732$fn__78733$fn__78734.invoke(log.clj:211)
at metabase.driver.sql_jdbc.execute.diagnostic$do_with_diagnostic_info.invokeStatic(diagnostic.clj:15)
at metabase.driver.sql_jdbc.execute.diagnostic$do_with_diagnostic_info.invoke(diagnostic.clj:9)
at metabase.server.middleware.log$log_api_call$fn__78732$fn__78733.invoke(log.clj:203)
at toucan.db$_do_with_call_counting.invokeStatic(db.clj:216)
at toucan.db$do_with_call_counting.invoke(db.clj:209)
at metabase.server.middleware.log$log_api_call$fn__78732.invoke(log.clj:202)
at metabase.server.middleware.browser_cookie$ensure_browser_id_cookie$fn__81968.invoke(browser_cookie.clj:38)
at metabase.server.middleware.security$add_security_headers$fn__60352.invoke(security.clj:148)
at metabase.server.middleware.json$wrap_json_body$fn__81107.invoke(json.clj:63)
at metabase.server.middleware.json$wrap_streamed_json_response$fn__81125.invoke(json.clj:99)
at metabase.server.middleware.offset_paging$handle_paging$fn__60376.invoke(offset_paging.clj:42)
at ring.middleware.keyword_params$wrap_keyword_params$fn__82235.invoke(keyword_params.clj:55)
at ring.middleware.params$wrap_params$fn__82254.invoke(params.clj:77)
at metabase.server.middleware.misc$maybe_set_site_url$fn__35182.invoke(misc.clj:59)
at metabase.server.middleware.session$bind_current_user$fn__46653$fn__46654.invoke(session.clj:291)
at metabase.server.middleware.session$do_with_current_user.invokeStatic(session.clj:270)
at metabase.server.middleware.session$do_with_current_user.invoke(session.clj:259)
at metabase.server.middleware.session$bind_current_user$fn__46653.invoke(session.clj:290)
at metabase.server.middleware.session$wrap_current_user_info$fn__46635.invoke(session.clj:240)
at metabase.server.middleware.session$wrap_session_id$fn__46619.invoke(session.clj:173)
at metabase.server.middleware.auth$wrap_api_key$fn__66757.invoke(auth.clj:27)
at ring.middleware.cookies$wrap_cookies$fn__82155.invoke(cookies.clj:216)
at metabase.server.middleware.misc$add_content_type$fn__35165.invoke(misc.clj:27)
at metabase.server.middleware.misc$disable_streaming_buffering$fn__35190.invoke(misc.clj:76)
at ring.middleware.gzip$wrap_gzip$fn__82197.invoke(gzip.clj:86)
at metabase.server.middleware.misc$bind_request$fn__35193.invoke(misc.clj:93)
at metabase.server.middleware.ssl$redirect_to_https_middleware$fn__81984.invoke(ssl.clj:38)
at metabase.server$async_proxy_handler$fn__78504.invoke(server.clj:73)
at metabase.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.TimeoutException: Timed out after 10.0 s
at metabase.util$deref_with_timeout.invokeStatic(util.clj:344)
at metabase.util$deref_with_timeout.invoke(util.clj:336)
at metabase.util$do_with_timeout.invokeStatic(util.clj:350)
at metabase.util$do_with_timeout.invoke(util.clj:347)
at metabase.driver.util$can_connect_with_details_QMARK.invokeStatic(util.clj:50)
... 119 more
[7f81c2d4-d0c1-4fdd-95f2-4a74bb0518b5] 2022-06-06T13:56:07-07:00 DEBUG metabase.server.middleware.log POST /api/database 400 10.3 s (1 DB calls)
{:valid false, :dbname "Timed out after 10.0 s", :message "Timed out after 10.0 s"}

Doing some research online, some recommend updating java but I'm not 100% sure how to do that on Elastic Beanstalk. I copied the Docker image to my own repository but I don't know how to update java from there or if that is even the right approach.

Any help would be greatly appreciated! Thanks!

2

Hi @CassieClark, this is 99% of the times a networking issue: so either you're not allowing your Metabase instance to connect to Snowflake, or you're not letting the connections out from AWS

3

Hi @dragonsahead. Thanks for your reply. That would make sense. Do you know how I would allow a connection between the Elastic Beanstalk instance and Snowflake? I'm thinking I would need to adjust the inbound rules on the security group for the instance. But I'm not sure what inbound rule will connect me to snowflake? Can you elaborate?
Thank you!

4

Hi again, @dragonsahead. Any thoughts on my questions above?

5

@CassieClark It's the other way around. Metabase (outbound) is connecting to Snowflake (inbound).
Make sure you check your Snowflake firewall, and make sure you're using the correct account identifier:
https://docs.snowflake.com/en/user-guide/admin-account-identifier.html#non-vps-account-locator-formats-by-cloud-platform-and-region

6

@CassieClark, in Snowflake you need to go to Admin->security-> click on the "+ Network Policy" icon

image
image880×594 27.2 KB

and then add a policy that allows inbound traffic from the public IP address of your Elastic Beanstalk instance

image
image548×543 19.9 KB

This greatly depends on how you configured Elastic Beanstalk (is it one instance? is it multi-instance? are these instances in a public subnet with internet access? are they on a private instance and you have a NAT gateway?), but to keep things short: try identifying the IP address of the instace/s and adding that as a CIDR range in your Snowflake config like:
x.x.x.x/32

remember that the IP addresses "might" change, unless you have Elastic IP's

7

@flamber that was it! Thank you so much! All I needed to do was change from xy12345.us-east-1.aws to xy12345.us-east-1

A link to that snowflake page could be beneficial to add to Metabase documentation.

Thank you for your help!

8

@CassieClark It is in the documentation:
https://www.metabase.com/docs/latest/administration-guide/databases/snowflake.html#snowflake-gotchas

9

@flamber. Wow I don't know how I missed that. My apologies. Thanks again!