Upgrading from to 0.46

littlegoblin · March 28, 2023, 1:22pm

Tried to flip to latest release, getting following error, reverted back to 0.45.3, started w/o a problem.

2023-03-28 09:21:14,228 ERROR db.update-h2 :: Failed to update H2 database: #error {
:cause the trustAnchors parameter must be non-empty
:via
[{:type java.security.InvalidAlgorithmParameterException
:message the trustAnchors parameter must be non-empty
:at [java.security.cert.PKIXParameters setTrustAnchors PKIXParameters.java 200]}]
:trace
[[java.security.cert.PKIXParameters setTrustAnchors PKIXParameters.java 200]
[java.security.cert.PKIXParameters PKIXParameters.java 120]
[java.security.cert.PKIXBuilderParameters PKIXBuilderParameters.java 104]

TonyC · March 28, 2023, 1:49pm

You should migrate away from H2:

Can you share the whole logs? How are you running metabase?

littlegoblin · March 28, 2023, 3:19pm

Its not production, its a simple installation I use to test a release before migrating to prod, hence the H2. Running on ubuntu, invoked manually using java -jar metabase.jar. Would use the same keystore as 0.45.3. As mentioned this previous version works fine.

java version "11.0.14" 2022-01-18 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.14+8-LTS-263)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.14+8-LTS-263, mixed mode)

Interestingly a clean install of 0.46 works fine.

Luiggi · March 28, 2023, 3:37pm

we would need all the information you can provide us for reproducing this:

you're mentioning keystore: what do you use the keystore for?
how did you deploy Metabase?
which env vars and which configuration on each?

littlegoblin · March 29, 2023, 3:32pm

Havent customised the deployment. Orginally downloaded 0.45.3, started and configured via UI - nothing fancy, no custom variables. Then downloaded 0.46 to same installation folder. Started up and got error.

The log mainly contains repeats of the Failed to update H2 database described below.

2023-03-29 11:26:55,987 INFO db.update-h2 :: H2 v1 database detected, updating...
2023-03-29 11:26:55,988 INFO db.update-h2 :: Downloading https://repo1.maven.org/maven2/com/h2database/h2/1.4.197/h2-1.4.197.jar
2023-03-29 11:26:56,052 ERROR db.update-h2 :: Failed to update H2 database: #error {
:cause the trustAnchors parameter must be non-empty
:via
[{:type java.security.InvalidAlgorithmParameterException
:message the trustAnchors parameter must be non-empty
:at [java.security.cert.PKIXParameters setTrustAnchors PKIXParameters.java 200]}]
:trace
[[java.security.cert.PKIXParameters setTrustAnchors PKIXParameters.java 200]
[java.security.cert.PKIXParameters PKIXParameters.java 120]
[java.security.cert.PKIXBuilderParameters PKIXBuilderParameters.java 104]
[sun.security.validator.PKIXValidator PKIXValidator.java 99]
[sun.security.validator.Validator getInstance Validator.java 181]
[sun.security.ssl.X509TrustManagerImpl getValidator X509TrustManagerImpl.java 300]
[sun.security.ssl.X509TrustManagerImpl checkTrustedInit X509TrustManagerImpl.java 176]
[sun.security.ssl.X509TrustManagerImpl checkTrusted X509TrustManagerImpl.java 189]
[sun.security.ssl.X509TrustManagerImpl checkServerTrusted X509TrustManagerImpl.java 129]
[sun.security.ssl.CertificateMessage$T12CertificateConsumer checkServerCerts CertificateMessage.java 638]
[sun.security.ssl.CertificateStatus$CertificateStatusConsumer consume CertificateStatus.java 295]
[sun.security.ssl.SSLHandshake consume SSLHandshake.java 392]
[sun.security.ssl.HandshakeContext dispatch HandshakeContext.java 478]
[sun.security.ssl.HandshakeContext dispatch HandshakeContext.java 456]
[sun.security.ssl.TransportContext dispatch TransportContext.java 198]
[sun.security.ssl.SSLTransport decode SSLTransport.java 172]
[sun.security.ssl.SSLSocketImpl decode SSLSocketImpl.java 1377]
[sun.security.ssl.SSLSocketImpl readHandshakeRecord SSLSocketImpl.java 1290]
[sun.security.ssl.SSLSocketImpl startHandshake SSLSocketImpl.java 411]
[sun.security.ssl.SSLSocketImpl startHandshake SSLSocketImpl.java 383]
[org.apache.http.conn.ssl.SSLConnectionSocketFactory createLayeredSocket SSLConnectionSocketFactory.java 436]
[org.apache.http.conn.ssl.SSLConnectionSocketFactory connectSocket SSLConnectionSocketFactory.java 384]
[org.apache.http.impl.conn.DefaultHttpClientConnectionOperator connect DefaultHttpClientConnectionOperator.java 142]
[org.apache.http.impl.conn.BasicHttpClientConnectionManager connect BasicHttpClientConnectionManager.java 313]
[org.apache.http.impl.execchain.MainClientExec establishRoute MainClientExec.java 393]
[org.apache.http.impl.execchain.MainClientExec execute MainClientExec.java 236]
[org.apache.http.impl.execchain.ProtocolExec execute ProtocolExec.java 186]
[org.apache.http.impl.execchain.RetryExec execute RetryExec.java 89]
[org.apache.http.impl.execchain.RedirectExec execute RedirectExec.java 110]
[org.apache.http.impl.client.InternalHttpClient doExecute InternalHttpClient.java 185]
[org.apache.http.impl.client.CloseableHttpClient execute CloseableHttpClient.java 83]
[clj_http.core$request invokeStatic core.clj 621]
[clj_http.core$request invoke core.clj 546]
[clj_http.core$request invokeStatic core.clj 547]
[clj_http.core$request invoke core.clj 546]
[clojure.lang.Var invoke Var.java 384]
[clj_http.client$wrap_request_timing$fn__17492 invoke client.clj 1083]
[clj_http.headers$wrap_header_map$fn__16586 invoke headers.clj 152]
[clj_http.client$wrap_query_params$fn__17382 invoke client.clj 817]
[clj_http.client$wrap_basic_auth$fn__17388 invoke client.clj 840]
[clj_http.client$wrap_oauth$fn__17393 invoke client.clj 857]
[clj_http.client$wrap_user_info$fn__17402 invoke client.clj 877]
[clj_http.client$wrap_url$fn__17474 invoke client.clj 1035]
[clj_http.client$wrap_decompression$fn__17169 invoke client.clj 416]
[clj_http.client$wrap_input_coercion$fn__17294 invoke client.clj 630]
[clj_http.client$wrap_additional_header_parsing$fn__17319 invoke client.clj 685]
[clj_http.client$wrap_output_coercion$fn__17281 invoke client.clj 574]
[clj_http.client$wrap_exceptions$fn__17122 invoke client.clj 250]
[clj_http.client$wrap_accept$fn__17334 invoke client.clj 728]
[clj_http.client$wrap_accept_encoding$fn__17341 invoke client.clj 750]
[clj_http.client$wrap_content_type$fn__17328 invoke client.clj 711]
[clj_http.client$wrap_form_params$fn__17439 invoke client.clj 959]
[clj_http.client$wrap_nested_params$fn__17460 invoke client.clj 993]
[clj_http.client$wrap_flatten_nested_params$fn__17469 invoke client.clj 1017]
[clj_http.client$wrap_method$fn__17407 invoke client.clj 893]
[clj_http.cookies$wrap_cookies$fn__16498 invoke cookies.clj 128]
[clj_http.links$wrap_links$fn__16926 invoke links.clj 63]
[clj_http.client$wrap_unknown_host$fn__17477 invoke client.clj 1046]
[clj_http.client$request_STAR_ invokeStatic client.clj 1174]
[clj_http.client$request_STAR_ invoke client.clj 1167]
[clj_http.client$get invokeStatic client.clj 1180]
[clj_http.client$get doInvoke client.clj 1176]
[clojure.lang.RestFn invoke RestFn.java 423]
[metabase.db.update_h2$update_BANG_ invokeStatic update_h2.clj 73]
[metabase.db.update_h2$update_BANG_ invoke update_h2.clj 68]
[metabase.db.update_h2$update_if_needed invokeStatic update_h2.clj 98]
[metabase.db.update_h2$update_if_needed invoke update_h2.clj 90]
[metabase.db.data_source.DataSource getConnection data_source.clj 29]
[com.mchange.v2.c3p0.WrapperConnectionPoolDataSource getPooledConnection WrapperConnectionPoolDataSource.java 161]
[com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager acquireResource C3P0PooledConnectionPool.java 213]
[com.mchange.v2.resourcepool.BasicResourcePool doAcquire BasicResourcePool.java 1176]
[com.mchange.v2.resourcepool.BasicResourcePool doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess BasicResourcePool.java 1163]
[com.mchange.v2.resourcepool.BasicResourcePool access$700 BasicResourcePool.java 44]
[com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask run BasicResourcePool.java 1908]
[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread run ThreadPoolAsynchronousRunner.java 696]]}
2023-03-29 11:26:56,057 ERROR metabase.core :: Metabase Initialization FAILED
clojure.lang.ExceptionInfo: Unable to connect to Metabase h2 DB. {}
at metabase.db.setup$fn__50150$verify_db_connection__50155$fn__50156$fn__50157.invoke(setup.clj:112)
at metabase.db.setup$fn__50150$verify_db_connection__50155$fn__50156.invoke(setup.clj:110)
at metabase.db.setup$fn__50150$verify_db_connection__50155.invoke(setup.clj:102)
at metabase.db.setup$fn__50235$setup_db_BANG___50240$fn__50241$fn__50244$fn__50245.invoke(setup.clj:156)
at metabase.util.jvm$do_with_us_locale.invokeStatic(jvm.clj:239)
at metabase.util.jvm$do_with_us_locale.invoke(jvm.clj:225)
at metabase.db.setup$fn__50235$setup_db_BANG___50240$fn__50241$fn__50244.invoke(setup.clj:153)
at metabase.db.setup$fn__50235$setup_db_BANG___50240$fn__50241.invoke(setup.clj:152)
at metabase.db.setup$fn__50235$setup_db_BANG___50240.invoke(setup.clj:146)
at metabase.db$setup_db_BANG_$fn__50270.invoke(db.clj:66)
at metabase.db$setup_db_BANG_.invokeStatic(db.clj:61)
at metabase.db$setup_db_BANG_.invoke(db.clj:52)
at metabase.core$init_BANG__STAR_.invokeStatic(core.clj:104)
at metabase.core$init_BANG__STAR_.invoke(core.clj:90)
at metabase.core$init_BANG_.invokeStatic(core.clj:144)
at metabase.core$init_BANG_.invoke(core.clj:139)
at metabase.core$start_normally.invokeStatic(core.clj:156)
at metabase.core$start_normally.invoke(core.clj:150)
at metabase.core$main.invokeStatic(core.clj:189)
at metabase.core$main.doInvoke(core.clj:183)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
at metabase.bootstrap$main.invokeStatic(bootstrap.clj:31)
at metabase.bootstrap$main.doInvoke(bootstrap.clj:28)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at metabase.bootstrap.main(Unknown Source)
Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:692)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140)
at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:372)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274)
at clojure.java.jdbc$db_query_with_resultset_STAR.invokeStatic(jdbc.clj:1111)
at clojure.java.jdbc$db_query_with_resultset_STAR.invoke(jdbc.clj:1093)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182)
at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160)
at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
at metabase.driver.sql_jdbc.connection$can_connect_with_spec_QMARK.invokeStatic(connection.clj:302)
at metabase.driver.sql_jdbc.connection$can_connect_with_spec_QMARK.invoke(connection.clj:299)
at metabase.db.setup$fn__50150$verify_db_connection__50155$fn__50156$fn__50157.invoke(setup.clj:110)
... 31 more
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1507)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685)
... 43 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at java.base/java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
at java.base/java.security.cert.PKIXParameters.(PKIXParameters.java:120)
at java.base/java.security.cert.PKIXBuilderParameters.(PKIXBuilderParameters.java:104)
at java.base/sun.security.validator.PKIXValidator.(PKIXValidator.java:99)
at java.base/sun.security.validator.Validator.getInstance(Validator.java:181)
at java.base/sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:300)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:176)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:189)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
at java.base/sun.security.ssl.CertificateStatus$CertificateStatusConsumer.consume(CertificateStatus.java:295)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:478)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:456)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:198)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1377)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1290)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:383)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:313)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at clj_http.core$request.invokeStatic(core.clj:621)
at clj_http.core$request.invoke(core.clj:546)
at clj_http.core$request.invokeStatic(core.clj:547)
at clj_http.core$request.invoke(core.clj:546)
at clojure.lang.Var.invoke(Var.java:384)
at clj_http.client$wrap_request_timing$fn__17492.invoke(client.clj:1083)
at clj_http.headers$wrap_header_map$fn__16586.invoke(headers.clj:152)
at clj_http.client$wrap_query_params$fn__17382.invoke(client.clj:817)
at clj_http.client$wrap_basic_auth$fn__17388.invoke(client.clj:840)
at clj_http.client$wrap_oauth$fn__17393.invoke(client.clj:857)
at clj_http.client$wrap_user_info$fn__17402.invoke(client.clj:877)
at clj_http.client$wrap_url$fn__17474.invoke(client.clj:1035)
at clj_http.client$wrap_decompression$fn__17169.invoke(client.clj:416)
at clj_http.client$wrap_input_coercion$fn__17294.invoke(client.clj:630)
at clj_http.client$wrap_additional_header_parsing$fn__17319.invoke(client.clj:685)
at clj_http.client$wrap_output_coercion$fn__17281.invoke(client.clj:574)
at clj_http.client$wrap_exceptions$fn__17122.invoke(client.clj:250)
at clj_http.client$wrap_accept$fn__17334.invoke(client.clj:728)
at clj_http.client$wrap_accept_encoding$fn__17341.invoke(client.clj:750)
at clj_http.client$wrap_content_type$fn__17328.invoke(client.clj:711)
at clj_http.client$wrap_form_params$fn__17439.invoke(client.clj:959)
at clj_http.client$wrap_nested_params$fn__17460.invoke(client.clj:993)
at clj_http.client$wrap_flatten_nested_params$fn__17469.invoke(client.clj:1017)
at clj_http.client$wrap_method$fn__17407.invoke(client.clj:893)
at clj_http.cookies$wrap_cookies$fn__16498.invoke(cookies.clj:128)
at clj_http.links$wrap_links$fn__16926.invoke(links.clj:63)
at clj_http.client$wrap_unknown_host$fn__17477.invoke(client.clj:1046)
at clj_http.client$request_STAR_.invokeStatic(client.clj:1174)
at clj_http.client$request_STAR_.invoke(client.clj:1167)
at clj_http.client$get.invokeStatic(client.clj:1180)
at clj_http.client$get.doInvoke(client.clj:1176)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at metabase.db.update_h2$update_BANG_.invokeStatic(update_h2.clj:73)
at metabase.db.update_h2$update_BANG_.invoke(update_h2.clj:68)
at metabase.db.update_h2$update_if_needed.invokeStatic(update_h2.clj:98)
at metabase.db.update_h2$update_if_needed.invoke(update_h2.clj:90)
at metabase.db.data_source.DataSource.getConnection(data_source.clj:29)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:161)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:213)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1176)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1163)
at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1908)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2023-03-29 11:26:56,087 INFO metabase.core :: Metabase Shutting Down ...
2023-03-29 11:26:56,088 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
2023-03-29 11:26:56,101 INFO metabase.core :: Metabase Shutdown COMPLETE

Luiggi · March 29, 2023, 8:43pm

I just tried to reproduce but couldn't:

curl https://downloads.metabase.com/v0.45.3/metabase.jar -o metabase.jar && java -jar metabase.jar
configure Metabase and save a few questions and dashboards
shut down the instance (the H2 DB was left untouched)
curl https://downloads.metabase.com/v0.46.0/metabase.jar -o metabase.jar && java -jar metabase.jar

everything worked correcty

can you check at your end if there's something different?

kemalrasyid · March 30, 2023, 4:15am

Happended in my environment too, not in production, followed your step and the error occurs