No connection after secret rotation

Hi all,

I am trying to connect to my RDS PostgreSQL database after a secret rotation, but unfortunately, it's no longer connecting.

I'm running Metabase via docker compose as seen in A docker-compose file for metabase with postgres. · GitHub.
After the initial setup, I was able to successfully connect to my PostgreSQL database running in AWS RDS via SSH tunnel. I was also able to analyze my existing data in Metabase with meaningful results. Unfortunately, the keys for RDS were rotated a week ago and Metabase is no longer connecting to the database when using the new key. I was able to verify that the key works using PgAdmin4, so I'm a bit confused on what the potential issue could be.

I receive two erros from Metabase. The former is the following

15:44:16.683 [qtp2041092744-26] ERROR metabase.driver.util - Failed to connect to Database
java.util.concurrent.TimeoutException: Timed out after 10.0 s

Is there any way to increase this timeout? I have tried out to set JDBC timeouts as follows:
loginTimeout=100, connectTimeout=100, socketTimeout=100 based on Initializing the Driver | pgJDBC but none seem to have worked.

The latter frequently occurring errors I receive from Metabase is the following:

No pg_hba.conf entry for host "<ip>", user "<username>", database "<databasename>", no encryption

This was also shown when I used the wrong password for connecting to the RDS which is showing me the following:

<username>@<databasename>:[26542]:FATAL: no pg_hba.conf entry for host "<ip>", user "<username>", database "<databasename>", no encryption

Checking the pg_hba.conf reveals the following:

<databasename>=> table pg_hba_file_rules ;
 line_number |  type   |   database    | user_name  | address  | netmask |  auth_method  |  options  | error
-------------+---------+---------------+------------+----------+---------+---------------+-----------+-------
           2 | local   | {all}         | {rdsadmin} |          |         | peer          | {map=rds} |
           6 | local   | {all}         | {all}      |          |         | scram-sha-256 |           |
          12 | host    | {all}         | {rdsadmin} | all      |         | reject        |           |
          13 | host    | {rdsadmin}    | {all}      | all      |         | reject        |           |
          14 | hostssl | {all}         | {all}      | all      |         | md5           |           |
          15 | host    | {replication} | {all}      | samehost |         | md5           |           |

This hasn't changed since the key rotation, so I'm really stumped on why this is suddenly breaking. I have also verified that the metabase_database table stores the correct password as shown in the secrets manager.

I have also logged in via SSH to the SSH tunnel and then connected via psql to the database using the same password and it worked as well.

All in all, I'm a bit stumped on why the supposedly correct password is not accepted.

If you have any idea, help would be appreciated. If there is more debugging information needed, let me know.

Kind regards,
Greifvogel

P.S. Diagnostics Info:

{
  "browser-info": {
    "language": "en-US",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.21+9",
    "java.vendor": "Eclipse Adoptium",
    "java.vendor.url": "https://adoptium.net/",
    "java.version": "11.0.21",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.21+9",
    "os.name": "Linux",
    "os.version": "5.15.133.1-microsoft-standard-WSL2",
    "user.language": "en",
    "user.timezone": "GMT"
  },
  "metabase-info": {
    "databases": [
      "postgres",
      "h2"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "16.1 (Debian 16.1-1.pgdg120+1)"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.6.0"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2023-12-19",
      "tag": "v0.48.1",
      "hash": "a8302d4"
    },
    "settings": {
      "report-timezone": null
    }
  }
}

I did some further research and it seems to be related to not using SSL. However, using SSL guarantees my connection to be too slow to connect to the database. I tried setting "connect_timeout", but unfortunately, this didn't help. I have also tried to set the several timeout parameters of the JDBC (Initializing the Driver | pgJDBC) but none worked. Is there any way to increase this timeout?

[

MB_DB_CONNECTION_TIMEOUT_MS

](Environment variables)

1 Like

Thanks a lot. This solved my issue. Seems like I looked at the wrong place :smiley: