Not able to open self-hosted metabase

#1

About 90% of the time when we try to load metabase it responds with a 403 Forbidden.

But when you are logged in there are no more problems.

Better descirption here: forbidden 403 before even getting to the login page · Issue #28359 · metabase/metabase · GitHub

It is hosted in AWS ECS and has an ALB in front of it. Has anyone had a similar experience?

Cheers

#2

I have a feeling that it might be the /api/ endpoint that returns 403. Could it do that if it is overloaded?

#3

you haven't provided any info at all about what's your metabase version.

the issue is that you're not using the health check endpoint so ECS is killing your containers and spinning up new ones constantly, so you get 403

#4

We use /api/health for health checking and it has been running since we started it. So I don't agree with you that that would be the problem.

#5 
{
  "browser-info": {
    "language": "sv-SE",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/110.0",
    "vendor": ""
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.18+10",
    "java.vendor": "Eclipse Adoptium",
    "java.vendor.url": "https://adoptium.net/",
    "java.version": "11.0.18",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.18+10",
    "os.name": "Linux",
    "os.version": "4.14.301-224.520.amzn2.x86_64",
    "user.language": "en",
    "user.timezone": "GMT"
  },
  "metabase-info": {
    "databases": [
      "mysql"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "13.7"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.5.0"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2023-01-27",
      "tag": "v0.45.2.1",
      "branch": "release-x.45.2.x",
      "hash": "1a59de7"
    },
    "settings": {
      "report-timezone": "Europe/Stockholm"
    }
  }
}
#6

Is there any logging on the /api endpoint? How do I figure out if the request actually reaches metabase?

#7

do you see that the container is alive when you get the 403's? can you check that in the ECS console?

#8

It says that it is "RUNNING" and started at "9 hours ago".

Right now I cannot replicate the issue anymore, but when it happened this morning it had been running for approx. 24 hrs. So I'll check tomorrow if the issue is back.