We are running a metabase instance on Amazon Elastic Beanstalk.
It has been running for one year and we've only done 2 version upgrades since.
This morning for some reason the IP address of the instance changed a couple of times (it never did in months, only with the upgrades) and then before changing the rules of our database firewall once again, we decided to upgrade to 0.40.
Problem is that it didn't let us upgrade because of the Docker thing (I think): an ugly "deprecated platform" thing showed up:
Ok, no idea how to fix it, but after some minutes I found out I could upgrade to a newer docker or something. I tried that. Now the instance doesn't start.
I also tried to upgrade to 0.40... still, same issue.
So, I'm locked in a situation where our instance no longer starts up, so no idea what to do. Any help would be appreciated.
@psantosl I don't know where version v0.40.0-1 is coming from - Metabase version is v0.40.0
Like I said initially, you have likely setup EB with a coupled RDS. You need to decouple it.
The data is stored in RDS - not in EB (unless you used H2).
Revert to your backup before you tried to do anything, decouple the RDS, and then upgrade.
I don't know what "RDS or EB". Do you mean if we setup an external database or something? I don't think so, we just clicked "next/next" on whatever Elastic Beanstalk thing was available.
@psantosl
Rule number 1 when dealing with software: Always have backups.
I have no idea if you have lost everything, since I don't know how you setup your EB (Elastic Beanstalk) initially with or without RDS (Relational Database Service)
You will have to figure out if you did setup with RDS or not, but given that you have upgraded Metabase before, then we'll guess that you have.
I already linked to how to decouple RDS - pretty please, slow down, don't make mistakes, and read everything - but first, make sure you have a backup.
Yeah, yeah. I didn't expect 0.40 to do special stuff after reading the release notes quickly. I read them inside metabase in the part where it explains the new version thing, but I didn't see anything special, or I failed to see it.
I believe I might have a postgres for this, so maybe I'm lucky.
Ok, it looks like we have a postgres database for this (I see the DB thing, not sure if this is related to metabase, but since we don't have anything else in this datacenter, it seems likely).
I'll try to follow the guide you sent me and see if I can connect the old metadata (postgres) to whatever the new 0.40 is doing.
@psantosl I am getting the exact same issue.. I had safely updated my stage environment to 0.39.2 without issue.. something changed in 0.40.0 i havent tried restarting the instance yet..
I am having the same problem. I think 0.40 is broken.
I did an incremental upgrade and successfully upgraded from v0.37.8 to v0.39.0 to v0.39.4 but ut the upgrade to the latest version (v0.40.0) failed. my config was same as @psantosl where Metabase running on Beanstalk and docker and I have dedicated posgresSQL to store Metabase data as well.
looking at eb-activity.log. here is the error with v0.40:
[2021-07-12T20:57:08.737Z] INFO [14849] - [Application update v0.40.0.metabase-aws-eb.zip@5/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_Performance_Team_Metabase/Command 07-install_cloudwatch_agent] : Starting activity...
[2021-07-12T20:57:09.085Z] INFO [14849] - [Application update v0.40.0.metabase-aws-eb.zip@5/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_Performance_Team_Metabase/Command 07-install_cloudwatch_agent] : Activity execution failed, because: Loaded plugins: priorities, update-motd, upgrade-helper
No package amazon-cloudwatch-agent available.
Error: Nothing to do
(ElasticBeanstalk::ExternalInvocationError)
Same problem here "No package amazon-cloudwatch-agent available". I am on 0.39.2 currently, noticed in the docker configs, this is a new config step not in 0.39.2
# do server_https first to avoid overwriting other config changes
01_server_https:
command: ".ebextensions/metabase_config/metabase-setup.sh server_https"
ignoreErrors: true
02_log_x_real_ip:
command: ".ebextensions/metabase_config/metabase-setup.sh log_x_real_ip"
ignoreErrors: true
03_install_papertrail:
command: ".ebextensions/metabase_config/metabase-setup.sh install_papertrail"
test: test $PAPERTRAIL_HOST
ignoreErrors: true
04_try_papertrail:
command: "/sbin/service remote_syslog restart"
test: test -e /etc/log_files.yml
ignoreErrors: true
05_try_nginx:
command: "/sbin/service nginx restart"
test: nginx -t
ignoreErrors: false
0.40.0 Container Commands
container_commands:
# do server_https first to avoid overwriting other config changes
01_server_https:
command: ".ebextensions/metabase_config/metabase-setup.sh server_https"
ignoreErrors: true
Nevertheless, you should be able to upgrade by downloading the Elastic Beanstalk deployment zip, opening it, deleting the steps 7 and 8 (where it searches for amazon-cloudwatch-agent package and runs it), zipping the file again and deploying. I'm investigating why this might be happening but it should be a very old Amazon Linux that is running and it doesn't find the package
For everyone watching this thread, the reason you can't upgrade is because you are using a deprecated Elastic Beanstalk platform which is based on Amazon Linux, the latest one is Amazon Linux 2 which you should move to ASAP.
delete the current deprecated Elastic Beanstalk deployment (make sure you save all the environment variables you use to connect to the Application Database or they will be lost when you delete the deployment)
start a new deployment connecting your new Metabase Elastic Beanstalk deployment with the old Application Database
I will try (I can't promise anything) to make an Elastic Beanstalk deployment that can work with old Amazon Linux Elastic Beanstalk platforms, but you should move from deprecated platforms for your own sake.
UPDATE: I did a custom deploy for upgrading deprecated platforms for .40 version (this will be reviewed, tested and if we consider that it's a good path forward we will merge it)
