Bigquery backup for migration

Wil that populates all the graphs and questions as well.
Where do i found this application database. Is it store somewhere on VM?

@Prajakta2021 Post "Diagnostic Info" from Admin > Troubleshooting.

{
"browser-info": {
"language": "en-US",
"platform": "Linux x86_64",
"userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0",
"vendor": ""
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "OpenJDK Runtime Environment",
"java.runtime.version": "11.0.11+9",
"java.vendor": "AdoptOpenJDK",
"java.vendor.url": "https://adoptopenjdk.net/",
"java.version": "11.0.11",
"java.vm.name": "OpenJDK 64-Bit Server VM",
"java.vm.version": "11.0.11+9",
"os.name": "Linux",
"os.version": "5.4.0-1046-gcp",
"user.language": "en",
"user.timezone": "GMT"
},
"metabase-info": {
"databases": [
"bigquery"
],
"hosting-env": "unknown",
"application-database": "postgres",
"application-database-details": {
"database": {
"name": "PostgreSQL",
"version": "12.3 (Debian 12.3-1.pgdg100+1)"
},
"jdbc-driver": {
"name": "PostgreSQL JDBC Driver",
"version": "42.2.18"
}
},
"run-mode": "prod",
"version": {
"date": "2021-06-15",
"tag": "v0.39.4",
"branch": "release-x.39.x",
"hash": "f538050"
},
"settings": {
"report-timezone": null
}
}
}

@Prajakta2021 Yes, the application database contains all the information stored in Metabase.
You are using PostgreSQL 12.3 for the application database, which you have defined somewhere during configuration on the container:
https://www.metabase.com/docs/latest/operations-guide/running-metabase-on-docker.html#using-postgres-as-the-metabase-application-database

I have define Database as Bigquery when setup metabase.

@Prajakta2021 You are misunderstanding the difference between the Metabase application database (where Metabase stores all it's info; questions, dashboard, settings, etc) and your data sources.
https://www.metabase.com/docs/latest/operations-guide/configuring-application-database.html

Thanks for clarity. :slight_smile:

Hi Got one more question.
Since i am running Metabase using google cloud platform. My Metabase is running on docker And my metabase application database is postgres. Now i have understood this things.
But when i am trying to find migration of metabase from GCP development environment to GCP test environment. I am getting confused to navigate to migration steps.
Please correct me if i am using incorrect steps

  1. currently i am following self hosting steps from below link -
    [How to run Metabase in production](https://www.metabase.com/blog/how-to-run-metabase-in-production/
  2. i collected docker server details and postgres details
  3. install metabase using gcp at test environment
  4. follow After you’ve installed Metabase steps from above link

@Prajakta2021 I don't understand your question, but you would simply make a duplicate of the application database and then setup a new container and point it to the duplicated database.
There's no migration involved, since you are already using Postgres both places. It's just a regular duplication using pg_dump or whatever tool you use for duplicating databases.
Just remember to change the Site URL (Admin > Settings > General) in Metabase on the new instance.

Can you please confirm me one thing about my understanding. I assume that when GCP deploys metabase using GCP deployment manager it creates one docker image and then deploy it. i want to know where do i found that docker image.

Below is my database tab looks in Metabase :

@Prajakta2021 GCP doesn't deploy Metabase - it depends on what you did, but let's continue in the other topic you've now created: Backup Metabsae

I used GCP platform marketplace free version to deploy metabase. Which deploy metabase for me using GCP deployment manager. i can share you path from where i found steps to deploy in supported document.

Below is the link which shows steps :
GCP Getting Started Guide (techlatest.net)

Below is GCP path which deploy metabase :

@Prajakta2021 Then you should contact them, since they are changing you ~$100/month for "support".

May I highly recommend that you do not use them.

It would be cheaper if you just used Metabase Cloud: https://www.metabase.com/pricing/

If you want/need to self-host, then setup a plain Debian/Ubuntu server on GCP and then follow this guide:
https://www.metabase.com/docs/latest/operations-guide/running-metabase-on-debian.html

As my assignment is related to GCP i need to use that only. Yesterday i dig down little bit found metabase.db.h2.db.
Could you please confirm if this is the only db i am able to means my application database is H2 or what.

@Prajakta2021

I don't understand what you are referring to. Where are you seeing metabase.db.h2.db?

You are using Postgres as the Metabase application database as you can see in Admin > Troubleshooting > "Diagnostic Info" - look at this "application-database": "postgres",

But you need to contact the people who have created the deployment/image for support - it's not an official Metabase image.

yah. since i didnt see postgres i choose to go with other approach. where i created new vm , deployed docker and now connected connected metabase image through docker. but now when i tried to connect bigquery through it. it gives me timeout error.

@Prajakta2021 There's way too many different problems going on and you're not telling what you're doing, so it's impossible to know.

If you are not deploying via the third-party image, but now using Docker, then you need to set the application database in the Docker run command:
https://www.metabase.com/docs/latest/operations-guide/running-metabase-on-docker.html#using-postgres-as-the-metabase-application-database

Hi
I have setup metabase internet restricted machine. but when i tried to add bigquery database its give me time out error.
Please find out below diagnostic info as well as error log -
{
"browser-info": {
"language": "en-US",
"platform": "Win32",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; rv:11.0) like Gecko",
"vendor": ""
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "OpenJDK Runtime Environment",
"java.runtime.version": "11.0.11+9",
"java.vendor": "AdoptOpenJDK",
"java.vendor.url": "https://adoptopenjdk.net/",
"java.version": "11.0.11",
"java.vm.name": "OpenJDK 64-Bit Server VM",
"java.vm.version": "11.0.11+9",
"os.name": "Linux",
"os.version": "5.4.0-1049-gcp",
"user.language": "en",
"user.timezone": "GMT"
},
"metabase-info": {
"databases": [
"h2"
],
"hosting-env": "unknown",
"application-database": "h2",
"application-database-details": {
"database": {
"name": "H2",
"version": "1.4.197 (2018-03-18)"
},
"jdbc-driver": {
"name": "H2 JDBC Driver",
"version": "1.4.197 (2018-03-18)"
}
},
"run-mode": "prod",
"version": {
"date": "2021-07-14",
"tag": "v0.40.1",
"branch": "release-x.40.x",
"hash": "ed8f9c8"
},
"settings": {
"report-timezone": null
}
}
}
Error :
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:04+00:00 INFO metabase.sync.util FINISHED: Sync h2 Database 1 'Sample Dataset' (4.3 s)
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:04+00:00 INFO metabase.core Metabase Initialization COMPLETE
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:05+00:00 INFO metabase.util.i18n.impl Reading available locales from locales.clj...
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:08+00:00 DEBUG metabase.server.middleware.log GET /api/user/current 401 1.7 ms (0 DB calls)
"Unauthenticated"

[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:08+00:00 DEBUG metabase.server.middleware.log GET /api/session/properties 200 2.5 ms (0 DB calls) App DB connections: 0/4 Jetty threads: 7/50 (3 idle, 0 queued) (52 total active threads) Queries in flight: 0 (0 queued)
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:14:09+00:00 DEBUG metabase.server.middleware.log GET /api/session/properties 200 2.4 ms (0 DB calls) App DB connections: 0/4 Jetty threads: 6/50 (3 idle, 0 queued) (52 total active threads) Queries in flight: 0 (0 queued)
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:15:58+00:00 DEBUG metabase.server.middleware.log GET /api/user/current 401 330.7 µs (0 DB calls)
"Unauthenticated"

[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:15:58+00:00 DEBUG metabase.server.middleware.log GET /api/session/properties 200 2.1 ms (0 DB calls) App DB connections: 0/4 Jetty threads: 6/50 (2 idle, 0 queued) (55 total active threads) Queries in flight: 0 (0 queued)
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:15:59+00:00 DEBUG metabase.server.middleware.log GET /api/session/properties 200 2.1 ms (0 DB calls) App DB connections: 0/4 Jetty threads: 6/50 (2 idle, 0 queued) (55 total active threads) Queries in flight: 0 (0 queued)
[d24d1c09-d23e-435e-acc3-abd7490d9c4a] 2021-08-03T07:23:44+00:00 ERROR metabase.util.stats Sending usage stats FAILED
java.net.ConnectException: Connection timed out (Connection timed out)

@Prajakta2021 Well, if Metabase does not have access to the internet, then how should it be able to connect to BigQuery, which is an internet database?

You're not reading what I'm writing. I don't think I can help you anymore.

Hi Sorry about confusion. I will create new ticket and explain in detailed that what exact problem i am facing.