Managing dev/QA/prod environments

#1

I’m starting to evaluate Metabase.

I’m wondering if there is any consideration to managing configuration across multiple environment to allow for creating metrics/questions/pulses/etc. in a development environment, promoting it to the QA environment, then promoting it to production.

Do any of you have similar needs and ideas of how to achieve it?

5 Likes
#2

I would really be interested in this as well. It would really be nice if there could be some sort of git integration as well, where you could check you changes into a repos, and deploy out to QA and Prod.

#3

@mlutze Great idea!

As someone who works in a large corporate I would like to +1 this query.
One way this can be done is allowing an export / import of each thing - dashboard, question etc.

If it were all exposed over rest that would be awesome!

#4

@just1689 Does anybody know if there is a way to export objects? I have done a bit of looking in the repository database, and see where the reports are defined (in the report_card) table. But there are foreign keys which reference other objects defined in other tables. ex: database/table/columns. So it would not be so simple as just extracting that row, and then inserting into the other environment.

I am wondering does anybody have a thought on how to just export a report definition?

#5

I am experiencing exactly the same needs / concerns.

I would suggest an integration with git, where (roughly):

  • A human-readable file format is defined for each object type (metrics/questions/etc) is defined (json?)
  • New settings are added, to support reading/writing to/from a github repo (repo name, branch, ssh key, perhaps a checkbox for allowing writing) by admins.
  • On some trigger (polling, or an admin button press?), DB objects are deserialized from the repo into the DB.
  • On some trigger (save, or an admin button press) DB objects are serialized to the repo, committed, pushed (perhaps with a dialog to ask for a new branch name, as github does)

This would allow my team to:

  • Develop locally (Metabase on top of local git repo)
  • Create pull requests that could be reviewed for correctness and tested locally
  • Merge pull requests and reload current branches in a staged environment
  • Merge the staged branch to prod after QA, on a release cycle.

Thoughts? Does any of this exist in Metabase already?

Edit: Here’s how Looker approaches it; I think there’s a much leaner version that would be plenty useful to developers, that could be done here:

4 Likes
#6

Yes to this! I do like how Looker incorporates git. Though they do not really have the concept of a multiple environments, rather enable the Dev mode and do development in the production.

#7

Bump! We started integrating Metabase at our company and need to deploy it for several different clients. We’d like to keep the metadata intact but allow the clients to create their own questions. The problem is how do we handle table additions and similar updates? Right now, when an update happens, we can either keep the client-specific questions and rewrite the generic metadata from scratch, or lose the client-specific questions and keep the metadata.

Although Metabase tries its best to infer as much as it possibly can about our large database, a lot of type information is still missing and many fields need to be corrected by hand (implying the need for a full manual inspection). This unfortunately renders it unusable in the long run.

Any effort that would push the project closer to scalability and manageability in such a scenario would be greatly appreciated! So far, we’ve had an amazing experience with the tool, and we definitely have big plans with Metabase. We will, of course, contribute any tweaks and improvements back to the community!

Thanks for Metabase, and thanks again for giving it such a sleek design.

~viluon

Copy dashboards and questions from one server to another
#8

+1 to bcipollini’s proposal above.
Here’s the github issue about this: https://github.com/metabase/metabase/issues/6485

#9

Working for a financial firm we have very strict rules on who can query a production database. All queries that access a production database need to go through rigorous load testing and qa to ensure they aren’t leaking data. Metabase is a fantastic product but not being able to version / ci/cd / qa queries before the make it to production severely limits our ability to adopt this product. Anything done to improve this that allows for versioning and deployment to different environments would be extremely useful

#10

Have anyone found a satisfying solution for this issue?

#11

We’ve been facing the same problem. We want to easily duplicate and updating questions between the environment. In our case, each environment has one separated collection.

We’ve ended up using Metabase REST API and create a script for it. We decided to make it open-source https://github.com/itmi-id/metabase-migration

#12

That’s pretty awesome, thanks for creating and making it open source! One thing that’s definitely coming up more often is a more programmatic way to create/update questions so we appreciate having this example.

I should note that the Enterprise Edition has aserialization feature that allows you to copy the contents of one Metabase instance to another. That’s definitely intended more for keeping different instances in sync, though, as opposed to iterative development on a specific question or dashboard.

#13

Thanks for this. I followed the instructions to get it going and I get the following error: “TypeError: Cannot read property ‘status’ of undefined”. It also says { ‘X-Metabase-Session’: undefined }.

Is this an error you’ve seen before?

#14

I know this question is really old, but I’ve been working on a solution that works not only for cards, but for dashboards.
https://www.nuget.org/packages/MetabaseCLI/
This is a cli tool that runs on dotnet 5.
You can use metabase-cli -h on the terminal to see the available commands.

#15

I was wondering if there is a way to export/import a metabase instance using their API

#16

This is largely because you may not be setting up correct METABASE_BASE_URL in .env file mentioned.

For example, it should be,
METABASE_BASE_URL=http://localhost:3000/api

Hope this helps.

#17

Hi ! you can use my tool https://github.com/Yambr/metabase-ci-tool

  • desktop app (configure and manual) and console (for ci/cd)
  • create and update collections/cards/dashboards

nodejs (electron +vue)

Open for any MR!)