Loading Metabase dumps not working property

  • Please take a look at this question yaml file created by the dump:

question_sample.yaml

dataset_query:
  database: /databases/db
  query:
    source-table: /databases/db/schemas/CLIENT/tables/table_name1
    filter:
		...
    aggregation:
		...
    breakout:
		...
    joins:
    - fields: all
      source-table: /databases/db/schemas/CLIENT/tables/table_name2
      condition:
      - =
      - - field-id
        - /databases/db/schemas/CLIENT/tables/table_name1/fields/userid
      - - joined-field
        - table_name2
        - - field-id
          - /databases/db/schemas/CLIENT/tables/table_name2/fields/userid
      alias: table_name2
    order-by:
    - - asc
      - - binning-strategy
        - - joined-field
          - table_name2
          - - field-id
            - /databases/db/schemas/CLIENT/tables/table_name2/fields/demo_age
        - default
  type: query
  • When loading the dump, this question is broken, here is the dataset_query created after loading the dump. Please notice source-table value inside Joins key is not numeric
{
	"database": 1,
	"query": {
		"source-table": 51,{
	"database": 1,
	"query": {
		"source-table": 51,
		"filter": ...,
		"breakout": ...,
		"joins": [{
			"fields": "all",
			"source-table": "/databases/db/schemas/CLIENT/tables/table_name2",
			"condition": ["=", ["field-id", 888],
				["joined-field", "table_name2", ["field-id", 794]]
			],
			"alias": "table_name2"
		}],
		]
	}
  • User gets displayed this error message:

Value does not match schema: {:query {:joins [{:source-table (not (matches-some-precondition? a-java.lang.String))}]}}

  • This question uses a join between 2 tables. Other questions with 1 single table data had no issues getting imported.

Thanks in advance

1 Like

Hi @rodxyz
Please post “Diagnostic Info” from Admin > Troubleshooting.
I have no idea what you’re doing or trying to do, so perhaps you should start with explaining that.

Hi @flamber,

I am following this guide to create a dump from an instance and load it into another

  • Create dump
    java -jar metabase.jar dump [dump_name] --user [example@example.com]

This generated the yaml files structure

  • Loading dump

java -jar metabase.jar load [my_dump] --mode [skip/update] --on-error

This loads the yaml files into metabase metadata

Diagnostic Info. We are using Metabase Enterprise on Trial

{
  "browser-info": {
    "language": "en-US",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "ANSI_X3.4-1968",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.7+10-LTS",
    "java.vendor": "Oracle Corporation",
    "java.vendor.url": "http://java.oracle.com/",
    "java.version": "11.0.7",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.7+10-LTS",
    "os.name": "Linux",
    "os.version": "4.14.158-129.185.amzn2.x86_64",
    "user.language": "en",
    "user.timezone": "UTC"
  },
  "metabase-info": {
    "databases": [
      "presto"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "9.6.16"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.2.8"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2020-04-29",
      "tag": "v1.35.3",
      "branch": "enterprise-release-1.35.x",
      "hash": "10740ae"
    },
    "settings": {
      "report-timezone": null
    }
  }
}

@rodxyz
When using the Enterprise Edition, please contact support via email.
Latest release is 1.35.4.1
Do you see any errors during the load? I’m going to try to reproduce.

Thank you @flamber

Please try using a question that joins 2 tables. Single table data question were loaded successfully.

Hi @rodxyz, I am facing the same issue. Did you get it solved?

Tried with multiple version incl 1.35.4.1 and 1.36.11 to no avail.

@mksrom When using the Enterprise Edition, please contact support via email.
There’s a lot of work being done to Serialization right now and will likely ship in 1.38.1

Cool, good to hear! Does it mean this is not supported now?

@mksrom Please use the support email, so the problem can be connected to your company and prioritized, and we are able to update you when the issue is fixed (or contact if we need further information to fix).

You can use the following query that you need to run the target instance after the Serialization load.
Run it until there’s no more rows to update - that’s a Postgres limitation.
Make sure to always do backups for change application database data manually.

UPDATE report_card
SET dataset_query=REGEXP_REPLACE(dataset_query, lookup.findstr, lookup.replacestr)
FROM (
    WITH wrongref AS (
        SELECT rc.id, REGEXP_MATCH(rc.dataset_query, '"source-table":("/databases/([^/]+)/schemas/([^/]+)/tables/([^"]+)")') AS findstr
        FROM report_card AS rc WHERE rc.dataset_query LIKE '%"source-table":"/databases/%'
    )
    SELECT
      wrongref.id,
      concat('"source-table":', wrongref.findstr[1]) AS findstr,
      concat('"source-table":', (
        SELECT mt.id::varchar FROM metabase_database AS md LEFT JOIN metabase_table AS mt ON mt.db_id=md.id WHERE md."name"=wrongref.findstr[2] AND mt."schema"=wrongref.findstr[3] AND mt."name"=wrongref.findstr[4])
      ) AS replacestr
    FROM wrongref
) AS lookup
WHERE report_card.id=lookup.id
1 Like

Thanks @flamber ! works like a charm.

Note that I had to replace REGEXP_MATCH by REGEXP_MATCHES as I am using PostgreSQL 9.6.