What we did:
We changed a few of our tables (dropped and renamed certain columns), which would break questions that were built on top of them. We were expecting to be able to amend the saved questions (either written in SQL or using the UI) and save questions with their new respective columns.
What we are seeing:
The new columns are not present in the GUI at all.
They are, however, there when you use the SQL engine: SELECT * FROM ....
This means we cannot at present fix any of the questions produced via the GUI.
What we have tried:
We have re-scanned the database and discarded the cache values, with no luck.
@Kieran_m
Upgrade to 0.33.5.1 if you are using the Docker version, since there were a build problem.
Just to make sure, are you also clicking “Sync database schema now” or just scanning? You need to sync.
Do you see any errors during the sync process? Admin > Troubleshooting > Logs
We were seeing some memory issues and so provided the following config to our docker deployment ==> - name: JAVA_TOOL_OPTIONS value: '-Xmx496m'
That helped prevent the container falling over, but Metabase is still not correctly scanning the schema to capture the new fields.
I noticed this error from the logs:
[a5a4f484-da99-449a-8760-c440209c88c5] 2019-11-18T09:52:41+00:00 WARN metabase.middleware.log GET /api/database/10/metadata 403 3.3 ms (3 DB calls) "You don't have permissions to do that."
Database 10 is explorable using the Metabase UI so the database connection permissions are correct.
There is then a large error message:
[a5a4f484-da99-449a-8760-c440209c88c5] 2019-11-18T09:52:49+00:00 ERROR metabase.sync.util Error generating fingerprint for Field 23,785 'engines': class java.lang.String cannot be cast to class java.lang.Number (java.lang.String and java.lang.Number are in module java.base of loader 'bootstrap') ("clojure.lang.RT.doubleCast(RT.java:1355)" "bigml.histogram.core$when_double.invokeStatic(core.clj:104)" "bigml.histogram.core$when_double.invoke(core.clj:103)" ...
Then the column error appears as so:
[a5a4f484-da99-449a-8760-c440209c88c5] 2019-11-18T09:53:18+00:00 ERROR metabase.driver.sql-jdbc.execute Failed to set timezone: PSQLException: Message: ERROR: column table_name.Total GWP does not exist Position: 129 SQLState: 42703 Error Code: 0
I am not sure on why the exception is thrown on timezone setting - any ideas?
The timezone error. Have you redacted some of error or does it actually say column table_name.Total GWP does not exist?
Can you check your Postgres log to see the full query? It might help finding the root cause.
@flamber thanks for sharing the links, I have left comments
No I haven’t redacted anything, the query is correct. Total GWP is a real column that was removed and replaced in the underlying source table. The error is what I would expect - albeit I don’t understand the context of the timezone element?!
But due to the possible fingerprint issue, I am unable to change the query to address the new column on the table. Rendering any affected questions built in the UI completely useless.
@Kieran_m
The timezone error might be two different queries, but it would really help if you checked the Postgres log, since it should show what queries are failing.
I think it’s trying to set timezone and then it’s also failing to find the non-existing column.
In case anyone reading this thread hits similar problems. A short-term fix to make Metabase usable again is to re-create the previously removed columns, with the old titles and blank ’ ’ entries.
This enables Metabase to properly sync with the new schema and bring in any new columns. Under the Admin panel you can hide these new “fake” columns from the UI. This is sub-optimal as your reporting layer now has obsolete code, but at least Metabase is useable again.
You can also create a new database source and see if the new one syncs. If so, something has run amock, hopefully it’s as simple as this date primary key issue.