When asking questions, some tables return "Every field is hidden right now"

esmeralda · 2017-12-21 17:00:19 UTC

Yesterday (12/20/17) I added a MongoDB database. I was able to add questions normally to some of the tables. However a few others return the message “Every field is hidden right now” when I want to pull up the raw data (no filters or groupings). At that point all I can do is return a count of the number of rows, but can’t see any data or query it.

Weirdly enough, converting a question to a native query allows me to see all the raw data. My research did not turn up any similar issues so I’m not sure what’s wrong.

So far I’ve:

Checked to make sure the tables are queryable in settings.
Removed and added back the database.
Let it sit overnight on the off chance it needed time to finish syncing.

Is there a setting somewhere I need to adjust perhaps?

Software versions:
Metabase v0.28.0-snapshot
OS macOS Sierra v.10.12.6
Browser Chrome v. 63.0.3239.84

jontsai · 2018-02-18 07:34:03 UTC

@esmeralda did you ever figure this out?

I am experiencing the same issue right now. I have a couple of tables where, no matter what, it will say “Every field is hidden right now”

jornh · 2018-02-18 11:32:10 UTC

In the Admin Panel when you Edit Metadata are either the tables or fields set to ‘hidden’?

jontsai · 2018-02-18 15:50:06 UTC

No, neither the tables nor fields are set to Hidden. It still won’t pick it up after I do “Re-scan this table” and “Discard cached field values”

jornh · 2018-02-18 15:58:14 UTC

Do you see the fields as expected in the admin UI?

Any specific pattern in how these tables differ from the one’s that show up? I’ve only tried MongoDB – a while ago – with a “flat” table, could e.g. be problematic with tables nested structures etc?

edit Oh, and please specify your environment … Metabase & MongoDB versions etc. + any clues in the Metabase log in particular when you do a scan/sync under the database setup page under Admin.

Just worked on this - which is MySQL though, so maybe not related?

jontsai · 2018-02-19 00:15:05 UTC

Using Metabase 0.28.1, hosted on AWS using Docker (image: metabase/metabase)
DB: MongoDB 3.4.13, hosted on MongoDB Atlas (M10)

From MongoDB Compass and other MongoDB admin UIs, I see the fields. But the tables are showing up as empty in MongoDB.

I am importing data from Shopify to MongoDB, and then using Metabase on top of it.

My tables are:

customer (empty in metabase)
customer_address (works)
order (empty)
product (works)
product_image (works)
product_tag (works)
product_variant (empty)

I will also try to dig into the logs.

jontsai · 2018-02-19 03:00:21 UTC

Did a sudo docker logs -f metabase while I did stuff on the app. Got a null pointer exception for some of the tables, which would explain why!

02-19 03:03:41 ERROR sync.util :: Error syncing fields for Table 9 'order': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:45 ERROR sync.util :: Error syncing fields for Table 12 'customer': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:46 ERROR sync.util :: Error syncing fields for Table 13 'product_variant': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:48 ERROR sync.util :: Error syncing fields for Table 14 'customer_address': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

jontsai · 2018-02-19 03:19:43 UTC

@jornh

Seems to be this file: https://github.com/metabase/metabase/blob/master/src/metabase/sync/sync_metadata/fields.clj#L306

But that is the extent of my investigation, as I don’t really know how to read Clojure

Filed a new issue here: https://github.com/metabase/metabase/issues/6975

esmeralda · 2018-02-22 16:04:03 UTC

Hello @jontsai

In my specific case, it happened because some of the data I was working with did not follow the schema structure. It took a lot of combing through to find those differences. I believe I also saw the null pointer exception in some of the other tables that were syncing. If I remember correctly, Metabase sometimes does that when it encounters empty fields. In my case it didn’t seem to cause other problems besides throwing an error in the logs.

esmeralda · 2018-02-22 16:10:05 UTC

I just read the issue you opened on github. On a separate project, when we edited our tables, Metabase did not pick it up right away. We left it overnight and the next morning the new fields were showing up. Just something to watch out for.

aviggiano · 2018-03-06 03:15:42 UTC

I am having the same issue

kramer1346 · 2019-02-24 15:15:25 UTC

Same issue

I’m scared. Metabase has been flawless for 2 years now for me.