When asking questions, some tables return "Every field is hidden right now"

esmeralda · December 21, 2017, 5:00pm

Yesterday (12/20/17) I added a MongoDB database. I was able to add questions normally to some of the tables. However a few others return the message “Every field is hidden right now” when I want to pull up the raw data (no filters or groupings). At that point all I can do is return a count of the number of rows, but can’t see any data or query it.

Weirdly enough, converting a question to a native query allows me to see all the raw data. My research did not turn up any similar issues so I’m not sure what’s wrong.

So far I’ve:

Checked to make sure the tables are queryable in settings.
Removed and added back the database.
Let it sit overnight on the off chance it needed time to finish syncing.

Is there a setting somewhere I need to adjust perhaps?

Software versions:
Metabase v0.28.0-snapshot
OS macOS Sierra v.10.12.6
Browser Chrome v. 63.0.3239.84

jontsai · February 18, 2018, 7:34am

@esmeralda did you ever figure this out?

I am experiencing the same issue right now. I have a couple of tables where, no matter what, it will say “Every field is hidden right now”

jornh · February 18, 2018, 11:32am

In the Admin Panel when you Edit Metadata are either the tables or fields set to ‘hidden’?

jontsai · February 18, 2018, 3:50pm

No, neither the tables nor fields are set to Hidden. It still won’t pick it up after I do “Re-scan this table” and “Discard cached field values”

jornh · February 18, 2018, 3:58pm

Do you see the fields as expected in the admin UI?

Any specific pattern in how these tables differ from the one’s that show up? I’ve only tried MongoDB – a while ago – with a “flat” table, could e.g. be problematic with tables nested structures etc?

edit Oh, and please specify your environment … Metabase & MongoDB versions etc. + any clues in the Metabase log in particular when you do a scan/sync under the database setup page under Admin.

Just worked on this - which is MySQL though, so maybe not related?

jontsai · February 19, 2018, 12:15am

Using Metabase 0.28.1, hosted on AWS using Docker (image: metabase/metabase)
DB: MongoDB 3.4.13, hosted on MongoDB Atlas (M10)

From MongoDB Compass and other MongoDB admin UIs, I see the fields. But the tables are showing up as empty in MongoDB.

I am importing data from Shopify to MongoDB, and then using Metabase on top of it.

My tables are:

customer (empty in metabase)
customer_address (works)
order (empty)
product (works)
product_image (works)
product_tag (works)
product_variant (empty)

I will also try to dig into the logs.

jontsai · February 19, 2018, 3:00am

Did a sudo docker logs -f metabase while I did stuff on the app. Got a null pointer exception for some of the tables, which would explain why!

02-19 03:03:41 ERROR sync.util :: Error syncing fields for Table 9 'order': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:45 ERROR sync.util :: Error syncing fields for Table 12 'customer': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:46 ERROR sync.util :: Error syncing fields for Table 13 'product_variant': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

02-19 03:03:48 ERROR sync.util :: Error syncing fields for Table 14 'customer_address': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
 "driver.mongo$describe_table_field.invoke(mongo.clj:120)"
 "driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
 "driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
 "driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
 "driver.mongo$describe_table.invoke(mongo.clj:159)"
 "driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
 "sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
 "sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
 "sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
 "sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
 "sync.util$do_with_error_handling.invoke(util.clj:119)"
 "driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
 "driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
 "util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
 "util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
 "driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
 "driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
 "driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
 "driver.mongo$sync_in_context.invoke(mongo.clj:68)"
 "driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
 "sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
 "sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
 "sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
 "sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
 "sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
 "sync.util$do_sync_operation.invokeStatic(util.clj:142)"
 "sync.util$do_sync_operation.invoke(util.clj:139)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
 "sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
 "api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")

jontsai · February 19, 2018, 3:19am

@jornh

Seems to be this file: https://github.com/metabase/metabase/blob/master/src/metabase/sync/sync_metadata/fields.clj#L306

But that is the extent of my investigation, as I don’t really know how to read Clojure

Filed a new issue here: https://github.com/metabase/metabase/issues/6975

esmeralda · February 22, 2018, 4:04pm

Hello @jontsai

In my specific case, it happened because some of the data I was working with did not follow the schema structure. It took a lot of combing through to find those differences. I believe I also saw the null pointer exception in some of the other tables that were syncing. If I remember correctly, Metabase sometimes does that when it encounters empty fields. In my case it didn’t seem to cause other problems besides throwing an error in the logs.

esmeralda · February 22, 2018, 4:10pm

I just read the issue you opened on github. On a separate project, when we edited our tables, Metabase did not pick it up right away. We left it overnight and the next morning the new fields were showing up. Just something to watch out for.

aviggiano · March 6, 2018, 3:15am

I am having the same issue

kramer1346 · February 24, 2019, 3:15pm

Same issue

I’m scared. Metabase has been flawless for 2 years now for me.

Jerry · November 17, 2020, 9:25am

I have the same problem too. At first,the fields order of my records does not the same. After reorder the fields,the problem of some tables disappear.I guess,the problem is related to the fields order of mongodb.

flamber · November 17, 2020, 10:09am

@Jerry Please post “Diagnostic Info” from Admin > Troubleshooting. And check the logs for any errors during the sync process - Admin > Troubleshooting > Logs.

Jerry · November 18, 2020, 2:59am

[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:29:29+08:00 ERROR metabase.models.field-values An error occurred when extracting field-values
com.mongodb.MongoCommandException: Command failed with error 237 (CursorKilled): ‘operation was interrupted’ on server 10.0.3.203:27017. The full response is { “operationTime” : { “$timestamp” : { “t” : 1605655766, “i” : 6 } }, “ok” : 0.0, “errmsg” : “operation was interrupted”, “code” : 237, “codeName” : “CursorKilled”, “$clusterTime” : { “clusterTime” : { “$timestamp” : { “t” : 1605655766, “i” : 6 } }, “signature” : { “hash” : { “$binary” : “AAAAAAAAAAAAAAAAAAAAAAAAAAA=”, “$type” : “00” }, “keyId” : { “$numberLong” : “0” } } } }
at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:179)
[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:35:05+08:00 ERROR metabase.driver.mongo Error introspecting collection: employee_operation #error {
:cause Java heap space
:via
[{:type java.lang.OutOfMemoryError
:message Java heap space}]
:trace
[]}
[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:35:33+08:00 ERROR metabase.driver.mongo Error introspecting collection: productlog #error {
:cause Java heap space
:via
[{:type java.lang.OutOfMemoryError
:message Java heap space}]
:trace
[]}

After checked the logs,I found errors above.And the JAVA_OPTS : -Xms1024M -Xmx4096M -Xmn2048M -XX:MetaspaceSize=512M -XX:MaxMetaspaceSize=1024M -Duser.timezone=GMT+08.

{
  "browser-info": {
    "language": "zh-CN",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; Tablet PC 2.0; rv:11.0) like Gecko",
    "vendor": ""
  },
  "system-info": {
    "file.encoding": "GBK",
    "java.runtime.name": "Java(TM) SE Runtime Environment",
    "java.runtime.version": "1.8.0_261-b12",
    "java.vendor": "Oracle Corporation",
    "java.vendor.url": "http://java.oracle.com/",
    "java.version": "1.8.0_261",
    "java.vm.name": "Java HotSpot(TM) Client VM",
    "java.vm.version": "25.261-b12",
    "os.name": "Windows Server 2016",
    "os.version": "10.0",
    "user.language": "en",
    "user.timezone": "Asia/Shanghai"
  },
  "metabase-info": {
    "databases": [
      "h2",
      "mongo",
      "postgres",
      "sqlserver"
    ],
    "hosting-env": "unknown",
    "application-database": "h2",
    "application-database-details": {
      "database": {
        "name": "H2",
        "version": "1.4.197 (2018-03-18)"
      },
      "jdbc-driver": {
        "nam

flamber · November 18, 2020, 9:26am

@Jerry Continued in new topic: Update field values failure