Yesterday (12/20/17) I added a MongoDB database. I was able to add questions normally to some of the tables. However a few others return the message “Every field is hidden right now” when I want to pull up the raw data (no filters or groupings). At that point all I can do is return a count of the number of rows, but can’t see any data or query it.
Weirdly enough, converting a question to a native query allows me to see all the raw data. My research did not turn up any similar issues so I’m not sure what’s wrong.
So far I’ve:
Checked to make sure the tables are queryable in settings.
Removed and added back the database.
Let it sit overnight on the off chance it needed time to finish syncing.
Is there a setting somewhere I need to adjust perhaps?
Software versions:
Metabase v0.28.0-snapshot
OS macOS Sierra v.10.12.6
Browser Chrome v. 63.0.3239.84
jontsai
February 18, 2018, 7:34am
2
@esmeralda did you ever figure this out?
I am experiencing the same issue right now. I have a couple of tables where, no matter what, it will say “Every field is hidden right now”
jornh
February 18, 2018, 11:32am
3
In the Admin Panel when you Edit Metadata are either the tables or fields set to ‘hidden’?
jontsai
February 18, 2018, 3:50pm
4
No, neither the tables nor fields are set to Hidden. It still won’t pick it up after I do “Re-scan this table” and “Discard cached field values”
jornh
February 18, 2018, 3:58pm
5
Do you see the fields as expected in the admin UI?
Any specific pattern in how these tables differ from the one’s that show up? I’ve only tried MongoDB – a while ago – with a “flat” table, could e.g. be problematic with tables nested structures etc?
edit Oh, and please specify your environment … Metabase & MongoDB versions etc. + any clues in the Metabase log in particular when you do a scan/sync under the database setup page under Admin.
Just worked on this - which is MySQL though, so maybe not related?
jontsai
February 19, 2018, 12:15am
6
Using Metabase 0.28.1, hosted on AWS using Docker (image: metabase/metabase)
DB: MongoDB 3.4.13, hosted on MongoDB Atlas (M10)
From MongoDB Compass and other MongoDB admin UIs, I see the fields. But the tables are showing up as empty in MongoDB.
I am importing data from Shopify to MongoDB, and then using Metabase on top of it.
My tables are:
customer (empty in metabase)
customer_address (works)
order (empty)
product (works)
product_image (works)
product_tag (works)
product_variant (empty)
I will also try to dig into the logs.
jontsai
February 19, 2018, 3:00am
7
Did a sudo docker logs -f metabase
while I did stuff on the app. Got a null pointer exception for some of the tables, which would explain why!
02-19 03:03:41 ERROR sync.util :: Error syncing fields for Table 9 'order': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
"driver.mongo$describe_table_field.invoke(mongo.clj:120)"
"driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
"driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
"driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
"driver.mongo$describe_table.invoke(mongo.clj:159)"
"driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
"sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
"sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
"driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
"util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
"util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
"driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
"driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
"driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
"driver.mongo$sync_in_context.invoke(mongo.clj:68)"
"driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
"sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
"sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
"sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
"sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
"sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
"sync.util$do_sync_operation.invokeStatic(util.clj:142)"
"sync.util$do_sync_operation.invoke(util.clj:139)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
"api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")
02-19 03:03:45 ERROR sync.util :: Error syncing fields for Table 12 'customer': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
"driver.mongo$describe_table_field.invoke(mongo.clj:120)"
"driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
"driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
"driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
"driver.mongo$describe_table.invoke(mongo.clj:159)"
"driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
"sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
"sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
"driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
"util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
"util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
"driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
"driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
"driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
"driver.mongo$sync_in_context.invoke(mongo.clj:68)"
"driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
"sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
"sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
"sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
"sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
"sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
"sync.util$do_sync_operation.invokeStatic(util.clj:142)"
"sync.util$do_sync_operation.invoke(util.clj:139)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
"api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")
02-19 03:03:46 ERROR sync.util :: Error syncing fields for Table 13 'product_variant': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
"driver.mongo$describe_table_field.invoke(mongo.clj:120)"
"driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
"driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
"driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
"driver.mongo$describe_table.invoke(mongo.clj:159)"
"driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
"sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
"sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
"driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
"util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
"util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
"driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
"driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
"driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
"driver.mongo$sync_in_context.invoke(mongo.clj:68)"
"driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
"sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
"sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
"sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
"sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
"sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
"sync.util$do_sync_operation.invokeStatic(util.clj:142)"
"sync.util$do_sync_operation.invoke(util.clj:139)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
"api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")
02-19 03:03:48 ERROR sync.util :: Error syncing fields for Table 14 'customer_address': class java.lang.NullPointerException
("driver.mongo$describe_table_field.invokeStatic(mongo.clj:123)"
"driver.mongo$describe_table_field.invoke(mongo.clj:120)"
"driver.mongo$describe_table$f__50514__auto____51301$iter__51302__51306$fn__51307.invoke(mongo.clj:165)"
"driver.mongo$describe_table$f__50514__auto____51301.invoke(mongo.clj:164)"
"driver.mongo$describe_table.invokeStatic(mongo.clj:160)"
"driver.mongo$describe_table.invoke(mongo.clj:159)"
"driver$fn__25511$G__25367__25520.invoke(driver.clj:51)"
"sync.fetch_metadata$fn__35668$table_metadata__35673$fn__35674.invoke(fetch_metadata.clj:18)"
"sync.fetch_metadata$fn__35668$table_metadata__35673.invoke(fetch_metadata.clj:15)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234$fn__36235.invoke(fields.clj:294)"
"sync.sync_metadata.fields$fn__36229$db_metadata__36234.invoke(fields.clj:290)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269$fn__36270.invoke(fields.clj:307)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266$fn__36269.invoke(fields.clj:306)"
"sync.sync_metadata.fields$fn__36257$sync_fields_for_table_BANG___36266.invoke(fields.clj:301)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300$fn__36301.invoke(fields.clj:318)"
"sync.sync_metadata.fields$fn__36295$sync_fields_BANG___36300.invoke(fields.clj:314)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870$fn__36871.invoke(sync_metadata.clj:28)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"driver.mongo$sync_in_context$f__50514__auto____51205.invoke(mongo.clj:70)"
"driver.mongo.util$_with_mongo_connection$fn__50507.invoke(util.clj:116)"
"util.ssh$with_ssh_tunnel_STAR_.invokeStatic(ssh.clj:106)"
"util.ssh$with_ssh_tunnel_STAR_.invoke(ssh.clj:91)"
"driver.mongo.util$_with_mongo_connection.invokeStatic(util.clj:95)"
"driver.mongo.util$_with_mongo_connection.invoke(util.clj:90)"
"driver.mongo$sync_in_context.invokeStatic(mongo.clj:69)"
"driver.mongo$sync_in_context.invoke(mongo.clj:68)"
"driver$fn__25526$G__25357__25535.invoke(driver.clj:51)"
"sync.util$sync_in_context$fn__34128.invoke(util.clj:115)"
"sync.util$with_db_logging_disabled$fn__34125.invoke(util.clj:106)"
"sync.util$with_start_and_finish_logging$fn__34120.invoke(util.clj:92)"
"sync.util$with_sync_events$fn__34117.invoke(util.clj:75)"
"sync.util$with_duplicate_ops_prevented$fn__34108.invoke(util.clj:54)"
"sync.util$do_sync_operation.invokeStatic(util.clj:142)"
"sync.util$do_sync_operation.invoke(util.clj:139)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869$fn__36870.invoke(sync_metadata.clj:23)"
"sync.sync_metadata$fn__36864$sync_db_metadata_BANG___36869.invoke(sync_metadata.clj:20)"
"api.database$fn__37519$fn__37520$fn__37521.invoke(database.clj:505)")
jontsai
February 19, 2018, 3:19am
8
Hello @jontsai
In my specific case, it happened because some of the data I was working with did not follow the schema structure. It took a lot of combing through to find those differences. I believe I also saw the null pointer exception in some of the other tables that were syncing. If I remember correctly, Metabase sometimes does that when it encounters empty fields. In my case it didn’t seem to cause other problems besides throwing an error in the logs.
I just read the issue you opened on github. On a separate project, when we edited our tables, Metabase did not pick it up right away. We left it overnight and the next morning the new fields were showing up. Just something to watch out for.
I am having the same issue
Same issue
I’m scared. Metabase has been flawless for 2 years now for me.
Jerry
November 17, 2020, 9:25am
13
I have the same problem too. At first,the fields order of my records does not the same. After reorder the fields,the problem of some tables disappear.I guess,the problem is related to the fields order of mongodb.
flamber
November 17, 2020, 10:09am
14
@Jerry Please post “Diagnostic Info” from Admin > Troubleshooting. And check the logs for any errors during the sync process - Admin > Troubleshooting > Logs.
Jerry
November 18, 2020, 2:59am
15
[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:29:29+08:00 ERROR metabase.models.field-values An error occurred when extracting field-values
com.mongodb.MongoCommandException: Command failed with error 237 (CursorKilled): ‘operation was interrupted’ on server 10.0.3.203:27017. The full response is { “operationTime” : { “$timestamp” : { “t” : 1605655766, “i” : 6 } }, “ok” : 0.0, “errmsg” : “operation was interrupted”, “code” : 237, “codeName” : “CursorKilled”, “$clusterTime” : { “clusterTime” : { “$timestamp” : { “t” : 1605655766, “i” : 6 } }, “signature” : { “hash” : { “$binary” : “AAAAAAAAAAAAAAAAAAAAAAAAAAA=”, “$type” : “00” }, “keyId” : { “$numberLong” : “0” } } } }
at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:179)
[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:35:05+08:00 ERROR metabase.driver.mongo Error introspecting collection: employee_operation #error {
:cause Java heap space
:via
[{:type java.lang.OutOfMemoryError
:message Java heap space}]
:trace
[]}
[3265bc6c-8e8b-4412-8dc6-eddf97b87345] 2020-11-18T07:35:33+08:00 ERROR metabase.driver.mongo Error introspecting collection: productlog #error {
:cause Java heap space
:via
[{:type java.lang.OutOfMemoryError
:message Java heap space}]
:trace
[]}
After checked the logs,I found errors above.And the JAVA_OPTS : -Xms1024M -Xmx4096M -Xmn2048M -XX:MetaspaceSize=512M -XX:MaxMetaspaceSize=1024M -Duser.timezone=GMT+08.
{
"browser-info": {
"language": "zh-CN",
"platform": "Win32",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; Tablet PC 2.0; rv:11.0) like Gecko",
"vendor": ""
},
"system-info": {
"file.encoding": "GBK",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_261-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_261",
"java.vm.name": "Java HotSpot(TM) Client VM",
"java.vm.version": "25.261-b12",
"os.name": "Windows Server 2016",
"os.version": "10.0",
"user.language": "en",
"user.timezone": "Asia/Shanghai"
},
"metabase-info": {
"databases": [
"h2",
"mongo",
"postgres",
"sqlserver"
],
"hosting-env": "unknown",
"application-database": "h2",
"application-database-details": {
"database": {
"name": "H2",
"version": "1.4.197 (2018-03-18)"
},
"jdbc-driver": {
"nam
flamber
November 18, 2020, 9:26am
16