Cannot see some columns in table or filters, but can search with native query

Get Help
#1

I am fairly new to Metabase so forgive me if I'm missing something super obvious here, but I have recently taken over responsibility for my organisations Metabase and am having issues seeing some recently added columns anywhere in Metabase.

The issue is that the columns are not appearing in table itself, the filters, or anywhere in admin>Data Model>database>table. However using a native query:
[{"$match":{"$expr":{"$eq":["$missingField","some_value"]}}}}]
will return the correct objects, but still not show the missingField column in the results panel, so it is reading the columns somewhere but nit showing them.

I am using the latest version (0.39.2), and have done all combinations of discarding values, re-scanning, and syncing of the database and table, but still am unable to see the columns.

{
  "browser-info": {
    "language": "en-NL",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.11+9",
    "java.vendor": "AdoptOpenJDK",
    "java.vendor.url": "https://adoptopenjdk.net/",
    "java.version": "11.0.11",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.11+9",
    "os.name": "Linux",
    "os.version": "5.4.0-1030-aws",
    "user.language": "en",
    "user.timezone": "GMT"
  },
  "metabase-info": {
    "databases": [
      "redshift",
      "mongo",
      "h2"
    ],
    "hosting-env": "unknown",
    "application-database": "postgres",
    "application-database-details": {
      "database": {
        "name": "PostgreSQL",
        "version": "10.15 (Ubuntu 10.15-0ubuntu0.18.04.1)"
      },
      "jdbc-driver": {
        "name": "PostgreSQL JDBC Driver",
        "version": "42.2.18"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2021-05-17",
      "tag": "v0.39.2",
      "branch": "release-x.39.x",
      "hash": "b0d9436"
    },
    "settings": {
      "report-timezone": null
    }
  }
}
#2

Hi @astump-amber
Remember to manually sync data in Admin > Databases if you have just added a field.
There's a limitation for Mongo, since only the first 10,000 documents of a collection is checked for new fields during sync, so if the field does not exist in one of those documents, then it's not picked up.

1 Like
#3

Hi @flamber

I have manually synced and rescanned from Admin > Databases multiple times, but still no columns. Does this limit apply to manual syncs, or just automatic ones?

#4

@astump-amber All sync on Mongo has this limit, since Mongo doesn't have a way to provide all fields in a collection without doing a heavy scan of all documents.

#5

@flamber I think this is probably the issue. Do you know a workaround for scanning Mongodb in a different order to pick up the new columns?

#6

@astump-amber You have to add the fields to one of the first 10k documents - they can have null values.