Missing field in filter dropdown

Get Help
1

Hi,

I just added a field to my database.

I can query it in the sql editor, but it does not show up in the 'filter by' drop down for

Really appreciate any suggestions here, I have been trying for a couple of day snow.

Thanks,
Jaz

About our setup:

I am on v0.35.2
The data is coming from panoply
The field shows up in another viz tool (redash) properly

What i tried so far:

I have tried using the redshift drive and postgres drivers
I have tried on the admin database menu

  1. Sync database schema now
  2. Re-scan field values now

On table settings cached field values i tried:
- Re-scan this table
- Discard cached field values

Thanks

2

Hi @jazz78
Please post “Diagnostic Info” from Admin > Troubleshooting.
Make sure that you do a browser refresh - sometimes fields can get caught up in cache.
And then check Admin > Troubleshooting > Logs for any errors during the sync/scan.

3

Please include these details in support requests. Thank you!

{
“browser-info”: {
“language”: “en-US”,
“platform”: “Win32”,
“userAgent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363”,
“vendor”: “”
},
“system-info”: {
“file.encoding”: “UTF-8”,
“java.runtime.name”: “OpenJDK Runtime Environment”,
“java.runtime.version”: “11.0.5+10”,
“java.vendor”: “AdoptOpenJDK”,
“java.vendor.url”: “https://adoptopenjdk.net/”,
“java.version”: “11.0.5”,
“java.vm.name”: “OpenJDK 64-Bit Server VM”,
“java.vm.version”: “11.0.5+10”,
“os.name”: “Linux”,
“os.version”: “4.14.138-89.102.amzn1.x86_64”,
“user.language”: “en”,
“user.timezone”: “GMT”
},
“metabase-info”: {
“databases”: [
“googleanalytics”,
“mysql”,
“postgres”,
“redshift”
],
“hosting-env”: “unknown”,
“application-database”: “h2”,
“application-database-details”: {
“database”: {
“name”: “H2”,
“version”: “1.4.197 (2018-03-18)”
},
“jdbc-driver”: {
“name”: “H2 JDBC Driver”,
“version”: “1.4.197 (2018-03-18)”
}
},
“run-mode”: “prod”,
“version”: {
“date”: “2020-04-10”,
“tag”: “v0.35.2”,
“branch”: “release-0.35.x”,
“hash”: “f3b2857”
},
“settings”: {
“report-timezone”: null
}
}
}

4

Thanks so much @flamber

Apols. I just added the diagnostic info.

I also tried, browser refresh, different browsers, clearing cache. logging out, and back in.

5

To be clear. I meant thanks for helping.

It is still not working

6

@jazz78 Check Admin > Troubleshooting > Logs for any errors during the sync/scan.

7

Hi @flamber

06673b36-2738-4a83-a3c4-fbe6bbedbe37] 2020-05-15T11:40:35-04:00 ERROR metabase.sync.util Error syncing Fields for Table ‘Table 132 ‘public.iwp_operations’’

I found this. this is the table i am having an issue with.

8

@jazz78

I’m guessing this is Postgres?
If you don’t see which column/field it’s failing on, then try starting Metabase with more debugging options.

JAR-file - add this to the java -jar metabase.jar command:

-Dlog4j.configuration=https://gist.github.com/flamber/53823764c9989415b76acdb9ed88bcc1/raw/e1cd731924c4ae6091a495feec54ab38f5a1543a/log4j-debug-sync.properties

Docker - add this environmental variable to the docker run ... command:

-e JAVA_OPTS="-Dlog4j.configuration=https://gist.github.com/flamber/53823764c9989415b76acdb9ed88bcc1/raw/e1cd731924c4ae6091a495feec54ab38f5a1543a/log4j-debug-sync.properties"
9

@flamber

yes it is postgres.

update - i got caught up on something and just came back to try your suggestion. i did one last check, and it is now showing up in the filter.

so strange, i did not change anything. it looks like sequence of events was

  1. new field added
  2. it shows up in sql query (not sure how fast < 24 hours, maybe right away i did not time it)
  3. it is not showing up in the filter at this time
  4. by around 48 or 72 hours mark it just shows up in the filter, without any specific action on my part.

Thanks so much for your advice all the same. I will try it next time, hopefully there is not one, but now I at least have something to try.

Best,
Jaz

I have had trouble in the past with fields.

10

@jazz78

When you change your database schema/structure, then the new columns will take up to 1 hour before they’re synced and up to 24 hours before they are scanned (example containing data in a dropdown) - unless you’ve changed the default sync+scan scheduling:
https://www.metabase.com/docs/latest/administration-guide/01-managing-databases.html#database-sync-and-analysis

But if you make queries in SQL, then that query will show changes instantly (Field Filters will follow the process above).

My guess is that there were more than 100 values with null in your column, which is one of the known issues, but your data changed and thereby made the scan process able to work.
There’s several fixes to the sync+scan process in the upcoming 0.36 release - it’s constantly getting more better and more optimized.

11

@flamber

Got it, very helpful.

My expectation for this particular field is zero Null values, and I just queried the data, and confirmed that. Based on how that field was set up, I can’t think of any scenario where it would have ever had NULL values (prior to sending it to metabase). But… maybe some strange unknown glitch on our db side there was ? Let’s hope that was it. Good to know for next time. I have often had issues with fields showing up in filters, some of the prior cases definitely could have been related to more than 100 null values, so I am glad to be aware of this.

Best

12

Just for reference:
https://github.com/metabase/metabase/issues/6924
https://github.com/metabase/metabase/issues/11700

1 Like