Snowflake default metabase types

When we open the data for a Snowflake view (didn't test for tables) the metabase types are mostly undefined which means that filters aren't appropriate unless we assign a type ourselves. Is there a way to have a more appropriate default type for things like numerics and datetimes? I noticed some varchars are appropriately defaulted to category but that's about it.

Hi @biellls

The Field Type under Admin > Data Model is only used for X-rays since 0.39 - or at least that's the intention, but there are a few edge cases, where Field Type is still needed - like:
https://github.com/metabase/metabase/issues/16177

I don't think I understand what you mean by "filters aren't appropriate".

There's 3 main processes of analysis: sync, fingerprinting and scan.

When the actual database column types are returned during sync, then Metabase can recognize those and group them into types, but if columns are returning unknown types, then Metabase just adds a generic type:
https://github.com/metabase/metabase/blob/master/modules/drivers/snowflake/src/metabase/driver/snowflake.clj#L76

If Metabase can see a column has only few distinct string values, then it's set as "Category".
Otherwise there are a couple of other things done via sync, where it tries to set the type.

The rest are done by a name-based strategy:
https://github.com/metabase/metabase/blob/master/src/metabase/sync/analyze/classifiers/name.clj

Metabase is guesstimating, which isn't perfect and something that we constantly try to improve.

You might be interested in this too:
https://github.com/metabase/metabase/issues/15963 - upvote by clicking :+1: on the first post

Hi @flamber,
We're on v0.39.1. What I mean is that for numerical and date values we only get text filter options like is/startswith etc. instead of more appropriate ones like greater than etc. If we change the Field Type on the Data Model then we get appropriate filters when we try to filter by those fields in questions. The source data types are correct and display correctly in metabase.

@biellls Post the details for one of the fields by going to the URL /api/field/:id - replacing :id with a field id

{"description":null,"database_type":"TIMESTAMPNTZ","semantic_type":null,"table_id":163,"coercion_strategy":null,"table":{"description":null,"entity_type":"entity/UserTable","schema":"NAVISION","db":{"description":null,"features":["full-join","basic-aggregations","standard-deviation-aggregations","expression-aggregations","percentile-aggregations","foreign-keys","right-join","left-join","native-parameters","nested-queries","expressions","set-timezone","regex","case-sensitivity-string-filter-options","binning","inner-join","advanced-math-expressions"],"cache_field_values_schedule":"0 0 7 * * ? *","timezone":"UTC","auto_run_queries":true,"metadata_sync_schedule":"0 45 * * * ? *","name":"Data Warehouse Live","caveats":null,"is_full_sync":true,"updated_at":"2021-06-03T09:11:42.858156Z","details":{"role":"analyst","warehouse":"COMPUTE_WH","additional-options":null,"schema":null,"regionid":"switzerland-north.azure","db":"LIVE_ANALYTICS","password":"**MetabasePass**","account":"eh92106","tunnel-enabled":false,"user":"metabase"},"is_sample":false,"id":5,"is_on_demand":false,"options":null,"engine":"snowflake","refingerprint":null,"created_at":"2021-06-02T18:36:31.841351Z","points_of_interest":null},"show_in_getting_started":false,"name":"G_L Account","caveats":null,"updated_at":"2021-06-03T09:43:18.322698Z","entity_name":null,"active":true,"id":163,"db_id":5,"visibility_type":null,"field_order":"database","display_name":"G L Account","created_at":"2021-06-03T09:42:08.695424Z","points_of_interest":null},"name":"Last Date Modified","fingerprint_version":5,"has_field_values":"none","settings":null,"caveats":null,"fk_target_field_id":null,"dimensions":[],"updated_at":"2021-06-04T10:45:07.824731Z","custom_position":0,"effective_type":"type/*","active":true,"parent_id":null,"id":2114,"last_analyzed":"2021-06-03T09:43:18.362969Z","position":16,"visibility_type":"normal","preview_display":true,"display_name":"Last Date Modified","database_position":16,"name_field":null,"fingerprint":{"global":{"distinct-count":1679,"nil%":0.0}},"created_at":"2021-06-03T09:42:13.604299Z","base_type":"type/DateTime","points_of_interest":null}

@biellls Can you also post the same field, where you have defined Field Type and the field works as expected, since I think I can see what's going on. It's kinda the same problem as 16177, but needs it's own issue.

Also, have you used versions before 0.39? If yes, did you have the same problems there?

{"description":null,"database_type":"TIMESTAMPNTZ","semantic_type":"type/UpdatedTimestamp","table_id":163,"coercion_strategy":null,"table":{"description":null,"entity_type":"entity/UserTable","schema":"NAVISION","db":{"description":null,"features":["full-join","basic-aggregations","standard-deviation-aggregations","expression-aggregations","percentile-aggregations","foreign-keys","right-join","left-join","native-parameters","nested-queries","expressions","set-timezone","regex","case-sensitivity-string-filter-options","binning","inner-join","advanced-math-expressions"],"cache_field_values_schedule":"0 0 7 * * ? *","timezone":"UTC","auto_run_queries":true,"metadata_sync_schedule":"0 45 * * * ? *","name":"Data Warehouse Live","caveats":null,"is_full_sync":true,"updated_at":"2021-06-03T09:11:42.858156Z","details":{"role":"analyst","warehouse":"COMPUTE_WH","additional-options":null,"schema":null,"regionid":"switzerland-north.azure","db":"LIVE_ANALYTICS","password":"**MetabasePass**","account":"eh92106","tunnel-enabled":false,"user":"metabase"},"is_sample":false,"id":5,"is_on_demand":false,"options":null,"engine":"snowflake","refingerprint":null,"created_at":"2021-06-02T18:36:31.841351Z","points_of_interest":null},"show_in_getting_started":false,"name":"G_L Account","caveats":null,"updated_at":"2021-06-03T09:43:18.322698Z","entity_name":null,"active":true,"id":163,"db_id":5,"visibility_type":null,"field_order":"database","display_name":"G L Account","created_at":"2021-06-03T09:42:08.695424Z","points_of_interest":null},"name":"Last Date Modified","fingerprint_version":5,"has_field_values":"none","settings":null,"caveats":null,"fk_target_field_id":null,"dimensions":[],"updated_at":"2021-06-12T09:31:53.032112Z","custom_position":0,"effective_type":"type/DateTime","active":true,"parent_id":null,"id":2114,"last_analyzed":"2021-06-03T09:43:18.362969Z","position":16,"visibility_type":"normal","preview_display":true,"display_name":"Last Date Modified","database_position":16,"name_field":null,"fingerprint":{"global":{"distinct-count":1679,"nil%":0.0}},"created_at":"2021-06-03T09:42:13.604299Z","base_type":"type/DateTime","points_of_interest":null}

Thanks @flamber, no first time user. Do you need me to do the same for a numeric field?

@biellls
I can already see the problem - notice effective_type is null in the first, and defined in the second.

I'm working on some other stuff right now, but I'll swing back to this a little later.
You're essentially seeing https://github.com/metabase/metabase/issues/15913
But it would be very helpful for the developers, when we have exact steps-to-reproduce.

Yes please, the numeric field would be nice to have too, but I'm sure it's the same problem.

Can you post the output of this query on the View - I'm just interested in the two columns (updated and numeric):
DESC TABLE your_view;