Cohorts chart

Get Help
1

Hello there!

I'm trying to do cohorts chart for retention rate. My SQL is below:

SELECT *
FROM bb_orders.crosstab($$

WITH cohort_clients AS (
SELECT
MIN(DATE_TRUNC('MONTH', created)::date) AS cohort_month,
client_id
FROM
bb_orders.orders
[[WHERE created BETWEEN {{start_date}} AND {{end_date}}]]
GROUP BY
client_id
ORDER BY
cohort_month, client_id
),

client_buys AS (
SELECT
o.client_id,
DATE_PART('MONTH', AGE(DATE_TRUNC('MONTH', created)::date, c.cohort_month)) AS month_number
FROM bb_orders.orders AS o
LEFT JOIN cohort_clients c ON c.client_id = o.client_id
[[WHERE created BETWEEN {{start_date}} AND {{end_date}}]]
GROUP BY
o.client_id, month_number
),

cohort_size AS (
SELECT
cohort_month, COUNT(cohort_month) AS total_users
FROM
cohort_clients
GROUP BY
cohort_month
ORDER BY
cohort_month
),

monthly_users AS (
SELECT
c.cohort_month,
b.month_number,
COUNT(cohort_month) AS num_users
FROM client_buys AS b
LEFT JOIN cohort_clients AS c ON b.client_id = c.client_id
GROUP BY
c.cohort_month, b.month_number
)

SELECT
m.cohort_month::date,
m.month_number::int,
ROUND((m.num_users::numeric * 100 / s.total_users), 2) AS retention_rate
FROM monthly_users AS m
LEFT JOIN cohort_size AS s ON m.cohort_month = s.cohort_month
WHERE
m.cohort_month IS NOT NULL
ORDER BY
m.cohort_month::date, m.month_number::int

$$) AS ct ("Cohort" date, "1" decimal, "2" decimal, "3" decimal, "4" decimal,
"5" decimal, "6" decimal, "7" decimal, "8" decimal, "9" decimal, "10" decimal, "11" decimal, "12" decimal);

Without date filter it works perfectly fine and without crosstab function too, but filtering returns with error: "The column index is out of range: 1, number of columns: 0"

How I can fix this? Or may be there is other way to pivot columns?

2

Hi @beyondfire94
What happens if you replace the variables with static dates?
I would check the database query log to see what it receives, which usually also gives better details about the error than what the database usually returns to Metabase.

3

It works with static dates.
I am working on sample DB on my local windows machine, so idk where the logs is. Can you help me with tahis?

4

Ok, I find logs and the problem is that Metabase don't even send this query to server. I can see every statement but not this one. I guess the problem is in $$ symbols, because Metabase use $ symbol for variables and it somehow vreates conflict with $$ in my query

5

@beyondfire94
Post "Diagnostic Info" from Admin > Troubleshooting, and which database type you're querying.

6

This one?

{:database_id 3,
:started_at #t "2021-06-26T10:12:06.577267Z[GMT]",
:state "22023",
:json_query
{:constraints {:max-results 10000, :max-results-bare-rows 2000},
:type :native,
:middleware {:js-int-to-string? true},
:native
{:query
"SELECT *\r\nFROM bb_orders.crosstab($$\r\n\r\nWITH cohort_clients AS (\r\n SELECT\r\n MIN(DATE_TRUNC('MONTH', created)::date) AS cohort_month,\r\n client_id\r\n FROM \r\n bb_orders.orders\r\n[[WHERE created BETWEEN {{start_date}} AND {{end_date}}]]\r\n GROUP BY\r\n client_id\r\n ORDER BY\r\n cohort_month, client_id\r\n),\r\n\r\nclient_buys AS (\r\n SELECT\r\n o.client_id,\r\n DATE_PART('MONTH', AGE(DATE_TRUNC('MONTH', created)::date, c.cohort_month)) AS month_number\r\n FROM bb_orders.orders AS o\r\n LEFT JOIN cohort_clients c ON c.client_id = o.client_id\r\n[[WHERE created BETWEEN {{start_date}} AND {{end_date}}]]\r\n GROUP BY \r\n o.client_id, month_number\r\n),\r\n\r\ncohort_size AS (\r\n SELECT \r\n cohort_month, COUNT(cohort_month) AS total_users\r\n FROM \r\n cohort_clients\r\n GROUP BY \r\n cohort_month\r\n ORDER BY \r\n cohort_month\r\n),\r\n\r\nmonthly_users AS (\r\n SELECT\r\n c.cohort_month,\r\n b.month_number,\r\n COUNT(cohort_month) AS num_users\r\n FROM client_buys AS b\r\n LEFT JOIN cohort_clients AS c ON b.client_id = c.client_id\r\n GROUP BY\r\n c.cohort_month, b.month_number\r\n)\r\n\r\n\r\nSELECT\r\n m.cohort_month::date,\r\n m.month_number::int,\r\n ROUND((m.num_users::numeric * 100 / s.total_users), 2) AS retention_rate\r\n FROM monthly_users AS m\r\n LEFT JOIN cohort_size AS s ON m.cohort_month = s.cohort_month\r\n WHERE\r\n m.cohort_month IS NOT NULL\r\n ORDER BY\r\n m.cohort_month::date, m.month_number::int\r\n \r\n$$) AS ct ("Когорта" date, "1" decimal, "2" decimal, "3" decimal, "4" decimal, \r\n\t\t\t"5" decimal, "6" decimal, "7" decimal, "8" decimal, "9" decimal, "10" decimal, "11" decimal, "12" decimal); ",
:template-tags
{"start_date"
{:id "6f95fcab-b69b-329f-6769-f7918f43935f", :name "start_date", :display-name "Start date", :type :date},
"end_date" {:id "f1e9a029-4bc7-fffc-051a-a4a2c42a84e6", :name "end_date", :display-name "End date", :type :date}}},
:database 3,
:parameters
[{:type "date/single", :value "2019-05-01", :target ["variable" ["template-tag" "start_date"]]}
{:type "date/single", :value "2019-07-01", :target ["variable" ["template-tag" "end_date"]]}],
:async? true,
:cache-ttl nil},
:status :failed,
:class org.postgresql.util.PSQLException,
:stacktrace
["org.postgresql.core.v3.SimpleParameterList.bind(SimpleParameterList.java:69)"
"org.postgresql.core.v3.SimpleParameterList.setStringParameter(SimpleParameterList.java:132)"
"org.postgresql.jdbc.PgPreparedStatement.bindString(PgPreparedStatement.java:1060)"
"org.postgresql.jdbc.PgPreparedStatement.setDate(PgPreparedStatement.java:1437)"
"org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:592)"
"org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:935)"
"com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.setObject(NewProxyPreparedStatement.java:1034)"
"--> driver.sql_jdbc.execute$set_object.invokeStatic(execute.clj:209)"
"driver.sql_jdbc.execute$set_object.invoke(execute.clj:201)"
"driver.sql_jdbc.execute$fn__79655.invokeStatic(execute.clj:217)"
"driver.sql_jdbc.execute$fn__79655.invoke(execute.clj:215)"
"driver.sql_jdbc.execute$set_parameters_BANG_$fn__79671.invoke(execute.clj:257)"
"driver.sql_jdbc.execute$set_parameters_BANG_.invokeStatic(execute.clj:253)"
"driver.sql_jdbc.execute$set_parameters_BANG_.invoke(execute.clj:249)"
"driver.sql_jdbc.execute$fn__79675.invokeStatic(execute.clj:272)"
"driver.sql_jdbc.execute$fn__79675.invoke(execute.clj:260)"
"driver.sql_jdbc.execute$prepared_statement_STAR_.invokeStatic(execute.clj:302)"
"driver.sql_jdbc.execute$prepared_statement_STAR_.invoke(execute.clj:299)"
"driver.sql_jdbc.execute$statement_or_prepared_statement.invokeStatic(execute.clj:326)"
"driver.sql_jdbc.execute$statement_or_prepared_statement.invoke(execute.clj:323)"
"driver.sql_jdbc.execute$execute_reducible_query.invokeStatic(execute.clj:466)"
"driver.sql_jdbc.execute$execute_reducible_query.invoke(execute.clj:453)"
"driver.sql_jdbc.execute$execute_reducible_query.invokeStatic(execute.clj:462)"
"driver.sql_jdbc.execute$execute_reducible_query.invoke(execute.clj:453)"
"driver.sql_jdbc$fn__81350.invokeStatic(sql_jdbc.clj:54)"
"driver.sql_jdbc$fn__81350.invoke(sql_jdbc.clj:52)"
"query_processor.context$executef.invokeStatic(context.clj:59)"
"query_processor.context$executef.invoke(context.clj:48)"
"query_processor.context.default$default_runf.invokeStatic(default.clj:68)"
"query_processor.context.default$default_runf.invoke(default.clj:66)"
"query_processor.context$runf.invokeStatic(context.clj:45)"
"query_processor.context$runf.invoke(context.clj:39)"
"query_processor.reducible$pivot.invokeStatic(reducible.clj:34)"
"query_processor.reducible$pivot.invoke(reducible.clj:31)"
"query_processor.middleware.mbql_to_native$mbql__GT_native$fn__47664.invoke(mbql_to_native.clj:25)"
"query_processor.middleware.check_features$check_features$fn__46778.invoke(check_features.clj:39)"
"query_processor.middleware.limit$limit$fn__47650.invoke(limit.clj:37)"
"query_processor.middleware.cache$maybe_return_cached_results$fn__46230.invoke(cache.clj:211)"
"query_processor.middleware.optimize_temporal_filters$optimize_temporal_filters$fn__47910.invoke(optimize_temporal_filters.clj:204)"
"query_processor.middleware.validate_temporal_bucketing$validate_temporal_bucketing$fn__49840.invoke(validate_temporal_bucketing.clj:50)"
"query_processor.middleware.auto_parse_filter_values$auto_parse_filter_values$fn__45349.invoke(auto_parse_filter_values.clj:43)"
"query_processor.middleware.wrap_value_literals$wrap_value_literals$fn__41497.invoke(wrap_value_literals.clj:161)"
"query_processor.middleware.annotate$add_column_info$fn__41372.invoke(annotate.clj:605)"
"query_processor.middleware.permissions$check_query_permissions$fn__46650.invoke(permissions.clj:81)"
"query_processor.middleware.pre_alias_aggregations$pre_alias_aggregations$fn__48768.invoke(pre_alias_aggregations.clj:40)"
"query_processor.middleware.cumulative_aggregations$handle_cumulative_aggregations$fn__46851.invoke(cumulative_aggregations.clj:60)"
"query_processor.middleware.resolve_joined_fields$resolve_joined_fields$fn__49067.invoke(resolve_joined_fields.clj:102)"
"query_processor.middleware.resolve_joins$resolve_joins$fn__49380.invoke(resolve_joins.clj:171)"
"query_processor.middleware.add_implicit_joins$add_implicit_joins$fn__44925.invoke(add_implicit_joins.clj:190)"
"query_processor.middleware.large_int_id$convert_id_to_string$fn__47614.invoke(large_int_id.clj:59)"
"query_processor.middleware.format_rows$format_rows$fn__47595.invoke(format_rows.clj:74)"
"query_processor.middleware.add_default_temporal_unit$add_default_temporal_unit$fn__44219.invoke(add_default_temporal_unit.clj:23)"
"query_processor.middleware.desugar$desugar$fn__46917.invoke(desugar.clj:21)"
"query_processor.middleware.binning$update_binning_strategy$fn__45736.invoke(binning.clj:227)"
"query_processor.middleware.resolve_fields$resolve_fields$fn__46453.invoke(resolve_fields.clj:34)"
"query_processor.middleware.add_dimension_projections$add_remapping$fn__44574.invoke(add_dimension_projections.clj:312)"
"query_processor.middleware.add_implicit_clauses$add_implicit_clauses$fn__44803.invoke(add_implicit_clauses.clj:147)"
"query_processor.middleware.upgrade_field_literals$upgrade_field_literals$fn__49789.invoke(upgrade_field_literals.clj:40)"
"query_processor.middleware.add_source_metadata$add_source_metadata_for_source_queries$fn__45088.invoke(add_source_metadata.clj:123)"
"query_processor.middleware.reconcile_breakout_and_order_by_bucketing$reconcile_breakout_and_order_by_bucketing$fn__48942.invoke(reconcile_breakout_and_order_by_bucketing.clj:100)"
"query_processor.middleware.auto_bucket_datetimes$auto_bucket_datetimes$fn__45296.invoke(auto_bucket_datetimes.clj:147)"
"query_processor.middleware.resolve_source_table$resolve_source_tables$fn__46500.invoke(resolve_source_table.clj:45)"
"query_processor.middleware.parameters$substitute_parameters$fn__48750.invoke(parameters.clj:111)"
"query_processor.middleware.resolve_referenced$resolve_referenced_card_resources$fn__46552.invoke(resolve_referenced.clj:79)"
"query_processor.middleware.expand_macros$expand_macros$fn__47301.invoke(expand_macros.clj:184)"
"query_processor.middleware.add_timezone_info$add_timezone_info$fn__45097.invoke(add_timezone_info.clj:15)"
"query_processor.middleware.splice_params_in_response$splice_params_in_response$fn__49742.invoke(splice_params_in_response.clj:32)"
"query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48953$fn__48957.invoke(resolve_database_and_driver.clj:31)"
"driver$do_with_driver.invokeStatic(driver.clj:60)"
"driver$do_with_driver.invoke(driver.clj:56)"
"query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48953.invoke(resolve_database_and_driver.clj:25)"
"query_processor.middleware.fetch_source_query$resolve_card_id_source_tables$fn__47541.invoke(fetch_source_query.clj:274)"
"query_processor.middleware.store$initialize_store$fn__49751$fn__49752.invoke(store.clj:11)"
"query_processor.store$do_with_store.invokeStatic(store.clj:44)"
"query_processor.store$do_with_store.invoke(store.clj:38)"
"query_processor.middleware.store$initialize_store$fn__49751.invoke(store.clj:10)"
"query_processor.middleware.validate$validate_query$fn__49796.invoke(validate.clj:10)"
"query_processor.middleware.normalize_query$normalize$fn__47677.invoke(normalize_query.clj:22)"
"query_processor.middleware.add_rows_truncated$add_rows_truncated$fn__44943.invoke(add_rows_truncated.clj:35)"
"query_processor.middleware.results_metadata$record_and_return_metadata_BANG_$fn__49727.invoke(results_metadata.clj:147)"
"query_processor.middleware.constraints$add_default_userland_constraints$fn__46794.invoke(constraints.clj:42)"
"query_processor.middleware.process_userland_query$process_userland_query$fn__48839.invoke(process_userland_query.clj:135)"
"query_processor.middleware.catch_exceptions$catch_exceptions$fn__46734.invoke(catch_exceptions.clj:173)"
"query_processor.reducible$async_qp$qp_STAR___38051$thunk__38052.invoke(reducible.clj:103)"
"query_processor.reducible$async_qp$qp_STAR___38051$fn__38054.invoke(reducible.clj:108)"],
:context :question,
:error "The column index is out of range: 1, number of columns: 0.",
:row_count 0,
:running_time 0,
:data {:rows [], :cols []}}

7

@beyondfire94 No, not the log. "Diagnostic Info" from Admin > Troubleshooting, not Admin > Troubleshooting > Logs.

8

I'm sorry

{
"browser-info": {
"language": "ru-RU",
"platform": "Win32",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36",
"vendor": "Google Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "OpenJDK Runtime Environment",
"java.runtime.version": "11.0.11+9",
"java.vendor": "AdoptOpenJDK",
"java.vendor.url": "https://adoptopenjdk.net/",
"java.version": "11.0.11",
"java.vm.name": "OpenJDK 64-Bit Server VM",
"java.vm.version": "11.0.11+9",
"os.name": "Linux",
"os.version": "5.4.72-microsoft-standard-WSL2",
"user.language": "en",
"user.timezone": "GMT"
},
"metabase-info": {
"databases": [
"postgres"
],
"hosting-env": "unknown",
"application-database": "h2",
"application-database-details": {
"database": {
"name": "H2",
"version": "1.4.197 (2018-03-18)"
},
"jdbc-driver": {
"name": "H2 JDBC Driver",
"version": "1.4.197 (2018-03-18)"
}
},
"run-mode": "prod",
"version": {
"date": "2021-06-15",
"tag": "v0.39.4",
"branch": "release-x.39.x",
"hash": "f538050"
},
"settings": {
"report-timezone": null
}
}
}

9

@beyondfire94 So, yeah, you're getting caught in the query parameter protection handling. One option is to change it to a function on the database, alternatively use single quote around the inner query.

10

@flamber Unfortunately, query don't work with single quotes because of single quotes inside of it (i.e. 'Month'). With double quotes ("month") postgress thinks that "month" is a column. I am in a trap:D
Do you have any other solutions?

11

@beyondfire94 You use two single-quotes to escape a single-quote
'some text with a '' single quite'

12

@flamber thank you! Will try it later. I found out how to pivot my raw data. From one point Metabase just allow me to pivot raw data. IDK why it hasn't been working before

13

@beyondfire94 I have created an issue for this:
https://github.com/metabase/metabase/issues/16949 - upvote by clicking :+1: on the first post
Also see Getting "The column index is out of range" error when adding WHERE clause with Text Variable for a possible alternate workaround.