Adding new MongoDB 4.2 collection now not working under Metabase cloud

Existing datasource is MongoDB 4.2 DB.

Since latest Metabase cloud updates can no longer add new tables to existing DB. Existing DB and tables (Mongodb4.2 collections ) are working fine. It just fails to populate the schema of anything new.

Here is the error- seems there is a process using $unionWith that is incompatible with 4.2

{"status":"failed","exception":"class clojure.lang.ExceptionInfo","message":"Error executing query: Command failed with error 40324 (Location40324): 'Unrecognized pipeline stage name: '$unionWith'' on server localhost:35035. The full response is {"operationTime": {"$timestamp": {"t": 1739391001, "i": 1}}, "ok": 0.0, "errms...","stacktrace":["--> driver.mongo.execute$execute_reducible_query$fn__134318.invoke(execute.clj:200)","driver.mongo.execute$execute_reducible_query.invokeStatic(execute.clj:198)","driver.mongo.execute$execute_reducible_query.invoke(execute.clj:180)","driver.mongo$fn__134873$f__129900__auto____134874.invoke(mongo.clj:470)","driver.mongo$fn__134873.invokeStatic(mongo.clj:469)","driver.mongo$fn__134873.invoke(mongo.clj:466)","query_processor.pipeline$STAR_execute_STAR.invokeStatic(pipeline.clj:52)","query_processor.pipeline$STAR_execute_STAR.invoke(pipeline.clj:39)","query_processor.pipeline$STAR_run_STAR.invokeStatic(pipeline.clj:102)","query_processor.pipeline$STAR_run_STAR.invoke(pipeline.clj:95)","query_processor.execute$run.invokeStatic(execute.clj:62)","query_processor.execute$run.invoke(execute.clj:56)","query_processor.middleware.update_used_cards$update_used_cards_BANG_76027__76028$fn__76029.invoke(update_used_cards.clj:60)","query_processor.execute$add_native_form_to_result_metadata$fn__76044.invoke(execute.clj:25)","query_processor.execute$add_preprocessed_query_to_result_metadata_for_userland_query$fn__76050.invoke(execute.clj:36)","query_processor.middleware.cache$maybe_return_cached_results$maybe_return_cached_results_STAR___76000.invoke(cache.clj:246)","query_processor.middleware.permissions$check_query_permissions$fn__72670.invoke(permissions.clj:148)","metabase_enterprise.advanced_permissions.query_processor.middleware.permissions$fn__116356$check_download_permissions__116357$fn__116358.invoke(permissions.clj:99)","query_processor.middleware.enterprise$check_download_permissions_middleware$fn__73391.invoke(enterprise.clj:51)","metabase_enterprise.sandbox.query_processor.middleware.column_level_perms_check$fn__118373$maybe_apply_column_level_perms_check__118374$fn__118375.invoke(column_level_perms_check.clj:38)","query_processor.middleware.enterprise$maybe_apply_column_level_perms_check_middleware$fn__73401.invoke(enterprise.clj:64)","query_processor.execute$execute76077__76078$fn__76079.invoke(execute.clj:94)","query_processor.setup$do_with_qp_setup73945__73946.invokeStatic(setup.clj:225)","query_processor.setup$do_with_qp_setup73945__73946.invoke(setup.clj:216)","query_processor.execute$execute76077__76078.invokeStatic(execute.clj:93)","query_processor.execute$execute76077__76078.invoke(execute.clj:89)","query_processor$process_query_STAR__STAR_.invokeStatic(query_processor.clj:49)","query_processor$process_query_STAR__STAR_.invoke(query_processor.clj:44)","metabase_enterprise.audit_app.query_processor.middleware.handle_audit_queries$fn__91429$handle_audit_app_internal_queries__91430$fn__91431.invoke(handle_audit_queries.clj:145)","query_processor.middleware.enterprise$handle_audit_app_internal_queries_middleware$fn__73429.invoke(enterprise.clj:103)","query_processor.middleware.process_userland_query$process_userland_query_middleware78690__78691$fn__78692.invoke(process_userland_query.clj:204)","query_processor.middleware.catch_exceptions$catch_exceptions78755__78756$fn__78757.invoke(catch_exceptions.clj:125)","query_processor$process_query78796__78797$fn__78798.invoke(query_processor.clj:80)","query_processor.setup$do_with_canceled_chan73940__73941$fn__73942.invoke(setup.clj:189)","query_processor.setup$do_with_database_local_settings73933__73934$fn__73935.invoke(setup.clj:181)","query_processor.setup$do_with_driver73926__73927$fn__73928$fn__73929.invoke(setup.clj:166)","driver$do_with_driver.invokeStatic(driver.clj:106)","driver$do_with_driver.invoke(driver.clj:101)","query_processor.setup$do_with_driver73926__73927$fn__73928.invoke(setup.clj:165)","query_processor.setup$do_with_metadata_provider73917__73918$fn__73919$fn__73922.invoke(setup.clj:151)","query_processor.store$do_with_metadata_provider59878__59879.invokeStatic(store.clj:170)","query_processor.store$do_with_metadata_provider59878__59879.invoke(store.clj:150)","query_processor.store$do_with_metadata_provider59878__59879.invokeStatic(store.clj:159)","query_processor.store$do_with_metadata_provider59878__59879.invoke(store.clj:150)","query_processor.setup$do_with_metadata_provider73917__73918$fn__73919.invoke(setup.clj:150)","query_processor.setup$do_with_resolved_database73907__73908$fn__73909.invoke(setup.clj:128)","query_processor.setup$do_with_qp_setup73945__73946.invokeStatic(setup.clj:232)","query_processor.setup$do_with_qp_setup73945__73946.invoke(setup.clj:216)","query_processor$process_query78796__78797.invokeStatic(query_processor.clj:78)","query_processor$process_query78796__78797.invoke(query_processor.clj:71)"],"ex-data":{"driver":"mongo","native":{"collection":"sys_logs","query":"[{"$sort":{"_id":1}},{"$limit":500},{"$unionWith":{"coll":"sys_logs","pipeline":[{"$sort":{"_id":-1}},{"$limit":500}]}},{"$project":{"path":"$ROOT","kvs":{"$map":{"input":{"$objectToArray":"$$ROOT"},"as":"item","in":{"k":"$$item.k","object":{"$con..."},"type":"invalid-query","toucan2/context-trace":[["execute SQL with class com.mchange.v2.c3p0.impl.NewProxyConnection",{"toucan2.jdbc.query/sql-args":["SELECT * FROM "metabase_table" WHERE (("active" = TRUE) AND ("visibility_type" IS NULL)) AND ("db_id" = ?)",10]}],["resolve connection",{"toucan2.connection/connectable":"class metabase.db.connection.ApplicationDB"}],["resolve connection",{"toucan2.connection/connectable":"default"}],["resolve connection",{"toucan2.connection/connectable":null}],{"toucan2.pipeline/rf":"clojure.core$map$fn__5950$fn__5951@590187f8"},["with compiled query",{"toucan2.pipeline/compiled-query":["SELECT * FROM "metabase_table" WHERE (("active" = TRUE) AND ("visibility_type" IS NULL)) AND ("db_id" = ?)",10]}],["with built query",{"toucan2.pipeline/built-query":{"select":["*"],"from":[["metabase_table"]],"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null,["=","db_id",10]]}}],["with resolved query",{"toucan2.pipeline/resolved-query":{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}}],["with parsed args",{"toucan2.pipeline/query-type":"toucan.query-type/select.instances","toucan2.pipeline/parsed-args":{"kv-args":{"db_id":10},"queryable":{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}}}],["with model",{"toucan2.pipeline/model":"model/Table"}],["with unparsed args",{"toucan2.pipeline/query-type":"toucan.query-type/select.instances","toucan2.pipeline/unparsed-args":["model/Table","db_id",10,{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}]}]]},"original-info":null}

why are you using a deprecated mongo version?

so the answer is that Metabase no longer supports MongoDB versions prior to 4.4...?

Same happens with MySQL < 8 and all other deprecated versions of all databases. We can’t keep supporting old versions simply because we will have to write fallback code and that’s really costly

understood and thats fine. Its that it only seems to be affecting the populating of newly added tables though. All existing tables prior the update are still working fine. So if I added to a previous metabase version, then serialize it over via api to the current version... it would work