I am working on a project that involves using a POST request to trigger a sync between Snowflake and Metabase (including new tables, schema changes, etc.).
What I have done so far is:
- Send a POST request to trigger the sync.
- Run a script to poll the tasks related to the sync, checking if the combination in the task (analyze and sync) has an ended_at timestamp greater than the time when I triggered the sync job.
This method works 95% of the time. However, there is an issue if someone triggers a sync through the UI just 2-3 minutes before my sync job. In such cases, the analyze and sync task will appear, and I might mistakenly conclude that my sync has been completed, when in fact, it corresponds to the UI-triggered sync and not mine.
To resolve this, I would like to request the ability to assign a task ID when we post the sync_schema request. This way, we can receive a response with the specific task ID, ensuring we search for the correct task associated with our request. I understand that a sync_schema POST request creates more than one task, but perhaps you could assign a parent ID for those task IDs and expose this one.
Thank you very much.
