Triggered Datasets are dependent datasets that auto-update when the parent dataset is modified.
For example, let's say that a query is run on raw data spanning terabytes of data with a query against it returning millions of records. The results of the query can be stored in the Knowi Elastic Store. Furthermore, any derived queries on the resulting dataset can be set up separately within Knowi so that when the original query is updated, all dependent datasets will also be automatically updated.
Note: In the case of Triggered Datasets, make sure that:
- The first dataset is from the Knowi data warehouse
- The triggered query schedule is based on the first dataset's schedule and will not be triggered by any other dataset. Hence, ensure that the first dataset is the one you want to schedule. To trigger on every Knowi dataset refresh instead, enable Trigger on Each Dataset Refresh (see below).
To set up a triggered query, use the Linked Dataset menu from the query list, define the query against the dataset, and select the "Triggered Query" option.
Skip if Connected Query Running
When enabled, this option ensures that a query will not run if any of its immediate upstream or downstream connected queries are still running. If skipped, the next query run will occur at the next automated execution. By skipping the execution, it prevents potential conflicts and resource contention.
- Where is this option located?
The checkbox is located within the query edit page under the Data Strategy tab. This option is available for both the Triggered Queries and Scheduled Intervals data strategies.
This option is included in Scheduled Intervals because a scheduled query may need to be skipped if one of the datasets it triggers is still running from the previous execution (e.g., a downstream query).
Trigger on Each Dataset Refresh
When enabled, this query runs each time any Knowi data warehouse dataset it references is executed - whether scheduled or triggered manually. If the query references multiple Knowi datasets, each one independently triggers a run.
For example, if this query joins Join A (Knowi), Join B (MySQL), and Join C (Knowi), it will run whenever Join A or Join C is executed. Join B, being a non-Knowi datasource, will not trigger a run.
- Where is this option located?
The checkbox is located within the query edit page under the Data Strategy tab, and is available when Triggered Query is selected as the data strategy.