Changelog
Incremental column support for datasets

Ben Rose
on
In datasets console, you can now set an incremental column (e.g. updated_at
, order_updated_at
) on databases and data warehouses. This dramatically improves refresh efficiency by only processing rows that have changed since the last update.
When paired with a primary key, the refresh process works like this:
- Select rows from the source where
incremental_column
is newer than the last run. - Merge (upsert) those rows into the dataset using
primary_key
.
This makes dataset refreshes faster and more efficient, especially for large tables.