Changelog
Incremental column support for datasets

Ben Rose
on
In datasets console, you can now set an incremental column (e.g. updated_at, order_updated_at) on databases and data warehouses. This dramatically improves refresh efficiency by only processing rows that have changed since the last update.
When paired with a primary key, the refresh process works like this:
- Select rows from the source where
incremental_columnis newer than the last run. - Merge (upsert) those rows into the dataset using
primary_key.
This makes dataset refreshes faster and more efficient, especially for large tables.