Dataform

Incremental Refresh Techniques for Dataform – GA4

BigQuery, Dataform, GA4, Google Cloud / 2023-10-29

Incremental tables allow maintaining tables based on large source data by only performing small periodic queries. In this post, I’ll share three different methods for performing an incremental refresh. The code examples are all based on the GA4 BigQuery export data.

Run Dataform queries immediately after the GA4 BigQuery export happens

BigQuery, Dataform, GA4, Workflows / 2023-03-28

The daily BigQuery export schedules for GA4 are notoriously random. Sometimes the data is processed in the early-morning hours, sometimes at noon, and sometimes not even during the next day. In this article, I’ll walk you through how to set up a Cloud Workflow that compiles and executes the selected queries in your Dataform project.