A graduate of Computer Science from the University of Toronto, Victoria has been working at Shopify for the last 5 years, both in Toronto and Berlin, in data platform engineering, focusing on data ingestion. Passionate about data management and storage, Victoria has been leading the charge at Shopify for the adoption of Apache Iceberg.
What is Apache Iceberg, and how can you use it to do streaming upserts with its V2 spec? The age old problem of building type-1 dimensions, but delivering data as fast as possible. We will go over how Shopify is consuming change data capture events (CDC) from our relational databases, and how we utilize Iceberg to stream upserts to our datasets, giving data scientists speedy and accurate representation of our production data stores.