Writing
Articles, reflections, and decisions from my journey into data engineering — including how I think about technical choices and what I've learned along the way.
Showing 1–5 of 7 entries
Building an Airflow + S3 Pipeline That Runs Entirely on Your Laptop How I built a production-style Airflow 3.2.1 data ingestion pipeline with S3, without needing a single AWS credential, thanks to LocalStack and Docker. Read article How the Tableau Metadata API Saved Me from Manual Impact Analysis When upstream changes threatened dozens of dashboards, the Metadata API turned hours of manual tracing into a single query. Read article Parquet and Iceberg: Why I Stopped Treating My Data Lake Like a Folder Columnar file formats and table abstractions turned my messy S3 data lake into something I can actually query and trust. Read article Redshift Taught Me That Storage Format Matters More Than Query Logic Why moving from row-based to columnar storage in Redshift was the single biggest performance leap I've seen. Read article What dbt Actually Does for You dbt didn't just change how I write SQL — it changed how I think about data transformations. Read article