Module 2 (Week 2 - Workflow Orchestration) of Data Engineering Zoomcamp 2024

I have completed the second module (Week 2 - Workflow Orchestration) of Data Engineering Zoomcamp 2024 by DataTalksClub.

This week we used tools/techs:

Docker
Python
Postgres 
Mage 
GCP (IAM & Admin, Cloud Storage, BigQuery) 
Terraform

Hands-On Learning:
- ETL Pipeline for API to Postgres (Extract CSV file, Transform, Load data into Postgres ✅
- ETL Pipeline for API to Google Cloud Storage (GCS) (Extract CSV file, Export to GCS) ✅
- ETL Pipeline for GCS to BigQuery (Extract PARQUET file, Load to BigQuery) ✅
- Deploying to GCP with Terraform ✅
- Doing homework ✅

Thanks to Matt Palmer for the course slides and videos, Alexey Grigorev, Mage and Community.

my hands-on learning:

https://github.com/garjitads/de-zoomcamp-2024-week2