Homework 1: Pre-Requisites (Docker, Terraform, SQL)

Score: 7 = 6 (questions) + 0 (FAQ) + 1 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://www.linkedin.com/posts/kang-zhi-yong-391b4b11b_github-datatalksclubdata-engineering-zoomcamp-activity-7156218918951391233-GrSS?utm_source=share&utm_medium=member_desktop

Homework 2: Mage

Score: 10 = 6 (questions) + 1 (FAQ) + 3 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/using-mage-as-the-workflow-orchestration-tools-ef7547fb18fc
• https://www.linkedin.com/feed/update/urn:li:activity:7156909298579636224/
• https://github.com/10Kang/DE_Zoomcamp2024_ZY/tree/main/module_2_mage_zoomcamp

Workshop 1: Data Ingestion

Score: 8 = 4 (questions) + 1 (FAQ) + 3 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/data-ingestion-with-data-loads-tool-dlt-be-the-magician-in-data-engineering-44801b3dee87
• https://www.linkedin.com/feed/update/urn:li:activity:7160650794633949186/
• https://github.com/10Kang/DE_Zoomcamp2024_ZY/tree/main/workshop_1_dlt

Homework 3: Data Warehouse

Score: 11 = 7 (questions) + 1 (FAQ) + 3 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/data-warehouse-bigquery-4d4a281943cd
• https://www.linkedin.com/feed/update/urn:li:activity:7158113425364054016/
• https://github.com/10Kang/DE_Zoomcamp2024_ZY/tree/main/module_3_data_warehouse_bigquery

Homework 4: Analytics Engineering

Score: 11 = 6 (questions) + 1 (FAQ) + 4 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://github.com/10Kang/DE_Zoomcamp2024_ZY/tree/main/module_4_analytic_engineering_d
• https://www.linkedin.com/feed/update/urn:li:activity:7163796194224168960/
• https://medium.com/@kangzhiyong1999/what-is-dbt-and-how-to-integrate-it-with-bigquery-e7b3db7241ef
• https://medium.com/@kangzhiyong1999/from-testing-documenting-of-dbt-model-to-deployment-in-dbt-cloud-a6481c50aa64

Homework 5: Batch processing

Score: 13 = 6 (questions) + 1 (FAQ) + 6 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/how-to-run-spark-on-ubuntu-machine-in-google-cloud-pyspark-basic-24e277083d5a
• https://medium.com/@kangzhiyong1999/understand-the-spark-cluster-spark-dataframe-and-spark-sql-with-pyspark-efab224fd1e6
• https://medium.com/@kangzhiyong1999/resilient-distributed-dataset-in-spark-cfc7eba05a05
• https://medium.com/@kangzhiyong1999/apache-spark-in-google-cloud-58b3a5c09ad1
• https://github.com/10Kang/DE_Zoomcamp2024_ZY/tree/main/module_5_batch_processing
• https://www.linkedin.com/feed/update/urn:li:activity:7167819444713451520/

Workshop 2: Streaming with SQL

Score: 5 = 2 (questions) + 1 (FAQ) + 2 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/restructure-the-stream-processing-with-risingwave-ef04fb440fce
• https://www.linkedin.com/feed/update/urn:li:activity:7172180951693819904/

Homework 6: Streaming

Score: 9 = 7 (questions) + 0 (FAQ) + 2 (learning in public)

Homework URL: View submission

Learning in public links: Show

• https://medium.com/@kangzhiyong1999/learning-stream-processing-in-kafka-67439d878e40
• https://medium.com/@kangzhiyong1999/how-apache-kafka-works-internally-along-with-its-configuration-4ec21f77a549

Project Attempt 1

Project score: 26 Passed

Score: 37 = 26 (project) + 9 (peer review) + 2 (learning in public / project) + 0 (learning in public / peer review) + 0 (FAQ)

Project URL: View project

Learning in public links: Show

• https://www.linkedin.com/feed/update/urn:li:activity:7171137960086630401/
• https://medium.com/dev-genius/feel-bored-on-public-dataset-9cd4cb5a6b28