Wonderful Goldstine
Total Score: 105
Homework submissions
Homework 1: Docker, SQL and Terraform
Score: 12 = 5 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.docker.com/products/docker-hub/
- https://docs.docker.com/compose/install/
- https://docs.docker.com/desktop/
- https://developer.hashicorp.com/terraform?product_intent=terraform
- https://developer.hashicorp.com/terraform/cdktf
- https://dev.to/pawnsapprentice/postgresql-vs-neo4j-choosing-the-right-database-for-your-project-1o59
- https://www.docker.com/blog/how-to-use-the-postgres-docker-official-image/
Homework 2: Workflow Orchestration
Score: 13 = 6 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://airflow.apache.org/docs/
- https://airflow.apache.org/use-cases/etl_analytics/
- https://docs.mage.ai/introduction/overview
- https://kestra.io/docs/installation/gcp-vm
- https://kestra.io/docs/installation/docker-compose
- https://www.restack.io/docs/airflow-vs-mage-vs-kestra
- https://medium.com/geekculture/airflow-vs-prefect-vs-kestra-which-is-best-for-building-advanced-data-pipelines-40cfbddf9697
Workshop 1: Ingestion with dlt
Score: 11 = 4 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://dlthub.com/blog/dlt-dbt-semantic-layer
- https://www.element61.be/en/resource/when-use-dbt-or-delta-live-tables
- https://medium.com/dataops-tech/data-load-tool-dlt-write-disposition-strategies-of-data-ingestion-91d563a594bb
- https://dlthub.com/docs/dlt-ecosystem/verified-sources/kafka
- https://medium.com/@Eva_Nkem/data-load-tool-dlt-series-getting-started-with-pipelines-part-1-4063edb3fdca
- https://nagilla748.medium.com/simplifying-data-loading-with-dlt-an-open-source-solution-for-live-datasets-55dd2ed4eb71
- https://www.datacamp.com/tutorial/python-dlt
Homework 3: Data Warehousing
Score: 15 = 8 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.datacamp.com/blog/google-bigquery-vs-snowflake
- https://www.integrate.io/blog/redshift-vs-bigquery-comprehensive-guide/
- https://www.snowflake.com/guides/olap-vs-oltp/
- https://www.geeksforgeeks.org/difference-between-olap-and-oltp-in-dbms/
- https://neo4j.com/docs/spark/current/data-warehouses/bigquery/?msclkid=c56d8efbd9a0151f76ebea164f5643cb&utm_source=GSearch&utm_medium=PaidSearch&utm_campaign=Evergreen&utm_content=APAC-Search-SEMCE-DSA-None-SEM-SEM-NonABM&utm_term=&utm_adgroup=DSA&gad_source=1&gclid=CjwKCAiAzba9BhBhEiwA7glbanOeG4QeGARR9iIRNQ2BBi9MXohlenM2FdVGIAX7Ez_ipGuBTOw5SxoCleAQAvD_BwE
- https://cloud.google.com/bigquery/docs/introduction
- https://medium.com/@marko.zarkovic/big-query-best-practices-47f4aa86de10
Homework 4: Analytics Engineering
Score: 8 = 1 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.getdbt.com/lp/free-account?utm_medium=paid-search&utm_source=google&utm_campaign=q3-2024_apac-brand_cv&utm_content=cloud-account_kw-dbt-ex___&utm_term=all_apac__&cq_cmp=20411355825&utm_term=dbt%20cloud&utm_campaign=q2-2024_us-brand_cv&utm_source=adwords&utm_medium=ppc&hsa_acc=8253637521&hsa_cam=20411355825&hsa_grp=151311330385&hsa_ad=709936119319&hsa_src=g&hsa_tgt=kwd-1238816902544&hsa_kw=dbt%20cloud&hsa_mt=e&hsa_net=adwords&hsa_ver=3&gad_source=1&gclid=CjwKCAiAt4C-BhBcEiwA8Kp0CX15Hj9PnziPX1oKFhnQKHoFxmt2mWaEj2uleO0p6AwvLjevefTUdBoCCzwQAvD_BwE
- https://datacoves.com/post/dbt-alternatives
- https://aws.amazon.com/glue/
- https://sqlmesh.com/
- https://azure.microsoft.com/en-us/products/data-factory/?ef_id=_k_Cj0KCQjwxeyxBhC7ARIsAC7dS3-XiU85-4Hn993PUTMzYsBI7XVjCLF1eo7w1djbOd-Wk5bC0yy5LzUaAjyEEALw_wcB_k_&OCID=AIDcmm5edswduu_SEM__k_Cj0KCQjwxeyxBhC7ARIsAC7dS3-XiU85-4Hn993PUTMzYsBI7XVjCLF1eo7w1djbOd-Wk5bC0yy5LzUaAjyEEALw_wcB_k_&gad_source=1&gclid=Cj0KCQjwxeyxBhC7ARIsAC7dS3-XiU85-4Hn993PUTMzYsBI7XVjCLF1eo7w1djbOd-Wk5bC0yy5LzUaAjyEEALw_wcB
- https://datacoves.com/product
- https://learn.microsoft.com/en-us/fabric/data-warehouse/dimensional-modeling-fact-tables
Homework 5: Batch
Score: 13 = 6 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.upsolver.com/blog/spark-alternatives-use-case
- https://www.databricks.com/spark/getting-started-with-apache-spark
- https://www.w3schools.com/sql/sql_join.asp
- https://medium.com/@techsuneel99/20-advanced-sql-techniques-with-practical-examples-b47490d9896d
- https://medium.com/@rganesh0203/most-important-spark-sql-concepts-as-a-data-engineer-must-practice-ff6431b11d84
- https://medium.com/@amitjoshi7/spark-architecture-a-deep-dive-2480ef45f0be
- https://medium.com/qodea/bigquery-spark-or-dataflow-a-story-of-speed-and-other-comparisons-fb1b8fea3619
Project submissions
Project Attempt 2
Project score: 24 Passed
Score: 33 = 24 (project) + 9 (peer review) + 0 (learning in public / project) + 0 (learning in public / peer review) + 0 (FAQ)
Project URL: View project