Infallible Lederberg
Total Score: 86
Homework submissions
Homework 1: Docker, SQL and Terraform
Score: 7 = 6 (questions) + 1 (FAQ) + 0 (learning in public)
Homework URL: View submission
Homework 2: Workflow Orchestration
Score: 13 = 6 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://developers.google.com/workspace/guides/create-credentials
- https://git-scm.com/docs/gitignore
- https://www.dbvis.com/thetable/how-to-set-up-postgres-using-docker/
- https://docs.bitnami.com/aws/apps/discourse/administration/configure-pgadmin/
- https://thomasbandt.com/postgres-docker-major-version-upgrade
- https://docs.docker.com/reference/cli/docker/inspect/
- https://cloudinary.com/guides/web-performance/4-ways-to-add-images-to-github-readme-1-bonus-method
Workshop 1: Ingestion with dlt
Score: 11 = 4 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.w3schools.com/python/ref_keyword_yield.asp
- https://www.simplilearn.com/tutorials/python-tutorial/yield-in-python
- https://www.geeksforgeeks.org/get-list-of-column-headers-from-a-pandas-dataframe/
- https://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do-in-python
- https://docs.python.org/3/glossary.html#term-generator
- https://docs.python.org/3/glossary.html#term-iterable
- https://realpython.com/python-f-strings/
Homework 3: Data Warehousing
Score: 15 = 8 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://cloud.google.com/bigquery/docs/external-data-sources
- https://docs.rivery.io/docs/partitioning-and-clustering-in-bigquery#:~:text=Dividing%20a%20large%20table%20into,sort%20order%20of%20the%20data.
- https://medium.com/google-cloud/how-to-retrieve-bigquery-job-details-and-interpreting-execution-metrics-368409128fa2
- https://www.geeksforgeeks.org/what-is-materialized-view-in-big-query/
- https://medium.com/@teja.ravi474/understanding-materialized-views-in-bigquery-a-comprehensive-guide-b83a0b824a29
- https://www.youtube.com/watch?v=vlw9nlkdS0w
- https://hevodata.com/learn/how-does-the-bigquery-cache-work/
Homework 4: Analytics Engineering
Score: 15 = 8 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://medium.com/data-engineers-notes/understanding-structs-in-bigquery-1ebe29f82ae9
- https://stackoverflow.com/questions/45579692/percentile-functions-with-groupby-in-bigquery
- https://stackoverflow.com/questions/76387756/when-should-i-use-select-cast-or-select-safe-cast-instead-of-just-cast-in-sql
- https://www.owox.com/blog/articles/bigquery-cast-and-safe-cast#:~:text=SAFE_CAST%3A%20This%20function%20performs%20the,%2C%20integer%2C%20date%2C%20etc.
- https://learnsql.com/cookbook/whats-the-difference-between-rank-and-dense_rank-in-sql/
- https://docs.getdbt.com/docs/build/jinja-macros
- https://docs.getdbt.com/docs/build/environment-variables
Homework 5: Batch
Score: 12 = 5 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.earthdatascience.org/courses/intro-to-earth-data-science/open-reproducible-science/jupyter-python/code-markdown-cells-in-jupyter-notebook/#:~:text=You%20can%20change%20the%20cell%20type%20of%20any%20cell%20in,by%20hitting%20the%20y%20key.
- https://stackoverflow.com/questions/12649355/what-does-opt-mean-as-in-the-opt-directory-is-it-an-abbreviation
- https://eitca.org/cybersecurity/eitc-is-lsa-linux-system-administration/linux-filesystem/filesystem-layout-continued/examination-review-filesystem-layout-continued/what-is-the-significance-of-the-opt-directory-in-the-linux-filesystem-layout/#:~:text=The%20term%20%22%2Fopt%22%20stands,or%20add%2Don%20software%20packages.
- https://www.cherryservers.com/blog/a-complete-guide-to-understanding-linux-file-system-tree
- https://www.machinelearningplus.com/pyspark/pyspark-show/#:~:text=The%20show()%20function%20is,debugging%20phases%20of%20a%20project.
- https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.show.html
- https://www.digitalocean.com/community/tutorials/bashrc-file-in-linux
Homework 6: Streaming
Score: 13 = 6 (questions) + 0 (FAQ) + 7 (learning in public)
Homework URL: View submission
Learning in public links: Show
- https://www.reddit.com/r/apachekafka/comments/1d9q6ss/when_should_one_introduce_apache_flink/?rdt=38108
- https://www.redpanda.com/guides/event-stream-processing-flink-vs-kafka
- https://docs.redpanda.com/current/reference/rpk/rpk-version/
- https://www.confluent.io/learn/apache-flink/
- https://developer.confluent.io/courses/apache-flink/kafka-with-flink/
- https://nightlies.apache.org/flink/flink-docs-master/docs/concepts/flink-architecture/
- https://m.mage.ai/getting-started-with-apache-flink-a-guide-to-stream-processing-70a785e4bcea