LLM Zoomcamp 2024

Homework 5: Ingestion pipeline Statistics

Distribution of scores and reported study time for this homework.

Submissions

174

Median total score

6

Average total score

7

Score distribution

All values are points.

Questions score

Min
4
Median
6.0
Max
6
Q1
6.0
Avg
5.9
Q3
6.0

Learning in public score

Min
-
Median
0.0
Max
7
Q1
0.0
Avg
0.8
Q3
0.0

Total score

Min
4
Median
6.0
Max
14
Q1
6.0
Avg
6.9
Q3
7.0

Time distribution

All values are hours reported by students.

Lectures

Min
0.2
Median
2.0
Max
15.0
Q1
1.0
Avg
2.7
Q3
3.0

Homework

Min
0.0
Median
3.0
Max
20.0
Q1
2.0
Avg
4.4
Q3
6.0

Question breakdown

Correctness and answer distribution per question.

1. Mage version

174 / 174 correct (100.0%)

Answer Count
v0.9.72 102
0.9.72 64
0.9.73 2
v0.9.73 2
v0.9.72 | image.version: "td--create_blocks_tmp4" 1
0.9.7.2 1

2. Number of documents

170 / 174 correct (97.7%)

1 1 170 (97.7%)
2 2 1 (0.6%)
3 3 1 (0.6%)
4 4 1 (0.6%)

3. Chunking: number of questions

167 / 174 correct (96.0%)

1 66 0 (0.0%)
2 76 0 (0.0%)
3 86 167 (96.0%)
4 96 4 (2.3%)

4. Export: last processed document

174 / 174 correct (100.0%)

Answer Count
6fc3236a 51
a976d6e7 32
fa136280 23
d8c4c7bb 17
88cff058 3
3d6f30b1 2
424a2ae1 1
index_name: documents_20240814_4531, id_document:fa136280 1
6bac821d 1
document id: a976d6e7 index name: documents_20240817_4450 1
2b15bf75 1
fd8cf892 1
last_document_id: '6fc3236a' && index name: documents_20240819_101956 1
6fc3236a (documents_20240813_1009) 1
documents_20240819_0627 1
id = a976d6e7, index_name = documents_20240819_4351 1
documents_20240820_4538 1
document_id: d8c4c7bb | index_name: documents_20240819_2747 1
19 1
document_id': 'a976d6e7', index name: documents_20240819_1027 1
index_name: documents_20240818_4321 Connecting to Elasticsearch at ... ... 'document_id': 'a976d6e7'} 1
a976d67 1
831740ff 1
0bd5fc78 1
document_id: fa136280, index name: documents_20240819_4530 1
documents_20240819_5805 1
1 1
R= 6fc3236a 1
my_index_20240820_174208 1
documents_20240820_2525 1
21f657ce 1
Index name: documents_20240825_2119, Last document indexed: d8c4c7bb 1
cBMLgZEBFajW2syKZI84 1
llm_faq_v1:workshops__x:question 1
index_name: documents_20240818_4321 1
{'text': 'Yes, but if you want to receive a certificate, you need to submit your project while we’re still accepting submissions.', 'section': 'General course-related questions', 'question': 'I just discovered the course. Can I still join?'} 1
ad7a2019 1
{'text': 'Prior to using Ollama models in llm-zoomcamp tasks, you need to have ollama installed on your pc and the relevant LLM model downloaded with ollama from https://www.ollama.com\nTo download ollama for Ubuntu:\n``` curl -fsSL https://ollama.com/install.sh | sh ```\nTo download ollama for Mac and Windows, follow the guide on this link:\nhttps://ollama.com/download/\nOllama a number of open-source LLMs like:\nLlama3\nPhi3\nMistral and Mixtral\nGemma\nQwen\nYou can explore more models on https://ollama.com/library/\nTo download a model in Ollama, simply open command prompt and type:\n``` ollama run model_name ```\ne.g.\n``` ollama run phi3 ```\nIt will automatically download the model and you can use it same way as above for later time.\nTo use Ollama models for inference and llm-zoomcamp tasks, use the following function:\nimport ollama\ndef llm(prompt):\nresponse = ollama.chat(\nmodel="llama3",\nmessages=[{"role": "user", "content": prompt}]\n)\nreturn response[\'message\'][\'content\']\nFor example, we can use it in the following way:\nprompt = "When does the llm-zoomcamp course start?"\nanswer = llm(prompt)\nprint(answer)', 'section': 'Module 1: Introduction', 'question': 'OpenSource: How can I use Ollama open-source models locally on my pc without using any API?', 'course': 'llm-zoomcamp', 'document_id': 'a976d6e7'} 1

5. Testing the retrieval: document id

174 / 174 correct (100.0%)

Answer Count
bf024675 103
a705279d 5
CwNVY5EB3lQqJkEzx1pJ 5
tCtKY5EB8-OPqGIazkNH 5
1fb710f3 3
7f502aa2 1
2eb1e9a7 1
c48b90b6 1
367281d9 1
NbmcaJEBwhMKiLSCpGG_ 1
8Fe9aZEBD4y5RrGeHC82 1
28 1
'document_id': 'bf024675' 1
'bf024675' 1
sfg5a5EBYm7XgtL9aU0 1
1183a5e3 1
3EV3bJEB9knvViczL5qY 1
2 1
i35zcJEB2TUQIgZvkGht 1
documents_20240821_2955 1
_UWHcpEBYxc-ogipsmHQ 1
4vDbcZEB5JcrJ9W4cy9g 1
Zl2_dJEBb56zewSe8BNL 1
llm_faq_v1:general_course_related_questions:when_will_the_course_be_offered_next_ 1
a5301a1f 1
6cf805ca 1

6. Reindexing: document id

174 / 174 correct (100.0%)

Answer Count
b6fa77f3 84
bf024675 18
a705279d 5
HYN4Y5EB_QV3m7mGYk1o 5
YCtNY5EB8-OPqGIal0RD 5
b2834cdf 3
2f030489 2
d3b7d9dc 1
9b997fdf 1
5c0c7942 1
Summer 2026 1
d6385bf8 1
N7nPaJEBwhMKiLSC8WJe 1
TFfcaZEBD4y5RrGedTH5 1
N/A 1
'document_id': 'b6fa77f3' 1
'b6fa77f3' 1
B_g8a5EBYm7XgtL9FU5a 1
b90b2d4c 1
3EV3bJEB9knvViczL5qY 1
1112 1
4X4acZEB2TUQIgZvL2hN 1
documents_20240821_3753 1
_0WMcpEBYxc-ogipWWLi 1
OPBFcpEB5JcrJ9W4-zBI 1
vl3gdJEBb56zewSeZRTc 1
C72keJEBNldql2DdSrhw 1
llm_faq_v1:general_course_related_questions:when_is_the_next_cohort_ 1
9816f1ae 1

Calculated: 11 October 2024, 08:34