Homework 1: Agentic RAG
- Submissions
- 628
- Completion
- 40.6%
- Median score
- 6
- Lecture time
- 5h
- Homework time
- 3h
- Total time
- 8h
LLM Zoomcamp 2026
Course-level participation, homework, and project statistics.
Median time and score by homework. Hover values on desktop for interquartile ranges.
| Homework | Submissions | Completion | Lecture time | Homework time | Total time | Median score |
|---|---|---|---|---|---|---|
| Homework 1: Agentic RAG | 628 | 40.6% | 5h | 3h | 8h | 6 |
| Homework 2: Vector Search | 505 | 32.6% | 5h | 2h | 7.8h | 6 |
| Homework 3: Orchestration | 434 | 28.1% | 3h | 1.5h | 5h | 6 |
| Homework 4: Evaluation | 374 | 24.2% | 4h | 3h | 8h | 6 |
Ranked by lowest median question score relative to the maximum achievable. Normalizing by question count makes homeworks of different lengths comparable; a lower percentage indicates a harder assignment.
| Rank | Homework | Median score | Score % | Completion |
|---|---|---|---|---|
| 1 | Homework 1: Agentic RAG | 6 / 6 | 100.0% | 40.6% |
| 2 | Homework 2: Vector Search | 6 / 6 | 100.0% | 32.6% |
| 3 | Homework 3: Orchestration | 6 / 6 | 100.0% | 28.1% |
| 4 | Homework 4: Evaluation | 6 / 6 | 100.0% | 24.2% |
Share of submitted answers that were correct, per question. A lower percentage points to a harder or more confusing question. Participation-only questions are excluded.
| Homework | Question | Correct | Answers | % correct |
|---|---|---|---|---|
| Homework 1: Agentic RAG | How many lesson pages | 608 | 628 | 96.8% |
| Indexing and searching | 597 | 628 | 95.1% | |
| RAG | 578 | 628 | 92.0% | |
| Chunking | 598 | 628 | 95.2% | |
| RAG with chunking | 536 | 628 | 85.4% | |
| Turning it into an agent | 552 | 628 | 87.9% | |
| Homework 2: Vector Search | Embedding a query | 492 | 505 | 97.4% |
| Cosine similarity | 484 | 505 | 95.8% | |
| Chunking and search by hand | 484 | 505 | 95.8% | |
| Vector search with minsearch | 477 | 505 | 94.5% | |
| Text search vs vector search | 452 | 505 | 89.5% | |
| Hybrid search | 447 | 505 | 88.5% | |
| Homework 3: Orchestration | After trying the same prompt in ChatGPT vs Kestra AI Copilot, what is the primary reason AI Copilot generates better Kestra flows? | 428 | 434 | 98.6% |
| The non-RAG response about Kestra 1.1 features is best described as: | 419 | 434 | 96.5% | |
| What is the approximate output token count for multilingual_agent when running with summary_length = short? | 397 | 434 | 91.5% | |
| With summary_length = long, roughly how many times more output tokens does multilingual_agent use compared to the short summary? | 387 | 434 | 89.2% | |
| After changing english_brevity to ask for 3 sentences instead of 1, how does the output token count compare to the original 1-sentence version? | 329 | 434 | 75.8% | |
| For production workflows requiring deterministic, repeatable results with strict compliance requirements, which approach is most appropriate? | 410 | 434 | 94.5% | |
| Homework 4: Evaluation | When generating questions for the first 3 lesson pages, what is the average number of input tokens across these 3 calls? | 354 | 374 | 94.7% |
| After running text_search for the first ground truth question, what is the filename of the first result? | 321 | 374 | 85.8% | |
| After running vector_search for the same question, what is the filename of the first result? | 331 | 374 | 88.5% | |
| After evaluating text_search on the ground truth, what is the Hit Rate? | 336 | 374 | 89.8% | |
| After evaluating vector_search on the ground truth, what is the MRR? | 304 | 374 | 81.3% | |
| When evaluating hybrid_search for k values 1, 50, 100, and 200, which k gives the best MRR? | 300 | 374 | 80.2% |
When homework submissions arrive relative to the deadline, across all assignments.
Homework submissions per week, across all assignments. Bars are scaled to the busiest week.