Homework 4: Evaluation

Deadline: 13 July 2026 (Mon), 23:00

Shown in your timezone.

You can preview the questions, but submissions are saved only for logged-in students.

Questions

This form is only for submitting your answers. For the homework instructions, check the instructions.

1. When generating questions for the first 3 lesson pages, what is the average number of input tokens across these 3 calls? (1 point)

140

1400

14000

140000

2. After running text_search for the first ground truth question, what is the filename of the first result? (1 point)

01-agentic-rag/lessons/01-intro.md

01-agentic-rag/lessons/03-rag.md

01-agentic-rag/lessons/13-function-calling.md

01-agentic-rag/lessons/10-rag-next-steps.md

3. After running vector_search for the same question, what is the filename of the first result? (1 point)

01-agentic-rag/lessons/01-intro.md

01-agentic-rag/lessons/03-rag.md

04-evaluation/lessons/11-evaluation-intro.md

04-evaluation/lessons/12-rag-answers.md

4. After evaluating text_search on the ground truth, what is the Hit Rate? (1 point)

0.55

0.66

0.76

0.88

5. After evaluating vector_search on the ground truth, what is the MRR? (1 point)

0.35

0.45

0.55

0.65

6. When evaluating hybrid_search for k values 1, 50, 100, and 200, which k gives the best MRR? (1 point)

100

200