LLM Zoomcamp 2026

Homework 5: Monitoring Statistics

Distribution of scores and reported study time for this homework.

Submissions

315

Median total score

6

Average total score

6

Score distribution

All values are points.

Questions score

Min
2
Median
6.0
Max
6
Q1
6.0
Avg
5.8
Q3
6.0

Learning in public score

Min
-
Median
0.0
Max
7
Q1
0.0
Avg
0.7
Q3
1.0

Total score

Min
2
Median
6.0
Max
14
Q1
6.0
Avg
6.5
Q3
7.0

Time distribution

All values are hours reported by students.

Lectures

Min
0.5
Median
4.0
Max
40.0
Q1
3.0
Avg
5.4
Q3
7.0

Homework

Min
0.0
Median
2.3
Max
16.0
Q1
2.0
Avg
3.2
Q3
4.0

Question breakdown

Correctness and answer distribution per question.

1. How many spans does the trace produce?

309 / 315 correct (98.1%)

1 1 1 (0.3%)
2 3 309 (98.1%)
3 5 4 (1.3%)
4 7 1 (0.3%)

2. How many input tokens do we see for the LLM call?

301 / 315 correct (95.6%)

1 700 13 (4.1%)
2 7000 301 (95.6%)
3 70000 1 (0.3%)
4 700000 0 (0.0%)

3. For a typical query, roughly how long does the LLM call take?

295 / 315 correct (93.7%)

1 Under 100ms 10 (3.2%)
2 100-500ms 10 (3.2%)
3 500-2000ms 114 (36.2%)
4 Over 2000ms 181 (57.5%)

4. Which span names appear in the spans table?

307 / 315 correct (97.5%)

1 Only rag 0 (0.0%)
2 rag and llm 2 (0.6%)
3 rag, search, and llm 307 (97.5%)
4 search, llm, and judge 5 (1.6%)

5. Excluding the rag span, which span type takes the most total time?

311 / 315 correct (98.7%)

1 search 3 (1.0%)
2 llm 311 (98.7%)
3 They are all about the same 0 (0.0%)

6. How much do the input tokens vary across 4 runs of the same query?

289 / 315 correct (91.7%)

1 They are identical 289 (91.7%)
2 Within 10% of each other 19 (6.0%)
3 Within 50% of each other 7 (2.2%)
4 They vary more than 50% 0 (0.0%)

Calculated: 4 August 2026, 05:27