LLM Zoomcamp 2026

Homework 3: Orchestration Statistics

Distribution of scores and reported study time for this homework.

Submissions

434

Median total score

6

Average total score

6

Score distribution

All values are points.

Questions score

Min
2
Median
6.0
Max
6
Q1
5.0
Avg
5.5
Q3
6.0

Learning in public score

Min
-
Median
0.0
Max
7
Q1
0.0
Avg
0.6
Q3
1.0

Total score

Min
2
Median
6.0
Max
14
Q1
5.0
Avg
6.1
Q3
6.0

Time distribution

All values are hours reported by students.

Lectures

Min
0.0
Median
3.0
Max
72.0
Q1
2.0
Avg
4.6
Q3
5.0

Homework

Min
0.0
Median
1.5
Max
24.0
Q1
1.0
Avg
2.4
Q3
3.0

Question breakdown

Correctness and answer distribution per question.

1. After trying the same prompt in ChatGPT vs Kestra AI Copilot, what is the primary reason AI Copilot generates better Kestra flows?

428 / 434 correct (98.6%)

1 AI Copilot uses a more powerful model 2 (0.5%)
2 AI Copilot has access to current Kestra plugin documentation 428 (98.6%)
3 AI Copilot uses more tokens 2 (0.5%)
4 AI Copilot has internet access 2 (0.5%)

2. The non-RAG response about Kestra 1.1 features is best described as:

419 / 434 correct (96.5%)

1 Accurate and specific, matching the actual release notes 14 (3.2%)
2 Vague, generic, or fabricated — the model guesses from training data 419 (96.5%)
3 Empty — the model refuses to answer without context 0 (0.0%)
4 Identical to the RAG version 1 (0.2%)

3. What is the approximate output token count for multilingual_agent when running with summary_length = short?

397 / 434 correct (91.5%)

1 5-15 tokens 7 (1.6%)
2 60-100 tokens 397 (91.5%)
3 200-400 tokens 23 (5.3%)
4 500+ tokens 6 (1.4%)

4. With summary_length = long, roughly how many times more output tokens does multilingual_agent use compared to the short summary?

387 / 434 correct (89.2%)

1 About the same (within 20%) 29 (6.7%)
2 2-5x more 387 (89.2%)
3 10-20x more 13 (3.0%)
4 50x more 1 (0.2%)

5. After changing english_brevity to ask for 3 sentences instead of 1, how does the output token count compare to the original 1-sentence version?

329 / 434 correct (75.8%)

1 About the same (within 20%) 94 (21.7%)
2 2-4x more 329 (75.8%)
3 5-10x more 6 (1.4%)
4 10x+ more 2 (0.5%)

6. For production workflows requiring deterministic, repeatable results with strict compliance requirements, which approach is most appropriate?

410 / 434 correct (94.5%)

1 Always use AI agents for maximum flexibility and adaptation 11 (2.5%)
2 Use traditional task-based workflows for predictability and auditability 410 (94.5%)
3 Use only RAG without agents for better performance 9 (2.1%)
4 Use web search tools exclusively to ensure current data 3 (0.7%)

Calculated: 17 July 2026, 11:51