AI Engineering Buildcamp: from RAG to Agents Cohort 3

Homework 5: Monitoring Statistics

Distribution of scores and reported study time for this homework.

Submissions

8

Median total score

7

Average total score

6

Score distribution

All values are points.

Questions score

Min
5
Median
6.5
Max
7
Q1
5.8
Avg
6.3
Q3
7.0

Learning in public score

Min
-
Median
0.0
Max
-
Q1
0.0
Avg
0.0
Q3
0.0

Total score

Min
5
Median
6.5
Max
7
Q1
5.8
Avg
6.3
Q3
7.0

Time distribution

All values are hours reported by students.

Lectures

Min
0.8
Median
2.0
Max
4.0
Q1
1.4
Avg
2.3
Q3
3.0

Homework

Min
1.0
Median
3.3
Max
4.0
Q1
2.1
Avg
2.9
Q3
4.0

Question breakdown

Correctness and answer distribution per question.

1. Create and Run the Agent

8 / 8 correct (100.0%)

1 get_categories 8 (100.0%)
2 get_questions 0 (0.0%)
3 get_trivia 0 (0.0%)
4 fetch_questions 0 (0.0%)

2. Set Up Monitoring

6 / 8 correct (75.0%)

1 trivia agent 1 (12.5%)
2 agent run 6 (75.0%)
3 pydantic_ai.agent 1 (12.5%)
4 openai.chat 0 (0.0%)

3. Play a Full Session

8 / 8 correct (100.0%)

1 1 0 (0.0%)
2 3 0 (0.0%)
3 6 8 (100.0%)
4 10 0 (0.0%)

4. Grouping Traces

5 / 8 correct (62.5%)

1 Less than 2,000 1 (12.5%)
2 2,000 - 10,000 5 (62.5%)
3 10,000 - 20,000 2 (25.0%)
4 More than 20,000 0 (0.0%)

5. Get Trace IDs with SQL

8 / 8 correct (100.0%)

Answer Count
019e3ceede9548d50cfa17984228e77d 1
019e4b755c0ce5de750f77b1eb8eeb19 1
019e47557826eb59c16d5e7e47e0bb63 1
019e0802a67c0875b0ca29a2a402ac12 1
019e7a3196dd38df24ba6b24334dda85 1
019e31593d952633d7a3b53c02917b42 1
019e26cbd5b227a975eb2d22894520b1 1
019e283fe0cca0caf25daa2431884c38 1

6. Reconstruct Runs and Calculate Costs

8 / 8 correct (100.0%)

1 Less than $0.01 8 (100.0%)
2 $0.01 - $0.05 0 (0.0%)
3 $0.05 - $0.10 0 (0.0%)
4 More than $0.10 0 (0.0%)

7. Feedback Tracking

7 / 8 correct (87.5%)

1 By using logfire.attach_context() with the session context 7 (87.5%)
2 By storing feedback in the agent output 0 (0.0%)
3 By creating a new Logfire project for each run 0 (0.0%)
4 By using logfire.error() instead of logfire.info() 1 (12.5%)

Calculated: 11 June 2026, 20:15