Homework 6: Evaluations

Deadline: 25 May 2026 (Mon), 23:59
Shown in your timezone.
View public statistics
Correct answers are available below

This homework has already been scored. Log in to view your own submission, score, and feedback.

Log in to view my results

Correct answers

1. Did the agent hallucinate on the almond milk substitution question? (1 point)

2. Approximate cost of the almond milk substitution scenario (1 point)

3. How many scenarios did the agent get wrong? (1 point)

4. How many results does the judge label as bad? (1 point)

5. Did you and the judge agree on the almond milk scenario? (1 point)

6. Judge's reasoning for the almond milk scenario (1 point)

7. Number of disagreements before and after improving the judge (1 point)

8. Percentage of bad labels from synthetic dataset (bonus) (1 point)