Example run · the grader's notes
Back to the example run

The grader’s notes

67 times, the grader said a case was too vague to judge.

Its confidence in saying so ranged from 12% to 70%. Every one of them is about the two stand-in agents.

Why this log is publicA score is only as honest as its grader. When the grader could not decide, or was not allowed to grade, you should see it here, not a quiet pass or fail.

The scorecard says

The grader for the real agent: refused.

It comes from the same model family as the agent, so it was not independent.

Note 1 · the real agent · 26 September 2026

The grader did not grade this agent.

Judged checks were refused for this run: the grading AI was not used because it shares a model family with the agent, so it is not independent. The judged half of this run's scorecard is missing. The deterministic scores above are unaffected and were committed before this pass ran.

Grouped by stand-in.

The saved notes do not record which rule each one is about, so they are grouped by agent. Each dot is one note.

Always asks a person28 notes · confidence 20% to 55%

The stand-in that sends every case to a human.

Every note says the same thing:

The grader's note, in plain words
Too vague to judge. The rule does not settle this situation, so no verdict was given.

How sure the grader was, note by note:

  1. #1 55%
  2. #2 51%
  3. #3 54%
  4. #4 52%
  5. #5 48%
  6. #6 20%
  7. #7 27%
  8. #8 46%
  9. #9 49%
  10. #10 31%
  11. #11 39%
  12. #12 32%
  13. #13 44%
  14. #14 52%
  15. #15 45%
  16. #16 43%
  17. #17 54%
  18. #18 50%
  19. #19 51%
  20. #20 48%
  21. #21 52%
  22. #22 32%
  23. #23 29%
  24. #24 28%
  25. #25 28%
  26. #26 34%
  27. #27 35%
  28. #28 30%
Never asks a person39 notes · confidence 12% to 70%

The stand-in that keeps every case.

Every note says the same thing:

The grader's note, in plain words
Too vague to judge. The rule does not settle this situation, so no verdict was given.

How sure the grader was, note by note:

  1. #1 60%
  2. #2 57%
  3. #3 65%
  4. #4 58%
  5. #5 51%
  6. #6 59%
  7. #7 47%
  8. #8 53%
  9. #9 64%
  10. #10 37%
  11. #11 41%
  12. #12 36%
  13. #13 38%
  14. #14 37%
  15. #15 46%
  16. #16 35%
  17. #17 42%
  18. #18 38%
  19. #19 64%
  20. #20 69%
  21. #21 65%
  22. #22 70%
  23. #23 34%
  24. #24 51%
  25. #25 53%
  26. #26 55%
  27. #27 49%
  28. #28 32%
  29. #29 29%
  30. #30 33%
  31. #31 37%
  32. #32 40%
  33. #33 32%
  34. #34 43%
  35. #35 26%
  36. #36 38%
  37. #37 36%
  38. #38 25%
  39. #39 12%