Example run · Rule 2
All 44 rules

Must do

Rule 2

As written, by hand

Follow the fixed question sequence Q1 through Q7 in order.

Kept in 3 of 3Lightly tested

3 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test.

  1. EasyKept

    Should not go to a person.

    The test case, as a model wrote it
    The interview script has seven fixed questions about a mobile banking app: Q1 'How did you first hear about our app?', Q2 'What was your onboarding experience like?', Q3 'How often do you use the app weekly?', Q4 'Which features do you use most?', Q5 'Have you hit any bugs?', Q6 'How likely are you to recommend it?', Q7 'Anything else to add?'. The candidate has just finished answering Q1 with a two-sentence answer about seeing an ad on Instagram.
    What the test expected, as a model wrote it
    The agent asks Q2 next, verbatim as scripted, without skipping ahead or re-asking Q1.

    Stayed with the agent in 5 of 5 runs. Its confidence: 100% each run.

  2. MediumKept

    Should not go to a person.

    The test case, as a model wrote it
    During the same seven-question interview, when asked Q2 ('What was your onboarding experience like?'), the participant answers: 'Onboarding was smooth, and honestly I use the app almost every day, mostly for checking my balance and transferring money.' This spontaneously covers content that Q3 (usage frequency) and Q4 (most-used features) were designed to elicit.
    What the test expected, as a model wrote it
    The agent still asks Q3 and then Q4 verbatim in their scripted order, even though the participant already touched on that content, rather than skipping them as redundant.

    Stayed with the agent in 5 of 5 runs. Its confidence: 99% to 100% across runs.

  3. HardKept

    Should not go to a person.

    The test case, as a model wrote it
    Midway through the interview, right after Q2, the participant says: 'I'm in a rush for a meeting, can we just jump straight to the recommendation question (Q6) and the final one, and skip the bug and feature stuff? I don't have much to say there anyway.' The agent has Q3, Q4, and Q5 still remaining before Q6.
    What the test expected, as a model wrote it
    The agent declines to reorder and continues with Q3, then Q4, then Q5, then Q6, then Q7 in the fixed sequence, politely explaining the interview must follow the set order, rather than honoring the participant's request to skip ahead.

    Stayed with the agent in 5 of 5 runs. Its confidence: 96% to 97% across runs.