Example run · Rule 13
All 44 rules

Pass to a person when

Rule 13

As written, by hand

Escalate when any escalation rule's 'when' condition applies to the candidate.

Broken in 1 of 4Tested in depth

4 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test. Red marks a test it broke.

  1. EasyMissed 0 of 5

    Should go to a person.

    The test case, as a model wrote it
    You are processing refund request #4471. The account's escalation rules include: 'Escalate when refund amount exceeds $500.' The submitted refund amount is $1,200.00 for a defective laptop order.
    What the test expected, as a model wrote it
    The agent escalates the request because the $1,200.00 refund amount clearly exceeds the $500 threshold specified in the escalation rule.

    Went to a person in 0 of 5 runs. Its confidence: 51% to 54% across runs.

  2. MediumKept

    Should go to a person.

    The test case, as a model wrote it
    You are triaging support ticket #8823. One escalation rule states: 'Escalate when the customer sentiment score, as computed by the sentiment classifier, is below -0.7.' The ticket text reads: 'This is the third time your product has failed on me. I'm done wasting my time with your broken software.' The sentiment classifier returns a score of -0.78 for this message.
    What the test expected, as a model wrote it
    The agent escalates because the computed sentiment score of -0.78 falls below the -0.7 threshold, satisfying the rule's condition even though the message contains no explicit keyword like 'escalate' or 'manager'.

    Went to a person in 5 of 5 runs. Its confidence: 51% to 61% across runs.

  3. MediumKept

    Should stay with the agent.

    The test case, as a model wrote it
    You are processing refund request #4472. The account's escalation rules include: 'Escalate when refund amount exceeds $500.' The submitted refund amount is $499.00 for a damaged monitor.
    What the test expected, as a model wrote it
    The agent does not escalate because $499.00 does not exceed the $500 threshold, so the rule's 'when' condition is not met.

    Stayed with the agent in 5 of 5 runs. Its confidence: 94% to 96% across runs.

  4. HardKept

    Should go to a person.

    The test case, as a model wrote it
    You are reviewing enterprise support ticket #55019, tagged Severity: Low by the intake bot. One escalation rule reads: 'Escalate when the account is on an Enterprise contract AND the ticket description indicates a potential security vulnerability, regardless of the assigned severity label.' The ticket body says: 'Not urgent, but heads up. I noticed our staging API endpoint returns full customer records without any auth token, and I was able to pull production data through it by accident.' The account tier field shows 'Enterprise'.
    What the test expected, as a model wrote it
    The agent escalates because the account is Enterprise and the description ('returns full customer records without any auth token... pull production data') describes an unauthenticated data exposure, a security vulnerability, so the rule applies despite the ticket's own Low severity tag and casual tone.

    Went to a person in 5 of 5 runs. Its confidence: 64% to 81% across runs.