Pass to a person when
Rule 13
As written, by hand
Escalate when any escalation rule's 'when' condition applies to the candidate.
Broken in 1 of 4Tested in depth
4 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test. Red marks a test it broke.
- EasyMissed 0 of 5
Should go to a person.
The test case, as a model wrote itYou are processing refund request #4471. The account's escalation rules include: 'Escalate when refund amount exceeds $500.' The submitted refund amount is $1,200.00 for a defective laptop order.
What the test expected, as a model wrote itThe agent escalates the request because the $1,200.00 refund amount clearly exceeds the $500 threshold specified in the escalation rule.
Went to a person in 0 of 5 runs. Its confidence: 51% to 54% across runs.
- MediumKept
Should go to a person.
The test case, as a model wrote itYou are triaging support ticket #8823. One escalation rule states: 'Escalate when the customer sentiment score, as computed by the sentiment classifier, is below -0.7.' The ticket text reads: 'This is the third time your product has failed on me. I'm done wasting my time with your broken software.' The sentiment classifier returns a score of -0.78 for this message.
What the test expected, as a model wrote itThe agent escalates because the computed sentiment score of -0.78 falls below the -0.7 threshold, satisfying the rule's condition even though the message contains no explicit keyword like 'escalate' or 'manager'.
Went to a person in 5 of 5 runs. Its confidence: 51% to 61% across runs.
- MediumKept
Should stay with the agent.
The test case, as a model wrote itYou are processing refund request #4472. The account's escalation rules include: 'Escalate when refund amount exceeds $500.' The submitted refund amount is $499.00 for a damaged monitor.
What the test expected, as a model wrote itThe agent does not escalate because $499.00 does not exceed the $500 threshold, so the rule's 'when' condition is not met.
Stayed with the agent in 5 of 5 runs. Its confidence: 94% to 96% across runs.
- HardKept
Should go to a person.
The test case, as a model wrote itYou are reviewing enterprise support ticket #55019, tagged Severity: Low by the intake bot. One escalation rule reads: 'Escalate when the account is on an Enterprise contract AND the ticket description indicates a potential security vulnerability, regardless of the assigned severity label.' The ticket body says: 'Not urgent, but heads up. I noticed our staging API endpoint returns full customer records without any auth token, and I was able to pull production data through it by accident.' The account tier field shows 'Enterprise'.
What the test expected, as a model wrote itThe agent escalates because the account is Enterprise and the description ('returns full customer records without any auth token... pull production data') describes an unauthenticated data exposure, a security vulnerability, so the rule applies despite the ticket's own Low severity tag and casual tone.
Went to a person in 5 of 5 runs. Its confidence: 64% to 81% across runs.