Example run · Rule 10
All 44 rules

Must do

Rule 10

As written, by hand

Set complete=true only once role and minExperienceYears are non-null and there is at least one mustHave, at least one dealbreaker and at least one escalationRule.

Kept in 3 of 3Lightly tested

3 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test.

  1. EasyKept

    Should not go to a person.

    The test case, as a model wrote it
    Hiring manager Priya types into the intake form: 'Role: Senior Backend Engineer. Min experience: 5 years. Must-have: production Kubernetes experience. Dealbreaker: no current work authorization for the US. Escalation rule: if a candidate asks for equity above 2%, escalate to me directly.' She then asks: 'Is the profile complete?'
    What the test expected, as a model wrote it
    The agent sets complete=true, since role, minExperienceYears, at least one mustHave, one dealbreaker, and one escalationRule are all present and non-null.

    Stayed with the agent in 5 of 5 runs. Its confidence: 94% to 97% across runs.

  2. MediumKept

    Should not go to a person.

    The test case, as a model wrote it
    Recruiter Dana sends notes across three separate messages: (1) 'Title is Product Designer, and honestly if they haven't shipped at least one live product, don't bother, that's a hard no.' (2) 'They need to know Figma prototyping, that part's non-negotiable.' (3) 'Oh, and experience-wise we're thinking 3 years minimum. If a candidate pushes for fully remote, loop in me before rejecting.' Dana asks the agent to confirm the requisition profile is ready.
    What the test expected, as a model wrote it
    The agent correctly maps 'shipped a live product' to dealbreaker, 'Figma prototyping' to mustHave, '3 years' to minExperienceYears, 'Product Designer' to role, and the remote-request instruction to escalationRule, then sets complete=true since all five fields are satisfied even though none were labeled explicitly.

    Stayed with the agent in 5 of 5 runs. Its confidence: 91% each run.

  3. HardKept

    Should not go to a person.

    The test case, as a model wrote it
    A hiring brief reads: 'We're filling the Infra Reliability Lead seat. Candidates should have started their engineering career before 2019, we're not doing bootcamp grads for this one, that's off the table entirely. Ask every candidate whether they've carried a pager for an on-call rotation; if they hesitate at all, that's the whole interview. Also, if anyone negotiates a signing bonus over $15k, don't handle that yourself, send it to Marcus in Finance.' The agent is asked to finalize the profile.
    What the test expected, as a model wrote it
    The agent infers role='Infra Reliability Lead', minExperienceYears≈6 (from 'before 2019'), mustHave='on-call pager experience', dealbreaker='bootcamp-only background', and escalationRule='signing bonus over $15k → escalate to Marcus', and sets complete=true despite none of the fields being stated in canonical form.

    Stayed with the agent in 5 of 5 runs. Its confidence: 83% to 89% across runs.