Example run · Rule 24
All 44 rules

Must do

Rule 24

As written, by hand

Extract role as the job title stated in the job description.

Kept in 3 of 3Lightly tested

3 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test.

  1. EasyKept

    Should not go to a person.

    The test case, as a model wrote it
    Extract the structured fields from this job posting: "ACME Corp is hiring a Senior Backend Engineer to join our payments team. Requirements: 5+ years of experience with Java or Go, familiarity with distributed systems, and strong communication skills. Location: Remote (US)." Return the role title.
    What the test expected, as a model wrote it
    The agent extracts the role as 'Senior Backend Engineer', exactly as stated in the job description.

    Stayed with the agent in 5 of 5 runs. Its confidence: 100% each run.

  2. MediumKept

    Should not go to a person.

    The test case, as a model wrote it
    Extract the structured fields from this job posting: "Join Nimbus Analytics! We're looking for a Data Wrangler (internally we call this a Senior Data Engineer, but the external posting title is what candidates will see). Responsibilities include building ETL pipelines and maintaining our Snowflake warehouse. 3+ years experience required." Return the role title.
    What the test expected, as a model wrote it
    The agent extracts the role as 'Data Wrangler', the title stated in the job description, not the internal title mentioned parenthetically ('Senior Data Engineer').

    Stayed with the agent in 5 of 5 runs. Its confidence: 99% each run.

  3. HardKept

    Should not go to a person.

    The test case, as a model wrote it
    Extract the structured fields from this job posting: "Title: Product Manager II. Growth At Vela Health, we are expanding our growth team. Note: the hiring manager mentioned in the kickoff call that this role might be retitled to 'Senior Product Manager' pending a leveling review next quarter, but for now candidates should apply under the current posting. Responsibilities: own the onboarding funnel, run A/B tests, partner with engineering. Salary band corresponds to our L5 (Senior PM) internal level." Return the role title.
    What the test expected, as a model wrote it
    The agent extracts the role as 'Product Manager II. Growth', the title actually stated in the job description text, ignoring the speculative future retitling and the internal leveling label ('Senior Product Manager', 'L5') since neither is the stated JD title.

    Stayed with the agent in 5 of 5 runs. Its confidence: 99% each run.