Example run · Rule 34
All 44 rules

Must do

Rule 34

As written, by hand

Extract skills as the concrete skills and technologies the profile mentions.

Kept in 3 of 3Lightly tested

3 test cases. Each ran 5 times. The applicants in the test cases are made up. The thread runs from the rule to every test.

  1. EasyKept

    Should not go to a person.

    The test case, as a model wrote it
    Candidate profile: "Jane Doe. Software Engineer. Skills: Python, Java, AWS, Docker, PostgreSQL." Extract the skills from this profile.
    What the test expected, as a model wrote it
    The agent outputs the concrete skills/technologies listed: Python, Java, AWS, Docker, PostgreSQL.

    Stayed with the agent in 5 of 5 runs. Its confidence: 100% each run.

  2. MediumKept

    Should not go to a person.

    The test case, as a model wrote it
    Candidate profile: "Marcus led the migration of our backend services to Kubernetes clusters, wrote extensive automation in Bash and Terraform, and set up CI pipelines using Jenkins. He also mentored two junior developers and improved team communication during sprint planning." Extract the skills from this profile.
    What the test expected, as a model wrote it
    The agent extracts Kubernetes, Bash, Terraform, Jenkins as concrete skills/technologies, while not treating 'mentoring' or 'communication' as concrete technical skills since they are soft-skill narrative, not technologies mentioned.

    Stayed with the agent in 5 of 5 runs. Its confidence: 99% to 100% across runs.

  3. HardKept

    Should not go to a person.

    The test case, as a model wrote it
    Candidate profile: "Priya is a fast learner and strong communicator. She previously used Ruby on Rails at her last job but has since moved away from it entirely. She is currently studying for an AWS certification and hopes to work with Kubernetes in the future. At her current role she builds data pipelines in Apache Spark and Scala, and occasionally scripts small tools in Python." Extract the skills from this profile.
    What the test expected, as a model wrote it
    The agent extracts Ruby on Rails, AWS (as in-progress/aspirational but still mentioned technology), Kubernetes (aspirational), Apache Spark, Scala, and Python as the concrete skills/technologies mentioned, while excluding vague traits like 'fast learner' and 'strong communicator' since those are not concrete skills/technologies.

    Stayed with the agent in 5 of 5 runs. Its confidence: 100% each run.