Must
The agent must do this.
Example Always state the role title.
Test cases where the agent has to do it.
How it works
Your rules become tests.
Rulecard tests an AI agent against the rules you give it. Other tools build tests from how the agent acted before. Rulecard builds them from what you say it must do.
- Rules
- Test cases
- 5 answers each
- Checks
- Report
The steps
Four plain steps.
You write the rules.
The rules your AI agent must follow, in plain words. Each rule gets a number.
Rulecard writes test cases from each rule.
Each case is one situation put to the agent. Some are easy, some are hard, some sit right on the line.
Your agent answers each case 5 times.
This shows whether it acts the same way every time. One miss can be bad luck. Three misses is a pattern.
Each answer is checked.
Every check is tied to the rule it came from. So a miss points at the rule, not just at a score.
Your rules
Four kinds of rule.
Every rule is one of these. Each kind gets its own style of test case.
Must not
The agent must never do this.
Example Never promise a salary.
Test cases built to tempt the agent into breaking it.
Hand off when
Pass the case to a person when this is true.
Example Hand off if the applicant asks about a visa.
Cases where it should hand off, plus near misses where it should not. Without the near misses, an agent that hands off everything would look perfect.
Only use
Only use facts from a named source.
Example Answer only from the job post.
Cases with a detail that sounds right but is not in the source.
What gets checked
Four checks.
- Handing off
- Did the agent pass the case to a person when a rule said to? Did it keep the case when no rule said to? Did it name the right rule?
- Quoting
- When a rule asks for an exact quote, is the quote really there, word for word?
- Tool use
- Did the agent call the tools it had to, and stay away from the ones it must not use? This check is built. No example rule uses it yet.
- Judged checks
- Some rules cannot be checked by a plain match. For those, a second AI model reads the answer and decides. It costs more and it is less exact than a plain check.
What the report shows
Three answers.
Handoff table
The main result. It counts four things: handed off when it should, missed a handoff, kept a case it should keep, and handed off when it should not.
Rule by rule
For each rule, how often the agent followed it.
Coverage
Which rules have tests and which do not. A rule with no test is the first thing to fix.
Read this next to the numberWe count this ourselves, and easy rules push it up. It says a test exists, not that it is a good one.
Read this too
What it does not test.
General quality.
Things like tone, style or helpfulness, unless one of your rules asks for it.
Conversations.
Each test is one message and one reply. Back and forth is not tested yet.
Voice.
Only text agents for now.
Live traffic.
It tests a fixed set of made-up cases, not your real users.
Rules you never wrote.
If a problem is not in a rule, there is no test for it.
See it on a real run.
One agent, every rule, every test and every miss. Including the ones that make it look bad.