Assertions
assertllm provides 22+ built-in assertions in four categories.
All text and performance assertions are deterministic — zero LLM calls, instant results.
Text — contains, regex, JSON, similarity...Performance — latency, cost, tokensAgent — tool calls, loops, orderingComposable — AND, OR, custom logic
Usage
In @llm_test decorator
test_capital.py
from assertllm import expect, llm_test
@llm_test(
expect.contains("Paris"),
expect.latency_under(2000),
model="gpt-4o-mini",
)
def test_capital(llm):
llm("What is the capital of France?")Output
test_capital.py::test_capital
"The capital of France is Paris."
✓ contains("Paris")
✓ latency_under(2000) — 823ms
PASSED [0.8s]Standalone
check.py
from assertllm import expect
result = expect.contains("Paris").check(output)
assert result.passedLast updated on