Getting Started
assertllm is a pytest-based testing framework for LLM applications. It provides 22+ built-in assertions that are deterministic and instant — no paying an LLM to check if your output contains a word.
Zero LLM calls for most assertions — substring, regex, JSON, cost, latency checks are all local and instant.
Key Features
22+ Deterministic Assertionspytest Native — @llm_test & FixturesMulti-Provider — OpenAI, Anthropic, OllamaAgent Testing — Tools, Loops, OrderingRetry & Statistical TestingCI/CD — JSON, JUnit, GitHub Actions
Architecture
pip install "assertllm[openai]" # or [anthropic], [ollama], [all]- core
- pytest-plugin
The core has zero LLM dependencies. Provider extras are installed separately so you only pull in what you need: pip install "assertllm[openai]"
Last updated on