CLI

Initialize a project

Create a starter test file:

Terminal


llmtest init

Run LLM tests

Terminal


llmtest run tests/
llmtest run tests/test_agent.py

Check available providers

Terminal


llmtest providers

Output


Provider      Status    Models
──────────────────────────────────────
openai        ✓ ready   gpt-5-mini, gpt-5, gpt-4.1
anthropic     ✓ ready   claude-opus-4-6, claude-sonnet-4-6
ollama        ✗ not running

llmtest providers auto-detects which providers are available based on your environment variables and running services.