OpenAI

Terminal


pip install "assertllm[openai]"
export OPENAI_API_KEY=sk-...

Usage

test_openai.py


from assertllm import expect, llm_test
 
@llm_test(
    expect.is_not_empty(),
    expect.contains("4"),
    provider="openai",
    model="gpt-4o-mini",
)
def test_openai(llm):
    llm("What is 2+2?")

Output


test_openai.py::test_openai
   "2 + 2 = 4"
  ✓ is_not_empty()
  ✓ contains("4")
  PASSED    [0.3s]

Supported Models

gpt-4o, gpt-4o-mini, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, o3, o4-mini, and custom fine-tuned models.

Structured Output

test_structured.py


from pydantic import BaseModel
 
class MathResult(BaseModel):
    answer: int
    explanation: str
 
@llm_test(
    expect.structured_output(MathResult),
    provider="openai",
    model="gpt-4o-mini",
)
def test_structured(llm):
    llm("What is 2+2? Respond as JSON.", response_format=MathResult)

Tool Calling

test_tools.py


@llm_test(
    expect.tool_called("get_weather"),
    provider="openai",
    model="gpt-4o-mini",
)
def test_tools(llm):
    tools = [{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
            },
        },
    }]
    llm("Weather in Paris?", tools=tools)