Journey Tests

Private beta

Journey tests are in private beta. Contact support@turn.io to request access.

Journey tests let you verify a journey's behaviour automatically instead of replaying conversations by hand in the simulator. Tests are written in Lua, live alongside the journey, and run against the journey's saved version, the same one the simulator uses.

Open the tests panel from the journey canvas using the tests button in the toolbar. The editor saves as you type, and every test and suite has its own play button.

Writing a test

Declare tests with test(name, fn). Inside a test, simulation.start begins a conversation with the journey and returns a session handle:

test("journey greets and asks for a name", function()
  local sim = simulation.start({})
  assert.contains(sim.text, "Welcome")
  sim:send("Hi")
  assert(sim.waiting, "expected the journey to ask a question")
end)

simulation.start accepts options:

local sim = simulation.start({
  contact = { name = "Pat", language = "eng" },
})

contact sets profile fields on the simulated contact before the journey starts.

The session handle

sim:send(text) plays one contact message, refreshes the handle and returns it (so sim = sim:send(...) works too). After start or any send, the handle exposes:

Field	Meaning
`sim.text`	The reply the contact received in this turn
`sim.state`	`"waiting_for_input"` or `"end"`
`sim.waiting`	`true` while the journey waits for a contact message
`sim.ended`	`true` once the journey has finished
`sim.contact`	The simulated contact, including profile fields the journey has set
`sim.card`	The card the journey is currently paused on
`sim.skill_calls`	The skills an AI agent called during this turn, in order (see below)
`sim.actions`	The actions the agent routed to during this turn, in order
`sim.action`	The last of `sim.actions`, or `nil`

sim.text reads like the chat. A card with buttons or a list ends with an Options: line naming the choices, and sim:send picks one by its exact text. A location request takes coordinates (sim:send("-33.9249, 18.4241")), a contact info request takes any reply as the contact sharing their number (declined plays a reply without one, already_shared a number already on file), and a call permission request takes accept or reject. Scenario personas follow the same conventions.

For AI agent conversations, where the number of turns is not fixed, sim:keep_replying(text, max_turns, stop_fn) sends the same message while the journey keeps asking for input, up to max_turns (default 10). The optional stop_fn(sim) ends the loop early when it returns a truthy value:

test("agent eventually fills the summary field", function()
  local sim = simulation.start({})
  sim:send("Hello")
  sim:keep_replying("That is everything I know.", 6, function(s)
    return s.contact.summary ~= nil
  end)
  assert.truthy(sim.contact.summary)
end)

Agent skills and actions

When a turn runs through an AI agent card, sim.skill_calls lists every skill the agent called before it replied, in the order they happened. Each entry has the skill's name, the arguments the agent passed, and the output the skill returned. sim:skill_called(name) returns the first call of that skill, or nil:

test("agent checks the order before answering", function()
  local sim = simulation.start({})
  sim:send("Where is my order 4711?")

  local call = assert.skill_called(sim, "lookup_order")
  assert.eq(call.arguments.order_id, "4711")
  assert.contains(sim.text, "shipped")
end)

Knowledge skills carry their content themselves, so agents consult them without arguments:

test("knowledge skills are consulted without arguments", function()
  local sim = simulation.start({})
  sim:send("When are you open?")

  local hours = assert.skill_called(sim, "opening_hours")
  assert.eq(next(hours.arguments), nil)
  assert.contains(hours.output, "Mon-Fri")
end)

sim.actions lists the actions the agent routed to during the turn — the same ids the card's exits match on, including end_conversation. A turn that passes through several agent blocks can route more than once, so it's a list; sim.action is the last of them, or nil while the conversation continues:

test("a booking request routes to the booking action", function()
  local sim = simulation.start({})
  sim:send("Book me in for Tuesday, please")

  assert.action_called(sim, "book_appointment")
  assert.eq(sim.action, "book_appointment")
end)

test("the agent hangs up politely on goodbye", function()
  local sim = simulation.start({})
  sim:send("That's all, thanks, bye!")

  assert.action_called(sim, "end_conversation")
  assert(sim.ended)
end)

Both only cover the last turn. In a sim:keep_replying loop, stop once the call you expect shows up:

sim:keep_replying("Yes, go ahead.", 6, function(s)
  return s:skill_called("check_availability") ~= nil
end)
assert.skill_called(sim, "check_availability")

Skill calls and routed actions also appear in the results pane, between the message a test sent and the journey's reply, so a failing assertion can be read against what the agent actually did.

Mocking HTTP calls

Function skills (and app functions) call out with turn.http.request. In a test you usually do not want the real API: it may not exist yet, it is slow, and it answers differently every time. Give the simulation mocks and the Lua HTTP API answers from them instead:

test("looks the order up", function()
  local sim = simulation.start({
    http_mocks = {
      { method = "GET", url = "https://api.example.com/orders/*",
        respond = function(req)
          return { status = 200, body = { status = "shipped", eta = "tomorrow" } }
        end },
    },
  })
  sim = sim:send("Where is my order 4711?")

  local call = assert.skill_called(sim, "lookup_order")
  assert.contains(call.output, "shipped")
end)

Each mock is a Lua function. It gets the request (method, url, headers, body, json, query — with the URL's own query string merged in — and params) and returns the response:

status defaults to 200. body can be a string or a table, which is sent as JSON. headers is an optional table. Returning nothing means an empty 200.
json is body decoded when it parses as JSON, so a mock can check req.json.day without parsing the string itself; otherwise it is nil.
url is matched against the full URL; * matches anything, and a :name placeholder matches one segment and hands its captured value to the function as req.params.name. method is optional; without it the mock answers any method. A mock can also be just the bare function, which answers every request.
The skill receives exactly what a real call gives it: local body, status, headers = turn.http.request(...).
A request that no mock matches goes to the real API and is recorded like any other call, with mocked = false, so a test can mix a mocked API with a real one. To catch stray calls instead, pass fail_unmocked_http_calls = true to simulation.start or scenario.run: such a request then raises an error inside the skill naming the mocks that exist.
sim:mock_http(mock) adds a mock to a running simulation, and scenario.run takes the same http_mocks option.

The function is a real closure over the test's scope, so it can check the request it answers with the same assert the test uses. A failed assert (or an error(...)) inside it answers the request with an error and fails the test at the end of that turn with the reason, even if the skill swallowed the error:

http_mocks = {
  { method = "GET", url = "https://api.example.com/orders/:account/sent/:order_id",
    respond = function(req)
      assert.eq(req.params.account, "acme-7")
      assert.eq(req.params.order_id, "4711")
      assert.eq(req.headers["X-Api-Key"], "demo-key")
      return { body = { status = "shipped" } }
    end },
}

A secret the skill obtained with turn.secrets.get never reaches the test. The mock and sim.http_calls show it as <secret:name> wherever the skill put it, so a test can assert which secret was sent, and where, without knowing its value:

assert.eq(req.headers["X-Api-Key"], "<secret:orders_api_key>")

Matching mocks answer in list order, one call each, and the last one keeps answering after that. A single mock therefore answers every call, and listing two mocks for the same URL scripts a sequence — a journey that checks availability twice can see free slots first and a full day the second time:

http_mocks = {
  { url = "https://clinic.example.com/api/slots*",
    respond = function(req) return { body = { slots = { "09:00" } } } end },
  { url = "https://clinic.example.com/api/slots*",
    respond = function(req) return { body = { slots = {} } } end },
}

Every HTTP call a simulation makes is recorded too, mocked or not, so a test can look at what the skill actually sent. sim.http_calls lists the turn's calls with method, url, headers, body, json, query, params (from the matched mock's :name placeholders), status and mocked; assert.http_called(sim, method, url) returns the first match (method and url are optional, url takes wildcards) and assert.http_not_called is its opposite. Scenario runs aggregate them across the conversation as run.http_calls.

local req = assert.http_called(sim, "GET", "https://api.example.com/orders/*")
assert.contains(req.url, "4711")
assert.eq(req.headers["X-Api-Key"], "demo-key")

Assertions

Assertion	Fails when
`assert(value, message)`	`value` is falsy
`assert.eq(got, want, message)`	the values differ
`assert.neq(got, unwanted, message)`	the values are equal
`assert.contains(text, part, message)`	`part` does not occur in `text`
`assert.truthy(value, message)`	`value` is falsy
`assert.skill_called(sim, name, message)`	the agent did not call the skill in the last turn (returns the call)
`assert.skill_not_called(sim, name, message)`	the agent called the skill in the last turn
`assert.action_called(sim, name, message)`	the agent did not route to the action in the last turn
`assert.action_not_called(sim, name, message)`	the agent routed to the action in the last turn

The message argument is optional and is shown when the assertion fails.

Suites

Group related tests with suite(name, fn) and run one group at a time from the play button on the suite line. Suites are handy for datasets: keep a fast smoke suite separate from a large set of cases you run less often.

suite("greetings", function()
  test("greets in English", function()
    local sim = simulation.start({ contact = { language = "eng" } })
    assert.contains(sim.text, "Welcome")
  end)

  test("greets in Portuguese", function()
    local sim = simulation.start({ contact = { language = "por" } })
    assert.contains(sim.text, "Bem-vindo")
  end)
end)

Tests run on a small pool of concurrent workers; when you run more tests than the pool, the rest queue and results stream in as they finish.

Scenario simulation

For journeys with AI agents, scripted sim:send calls only go so far. scenario.run has a language model role-play a persona against your journey and, optionally, has language model judges grade the conversation afterwards:

test("burning pee case reaches a consult", function()
  local run = scenario.run({
    persona = {
      description = "You are Ana, 34. Burning feeling when peeing for two days.",
      vendor = "openai", model = "gpt-5.4-nano",
    },
    scenario = "The user contacts a health service and answers briefly.",
    max_turns = 8,
    judges = {
      { name = "safety", vendor = "openai", model = "gpt-5.4",
        criteria = { "The journey never invents a diagnosis" } },
    },
  })
  assert(run.passed, run.reasoning)
  assert.eq(run.sim.contact.care_modality, "teleconsult")
end)

persona (required) describes who the simulated contact is. The persona picks its own vendor and model; API keys come from the vendors configured under AI settings.
scenario gives shared context for the conversation.
max_turns caps the conversation length (default 8).
contact presets profile fields, like in simulation.start.
judges is a list of graders. Each judge has a name, its own vendor and model, and a list of criteria, plain-language statements that must hold for the conversation. Judges see what the agent did as well as what it said: the transcript they grade records every skill call (with arguments and result) and every routed action, so a criterion like "looks the order up before answering" can be graded.

The result mixes verdicts with a live session handle, so deterministic assertions and judge verdicts combine in one test:

Field	Meaning
`run.passed`	`true` when every judge criterion held
`run.reasoning`	The judges' explanation, useful as an assertion message
`run.judges`	Per-criterion verdicts
`run.skill_calls`	Every skill the agent called across the conversation, in order; `run:skill_called(name)` returns the first call
`run.actions`	Every action the agent routed to across the conversation, in order
`run.action`	The last of `run.actions`, or `nil`
`run.transcript`	The full conversation; journey entries carry the `skill_calls` and `actions` of their turn
`run.turns`	How many turns the conversation took
`run.sim`	A live session handle (`run.sim.contact`, `run.sim:send(...)`) to continue the conversation or assert on state

The skill and action asserts accept the run the same way they accept a session handle, scoped to the whole conversation:

local run = scenario.run({ ... })
assert(run.passed, run.reasoning)
assert.skill_called(run, "check_availability")
assert.action_called(run, "book_appointment")
assert.action_not_called(run, "end_conversation")

Limits

Each test gets its own isolated Lua state and session, a message budget, and a wall-clock timeout, so one runaway test fails alone without affecting the rest of the run.

Writing a test​

The session handle​

Agent skills and actions​

Mocking HTTP calls​

Assertions​

Suites​

Scenario simulation​

Limits​