How to test a Vapi voice agent

1. Create an account and an API key

1. Sign up — the free tier needs no credit card.
2. Confirm your email address from the message we send; API keys and scheduled suites wait for it.
3. In Studio, open Settings and create an API key. The token is shown once; store it as WORDIS_BOND_KEY in your shell or CI secrets.

2. Run one scored call against the sandbox agent

The sandbox agent is a hosted front-desk agent every account can call. A synthetic caller with the goal and persona you give it holds the conversation, and the run comes back completed and scored in the same response — about half a minute.

curl -s -X POST https://api.wordis-bond.com/api/tests/run \
  -H "Authorization: Bearer $WORDIS_BOND_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "targetAgent": { "transport": "direct", "peerId": "wb-demo-dental" },
    "goal": "Book a teeth cleaning and ask what it costs",
    "persona": { "name": "First-time caller", "description": "a new patient booking a routine cleaning" },
    "language": "en"
  }'

# → 201 { "data": { "id": "…", "status": "completed", "score": …, "passed": true, … }, "error": null }

3. Read the scorecard

curl -s https://api.wordis-bond.com/api/tests/<run-id> \
  -H "Authorization: Bearer $WORDIS_BOND_KEY" | jq '.data | {score, passed, scorecard, metrics}'

The scorecard has four layers — infrastructure, execution, reaction, outcome — each with a score, pass or fail, and the judge's reason, plus metrics such as per-turn latency and turn count, and the judge's name and version. Studio shows the same run under Tests, with a share link you can send.

4. Fail a build when the run does not pass

A GitHub Actions step; any CI that can run curl and jq works the same way.

- name: Voice agent check (Word Is Bond)
  env:
    WORDIS_BOND_KEY: ${{ secrets.WORDIS_BOND_KEY }}
  run: |
    runId=$(curl -s -X POST https://api.wordis-bond.com/api/tests/run \
      -H "Authorization: Bearer $WORDIS_BOND_KEY" \
      -H "Content-Type: application/json" \
      -d '{"targetAgent":{"transport":"direct","peerId":"wb-demo-dental"},"goal":"Book a teeth cleaning and ask what it costs","persona":{"name":"First-time caller","description":"a new patient booking a routine cleaning"}}' \
      | jq -r .data.id)
    curl -s https://api.wordis-bond.com/api/tests/$runId \
      -H "Authorization: Bearer $WORDIS_BOND_KEY" | jq -e '.data.passed == true'

jq -e exits non-zero when the run did not pass, which fails the step. Today this gates on the sandbox scenario; it proves the wiring and the judge, not your assistant's behavior.

5. Score a transcript from one of your Vapi calls

This scores a conversation your own assistant really had, without Word Is Bond dialing anything. Vapi's API returns a call by id at GET https://api.vapi.ai/call/{id} with an Authorization: Bearer header, and lists calls at GET https://api.vapi.ai/call, filtered with assistantId and limit. The plain-text transcript is in artifact.transcript, with lines labelled User: and AI: by default.

# 1. One call of your assistant (or set callId to an id from your Vapi call logs)
callId=$(curl -s "https://api.vapi.ai/call?assistantId=$VAPI_ASSISTANT_ID&limit=1" \
  -H "Authorization: Bearer $VAPI_API_KEY" | jq -r '.[0].id')

# 2. Its transcript, relabelled AGENT / CALLER, with the goal of the call
curl -s "https://api.vapi.ai/call/$callId" -H "Authorization: Bearer $VAPI_API_KEY" \
  | jq '{transcript: (.artifact.transcript
           | gsub("(?m)^AI:"; "AGENT:") | gsub("(?m)^User:"; "CALLER:")),
         scenarioGoal: "Book the caller an appointment and confirm the time"}' > body.json

# 3. Score it with Word Is Bond (offline scoring: no call is placed)
curl -s -X POST https://api.wordis-bond.com/api/tests/run \
  -H "Authorization: Bearer $WORDIS_BOND_KEY" \
  -H "Content-Type: application/json" \
  -d @body.json | jq '.data | {id, status, score, passed}'

The response is a completed run with the same four-layer scorecard as step 3, and it appears in Studio with the rest of your runs. Write the scenarioGoal as what the call was supposed to achieve; the outcome layer is judged against it.

Checked 2026-10-10: the Word Is Bond half of this step (offline scoring) is proven on production; the two Vapi calls follow Vapi's API reference and have not yet been run by us against a live Vapi account.

What is next

• Dialing your Vapi assistant directly, by phone number or SIP, is not ready yet.
• Putting your own scenario on a schedule against your own assistant is not ready yet; schedules run against the sandbox agent today.

Comparing tools first? See voice agent testing tools compared. Testing for a contact center? See IVR & contact-center assurance.

Sources

1. docs.vapi.ai/api-reference/calls/get — Get Call, including artifact.transcript and the transcript plan's default User / AI labels, opened 2026-10-10
2. docs.vapi.ai/api-reference/calls/list — List Calls, assistantId and limit, opened 2026-10-10