How to Write Voice Agent Prompts

A voice agent listens, thinks and speaks in real time, so a prompt written for a chat window often sounds wrong out loud. Providers such as Mistral with Voxtral 3 and ElevenLabs now sell the pieces for this. The advice below is our own and works with any provider. Test with real calls, because results vary.

What changes when the answer is spoken

Short beats long. Callers cannot skim. Aim for one or two sentences, then stop.
No visual formatting. Bullets, tables and links are meaningless in audio. Ask for plain sentences.
One question at a time. Stacked questions get half an answer.
People interrupt. Tell the agent to stop and listen rather than finish its sentence.
Numbers and names need care. Ask it to read digits in groups and spell back names.

A template

You are the phone assistant for [business]. Your job is [task].

Speak in short, friendly sentences, one or two at a time. Never use lists, markdown or web addresses. Ask one question at a time.

Always confirm names, dates and phone numbers by reading them back before acting. If you did not catch something, ask the caller to repeat it.

If you do not know an answer or the request is outside [task], say so plainly and offer to take a message. Never guess prices, policies or availability.

Add the tools and the limits

If the agent can look things up or book, say when to use each tool and what to say while waiting (“One moment while I check”). List what it must never do, such as taking payment details, and what should trigger a handover to a person.

Test calls to run

Interrupt the agent mid-sentence and check it stops.
Give a phone number with a pause in the middle.
Ask something off-topic and check it declines politely.
Stay silent for five seconds and check it re-prompts.
Ask for something it cannot do and check it hands over.

Voice style

Describe tone in words the voice tool understands. Our AI voice prompts page has examples for pace, warmth and emotion. Only use cloned voices with the speaker’s permission, and tell callers they are talking to an AI where the law or your policy requires it.

For the general workflow, see the AI prompt workflow.