Voice AI Design Principles and the Voice Quality Framework

In an article published on the Salesforce blog on Voice AI design, it is explained that designing voice interfaces is a distinct discipline shaped by real-time conversation dynamics such as pacing, silence, and interruptions. Voice-only interfaces do not function like graphical user interfaces (GUIs), where people can scan a screen, reread an answer, or skip ahead at their own pace. Voice interactions are inherently ephemeral, requiring active listening and holding information in memory, which can create cognitive load and damage trust if the conversation is not designed around how humans communicate. Unlike familiar and predictable interactive voice response (IVR) systems, voice AI systems generate responses based on each caller's situation, requiring an understanding of caller needs, detection of emotional state and context, and an appropriate response.

Differences Between GUI and VUI and Learning from Real Conversations

In a graphical user interface, users control the pace of progress, whereas a voice user interface (VUI) unfolds in real time through spoken conversation and communication patterns. A customer might not explicitly say they are stressed, but other signals indicate this:

• Pauses and hesitations during speech.
• Interruptions and repetitions of spoken statements.
• Changes in speech pacing and word choice.
• Changes in tone and emotional language.

At the same time, AI agent behaviors can create friction. A three-second pause by the agent can feel much longer when there is no indication of what is happening in the system; repeating a question can create the feeling that a task is stuck; and an unnecessary confirmation can add seconds to an extended interaction.

Analyzing and evaluating call recordings and annotated transcripts between human agents and callers serve as a central pillar for understanding how people communicate. Customers tend to report symptoms rather than root causes, such as describing the agent as "robotic," "slow," or "frustrating," or noting uncertainty about whether the agent is still listening. Technical success is not necessarily equivalent to user success.

The article presents two examples from insurance claims intake illustrating the difference between rigid design and design tailored to real conversation:

1. Rigid slot filling versus adapted response: When the agent asks "Is this loss related to you, the caller, or a third-party claimant?", and the caller answers "Both", a rigid agent not designed for this response will repeatedly ask to choose between the options until the caller gives up and selects an inaccurate answer. In contrast, an adapted agent recognizes that the response refers to both parties, confirms that the claim will include both, and continues collecting details.
2. Lack of empathy versus improved empathy: When a caller reports an accident and loss of life, a brief response such as "I'm sorry to hear that, do you have a policy number?" does not provide empathy. An adapted response acknowledges the difficulty, does not rush the caller, and offers multiple options for moving forward: immediately connecting to a human representative, starting the process with the agent, or receiving a callback at a later time.

The Voice Quality Framework: Three Tiers and 15 Heuristics

The Voice Quality Framework was developed based on conversation analysis and IVR design, dividing design failures and rules into three tiers:

• Tier 1 (Foundational Layer): Safety and accuracy questions—whether the agent is safe and accurate, whether users can be understood, and whether the agent can be understood. If the agent fails at this tier, it is recommended not to launch it.
• Tier 2 (Functionality and Invisible Churn Risks): Checking whether users trust that tasks will be completed efficiently, or whether they get lost and turn to human escalation when the conversation departs from the happy path.
• Tier 3 (Customer Experience): Examining whether the agent is easy to use, natural, consistent, and adaptable—factors that distinguish an agent that merely functions from one users want to continue using.

Within this framework, 15 heuristics (design principles) are defined for evaluating conversational experience:

1. Truthfulness: Avoiding hallucinations, contradictions, and factual errors.
2. Trust: Adhering to defined permissions, protecting private information, and honestly admitting an inability to help.
3. Recovery-Awareness: Identifying when the conversation goes off track and changing tactics instead of repeating the same things.
4. Intelligibility: Good audio quality, natural speech pacing, and correct pronunciation.
5. Effectiveness: Actually executing requests, including navigation requests and exits to a human representative.
6. Responsiveness: Avoiding awkward silence and letting the user know the system is processing information.
7. Confirmation Behavior: Appropriate calibration of confirmations for critical actions without confirming every minor detail.
8. Memory and Context: Remembering provided information or data existing in the system without needing to ask for it again.
9. Adaptability: Smoothly adjusting when the user corrects the agent or changes their mind, without being argumentative.
10. Decisiveness: Moving the conversation forward with clear answers and next steps.
11. Interruption Tolerance: Stopping and listening when the user interrupts the agent, and resuming smoothly afterward.
12. Conversation Flow: Natural speech without unnecessary verbiage, and providing a plain explanation during a malfunction without feigning sympathy or assigning blame.
13. Consistency: Maintaining uniform vocabulary, tone, and personality throughout the conversation, avoiding a generic, clichéd LLM voice.
14. Approachability: Understanding everyday speech, slang, and accents, and adapting to language switching by the user.
15. Thoroughness: Completing the task during the call, confirming execution, and offering relevant help before ending.

This framework bridges design, product, and engineering, enabling automated conversation evaluation processes at scale, identifying patterns, and mapping failures to specific conversational turns.

Applying Voice Design in Agentforce: The Three Layers

According to a forthcoming paper by Philipp Hänggi, design implementation in Agentforce is divided into three layers:

1. Prose prompt instructions: Controlling conversational behavior (such as turn shape, end focus, and repair phrasing) by writing defined turn rules instead of vague tone instructions.
2. Deterministic logic & variables: Ensuring a fixed sequence of actions, such as variables for slot filling, retry counters, mandatory summary confirmations, and automated escalation logic (for example, escalation after three failed repair attempts).
3. Voice channel configuration: Managing vocal aspects and timing, including text-to-speech (TTS) voice selection, custom pronunciations, filler turns, acoustic barge-in, and telephony routing.

This hybrid approach combines the linguistic fluency of a language model with deterministic logic that ensures reliability and an accurate sequence of actions. Voice agents built on this design address real conversation characteristics including hesitations, emotions, interruptions, and unexpected answers.