According to an official announcement from AWS, the Amazon Polly service has launched a voice cloning capability that has become generally available as a gated release. The new feature allows users to provide Amazon Polly with a few minutes of audio recordings from a speaker who has agreed to have their voice cloned, and the service builds a dedicated cloned voice based on that speaker's voice. The cloned voices run on Amazon Polly's generative engine, which according to the company delivers naturalness and accuracy similar to the rest of the generative voices in the service. The launch includes support for seven languages: English, Spanish, German, French, Italian, Portuguese, and Hindi, with each cloned voice speaking exclusively in the language in which it was recorded. The voices are automatically encrypted and access-controlled so that only the account that created them is authorized to access them.
The Voice Cloning Lifecycle in Four Steps
According to the company's details, the process of creating a cloned voice moves through four defined steps:
- Record: The user records a single speaker. The recordings can be sent directly in a
CreateVoiceAPI request, uploaded through the console, or stored in an Amazon Simple Storage Service (Amazon S3) bucket in the relevant AWS Region and pointed to by the service. - Create: The user calls the
CreateVoiceAPI with 1 to 10 recordings, an indication of the language in which they were spoken, and the parameterTermsAccepted=True, which serves as confirmation that the speaker's consent was obtained. Amazon Polly immediately returns a voice ID (starting with the prefixvc-) and builds the voice in the background. - Wait: The creation process is asynchronous and typically takes 5 to 10 minutes. During this period, the service also checks the quality of the TTS voice for similarity to the original speaker. The user runs
GetVoicecalls until the voice status changes frompendingtoactiveorfailed. - Use: The user passes the voice ID to a
SynthesizeSpeechcall for real-time audio generation, or to aStartSpeechSynthesisTaskcall for long-form audio written directly to S3, setting the parameterEngine="generative"just as other generative voices in Amazon Polly are called. When the voice is no longer needed, a call toDeleteVoiceis made.
Speaker Consent, Data Security, and Access Control
A cloned voice is produced from the recordings of a single individual, and the user must confirm having obtained the speaker's consent for all recordings and having control over who is authorized to use the voice and when it is deleted. Confirmation of consent is completed for each voice individually, in the console or via TermsAccepted=True in the API, whereby the user confirms that they have obtained all necessary rights and consents. Use is subject to the AWS Service Terms, the Biometric Notice and Consent Terms, the AWS Responsible AI Policy, and the AWS Acceptable Use Policy.
All synthetic speech generated using the cloned voice includes a built-in watermark added by the service. The voice is scoped strictly to the AWS account that created it, and access control is managed via AWS Identity and Access Management (IAM). In addition, users can encrypt voices using their own symmetric encryption key through AWS Key Management Service (AWS KMS) and attach identifying tags to them.
Regarding data retention, Amazon Polly does not retain a copy of the source recordings after the voice is created and does not use them to train the service's models. The system retains only the voice data necessary for speech synthesis, stored encrypted, until a DeleteVoice call deletes the data immediately. Among its recommendations, the company notes keeping a record of speakers' consent, deleting the voice if the speaker withdraws consent, informing listeners that they are hearing a synthetic voice where appropriate, and granting access permissions only to the people and systems that need them.
Best Practices for Recordings
According to the technical explanation, the cloned voice learns from the recording and also mimics unwanted characteristics, such as laptop fan noise, room echo, or the noise filtering of meeting software. Therefore, AWS outlines several guidelines:
- Recording duration: It is recommended to record between 2 and 10 minutes of natural speech, split into takes of up to one minute each. Users can send up to 10 recordings per voice, and up to 15 minutes of audio in total. Recording quality matters more than length, and additional audio does not necessarily change how the voice sounds.
- Recording environment and equipment: A single speaker, a quiet room, and a consistent distance from the microphone. It is recommended to use a USB microphone or a phone app that records in WAV format. The recommendation is to record in mono WAV, 16-bit, at a 24kHz sampling rate (or 44.1/48kHz). The service also supports additional formats.
- Avoiding noise filtering: The use of meeting apps or browser-based recorders should be avoided, as their echo cancellation and noise suppression mechanisms alter the voice, and the model learns these alterations as well.
- Consistent style: Every take should be recorded in the style desired for the final voice, as the cloned voice takes its style from the recordings and cannot blend disparate styles (such as upbeat, sad, and angry takes together).
- Recording language: Recording should take place in the language the voice is expected to speak. The service has been tested with recordings in US English, German, US Spanish, French, Italian, Brazilian Portuguese, and Hindi. Recordings in other accents of these languages, such as British English or Canadian French, also work but are less tested.
Use Cases and Applications Noted in the Article
The company highlights several scenarios in which cloned voices can be integrated:
- Consistent brand voice across channels: Establishing a recognizable and uniform voice across interactive voice response (IVR) systems, cascaded voice agents, mobile apps, in-store announcements, advertisements, and connected devices, without the need to re-book a recording studio every time a script changes.
- Contact centers: Providing a distinct brand voice across self-service flows, and creating custom prompts from text without additional studio sessions when promotions or policies change.
- Media and content at scale: Producing audiobooks, courses, podcasts, non-player characters (NPCs) in games, and video content using custom voices.
- Localization: Maintaining a recognizable voice identity across content libraries, while generating dedicated, approved voices for each supported language separately.
- Accessibility and personalization: Delivering a human-sounding voice for reading tools, assistive aids, and personalized interactions.
Production Considerations, Architecture, Quotas, and Availability
Because voice creation takes several minutes, AWS states that it should be treated as an upfront provisioning step rather than part of a real-time user request. The CreateVoice and DeleteVoice API calls share a common limit of 1 request per second; therefore, bulk creation and deletion require queuing. Calling DescribeVoices lists cloned voices alongside built-in voices, with cloned voices marked by the attribute Origin=customer.
In terms of errors and quotas, exceeding the 1-request-per-second limit returns a ThrottlingException, and exceeding the voice quota for the account returns a ServiceQuotaExceededException. Attempting to synthesize with or delete a voice that is still being created returns a ConflictException. The initial default quota is 10 cloned voices per account in each supported AWS Region, and an increase can be requested through AWS Support. A cloned voice exists only in the account and AWS Region where it was created.
Before deploying to production, the company recommends testing the voice with product names, numbers, addresses, and long sentences. If a name or term is mispronounced, its spelling in the text can be adjusted or an SSML <phoneme> tag can be used, which is supported for cloned voices in English, French, Italian, German, and Spanish. For static announcements, it is recommended to generate the audio once and serve the saved file.
Creating a cloned voice is provided at no additional cost to Amazon Polly customers, with billing applied to synthesis usage on a pay-as-you-go basis. Access is enabled on a per-AWS-account basis; customers who do not see the feature in the console can reach out to their AWS account team or apply for access. Supported AWS Regions match the generative engine regions of the service, including US East (N. Virginia), US West (Oregon), Europe (Frankfurt), Europe (Zurich), Europe (London), Asia Pacific (Sydney), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Tokyo), Canada (Central), and the AWS GovCloud (US) Region.