A Better Persian Voice Interface: Language Is More Than Transcription

A Persian voice interface can post a respectable word error rate and still fail at the moment that matters. The transcript may be mostly correct while the beneficiary name is wrong, a toman amount is interpreted as rials, a colloquial date is normalized to the wrong day, or an English product code is rendered in Persian script. From the user's perspective, these are not small transcription errors. They are failed tasks.

The design problem is larger than automatic speech recognition (ASR). A production voice journey combines audio capture, speech segmentation, language and dialect recognition, transcription, text normalization, intent and entity extraction, policy checks, dialogue repair, and text-to-speech (TTS). Each layer carries Persian-specific choices. A team that merely attaches fa-IR to a multilingual model has localized a configuration value, not a product.

Recent open resources make evaluation more practical, but they also show why one aggregate benchmark is insufficient. Mozilla's Common Voice Persian program provides community-contributed scripted and spontaneous speech. FLEURS offers a multilingual evaluation set with roughly twelve hours per language. The Persian Speech Recognition Benchmark, or PSRB, evaluates diverse conditions and reports that systems which perform well on standard Persian struggle more with regional accents, children's speech, and particular linguistic phenomena. These are useful starting points—not substitutes for the speech distribution of a bank, clinic, retailer, or public service.

Define the Voice Contract Before Choosing a Model

Write down what the interface promises to understand and what it is allowed to do. A narrow appointment-booking assistant has a different risk profile from a banking agent that can submit transfers. For every supported task, define:

• the expected speakers, regions, age groups, channels, and noise conditions;
• the languages and code-switch patterns that are in scope;
• the entities that must be exact, such as names, dates, addresses, amounts, medicine names, or identifiers;
• the acceptable response time and interruption behavior;
• the actions that require confirmation or human review;
• the audio, transcript, and derived-data retention policy.

This contract prevents a common mistake: measuring general dictation while shipping a transactional product. It also makes release decisions possible. “Persian is supported” is not a testable statement. “The assistant can schedule a clinic visit in standard and colloquial Tehran Persian, under the listed acoustic conditions, with exact confirmation of patient name and date” is.

This work should sit beside a broader Persian NLP localization strategy, because speech inherits the same script, terminology, retrieval, and content-governance problems as text.

Build an Evaluation Corpus That Resembles Real Use

Read speech and spontaneous speech behave differently. Scripted corpora are valuable for controlled comparison, but callers pause, restart, self-correct, trail off, speak over prompts, and compress common phrases. Domain vocabulary changes the distribution again. A logistics caller says street names and tracking codes; a clinician says medicine names and measurements; a support caller mixes a Persian sentence with an English model name.

Construct a consented, versioned evaluation suite with independent slices. At minimum, include formal and colloquial registers, several regions, different ages and voices, quiet and noisy channels, handset and speakerphone audio, short and long utterances, code-switching, and interrupted speech. Do not infer sensitive demographics from voice. Collect only the metadata needed for evaluation, with explicit consent and a documented deletion path.

Open data can expand coverage, but its license, speaker distribution, recording style, transcript quality, and collection purpose must be documented. ManaTTS, for example, is a Persian TTS resource and dataset-building recipe, while its associated informal material was used to evaluate ASR during alignment. It is useful for specific questions; it is not evidence that a transactional agent works across all Persian speakers.

Keep the test set outside training and prompt-tuning loops. Version the audio, reference transcript, normalization target, entity labels, and expected dialogue outcome together. When the product changes its interpretation of a spoken date or amount, create a new version instead of silently rewriting the old answer key.

Treat Persian Normalization as a Product Policy

The ASR transcript is not necessarily the value an application should store. Consider:

«بیست و سوم تیر ساعت پنج و نیم، برای دکتر نادری»

A useful system may need to produce a calendar date in the user's intended calendar and time zone, 17:30, and the exact person name. Yet the raw transcript should remain available to the next stage so that an irreversible normalization does not erase ambiguity.

Persian introduces several recurring decisions:

• Persian and Arabic forms of visually similar characters, especially ی/ي and ک/ك;
• Persian, Arabic-Indic, and Latin digits;
• spacing and half-space variants;
• written versus spoken forms of numbers;
• solar Hijri, Gregorian, and occasionally lunar dates;
• rials versus tomans;
• colloquial contractions and alternative spellings;
• English acronyms, model names, URLs, and alphanumeric identifiers.

Apply Unicode normalization deliberately, then add language-aware canonicalization rather than relying on character replacement alone. Preserve three representations where consequences are material: the audio segment, the literal transcript, and the normalized application value. Record which normalization rule produced the value.

For money, date, address, and identity fields, use typed parsers and validation rules after ASR. A language model can propose a candidate, but arithmetic and calendar conversion should be deterministic and testable. If “سه و نیم” could mean 3.5 units or 3:30 in context, the dialogue should resolve the ambiguity rather than guessing.

Measure Entity and Task Accuracy, Not Only WER

Word error rate is informative, but its equal treatment of words does not match product consequence. Replacing a filler word and replacing the recipient name both count as substitutions. Character error rate can be helpful for spelling and script differences, yet it also does not answer whether the task succeeded.

Maintain a scorecard with:

1. WER and character error rate by slice. Report confidence intervals and the number of speakers and utterances, not just a global mean.
2. Critical entity error rate. Measure exact and normalized accuracy separately for names, amounts, dates, addresses, and domain terms.
3. Semantic frame accuracy. Did the system recover the correct intent and all required slots?
4. Task completion and safe-abandon rate. Did the user finish, reach a human, or leave after repeated repair?
5. False-action rate. How often did the system execute or prepare the wrong consequential action?
6. Repair burden. Track clarification turns, repeated prompts, corrections, and transfers per completed task.
7. Slice gaps. Compare performance across accent, channel, noise, register, and age slices that were ethically collected.
8. Latency. Report end-of-speech to partial transcript, final transcript, and first audible response at p50 and p95.

Set stricter gates for critical entities than for conversational prose. A customer-service summary may tolerate minor surface errors. A transfer amount should not. The deployment decision should be tied to the worst relevant slice and consequence, not the average.

Design Repair as Part of Persian Conversation

Low confidence is inevitable; poor repair is optional. Repeating the entire sentence forces the user to do unnecessary work and often produces the same recognition error. Ask about the uncertain span:

«مبلغ را سه میلیون تومان گفتید؟»

«نام خانوادگی “نادری” است یا “نظری”؟»

Confirmation should reflect consequence. Explicitly confirm money movement, medication, legal consent, address changes, and identity attributes even when acoustic confidence is high. For low-risk search, a lightweight implicit confirmation may be enough.

The wording needs native review. Persian politeness is not a single “formal” switch. An excessively bureaucratic prompt can sound distant; a very colloquial prompt can be inappropriate in healthcare or finance. Test pronouns, honorifics, turn length, interruption cues, and apologies with users from the intended context. A reliable production voice-agent architecture should expose ASR confidence, entity confidence, policy state, and repair history to the dialogue manager rather than hiding them behind one generated response.

Evaluate Synthesis Separately From Recognition

TTS quality cannot be inferred from ASR results. Review intelligibility, pronunciation, stress, rhythm, speaking rate, emotional appropriateness, and consistency across generated segments. Borrowed English words, proper names, abbreviations, numbers, and compound words require a pronunciation lexicon or override mechanism.

Use native listeners and task-based tests. Mean opinion scores can compare voice quality, but add comprehension questions and pronunciation error counts. Check whether a spoken amount, date, medicine name, or one-time code is understood correctly on the first listen. Measure time-to-interrupt and barge-in recovery because a beautiful voice that ignores the user still creates a poor interface.

Do not clone or imitate a person's voice without documented rights and consent. Disclose synthetic speech where confusion about human identity could matter. Secure pronunciation dictionaries and custom recordings as production data, not as informal files on a developer laptop.

Run a Layered Release Process

A practical rollout can use four gates:

Offline gate. Evaluate frozen audio and annotated outcomes. Review every critical error category manually and compare against the current production baseline.

Shadow gate. Run the new system on consented live traffic without controlling the customer interaction. Measure slice performance, latency, and disagreement with the current path.

Limited-action gate. Allow low-consequence tasks for a small cohort. Require confirmation and provide an immediate human escape route.

Expanded gate. Increase coverage only when entity accuracy, repair burden, false-action rate, and user outcomes remain within limits. Keep rollback triggers tied to live metrics.

Every model, language pack, normalization rule, prompt, pronunciation dictionary, and endpointing configuration belongs in the release manifest. A vendor's silent model update can alter behavior even when application code does not change. The operational readiness checklist is a useful companion for ownership, rollback, and incident response.

Privacy, Security, and Representation Risks

Voice can reveal more than the words spoken. Recordings may contain bystanders, background media, health information, or identifiers; voiceprints can become biometric data depending on use and jurisdiction. Minimize collection, separate authentication from general transcription, encrypt retained material, define access by role, and verify deletion across raw audio, derived clips, annotations, backups, and vendor systems.

Prompt injection can also arrive through audio: a recording, television, or second speaker may issue instructions. Treat recognized speech as untrusted input. Bind actions to the authenticated session, apply tool permissions independently of language-model output, and require confirmation for consequential operations.

Finally, do not market an accent gap as an individual user's failure. Publish known limitations, offer text and human channels, and direct data collection toward underserved conditions without exploiting speakers. Better coverage is a continuing service obligation.

Frequently Asked Questions

Is a multilingual foundation model enough for Persian?

It can be an excellent baseline. It is not proof of product fitness. Evaluate it on the intended dialects, channels, entities, code-switch patterns, and actions, then add domain adaptation and repair where evidence shows a gap.

Should we convert every transcript to standard written Persian?

No. Standardization can help search and downstream logic, but it may erase meaning or speaker intent. Preserve the literal transcript and create a separate normalized value with traceable rules.

What is the most important metric?

For a transactional interface, critical entity accuracy and false-action rate usually matter more than aggregate WER. For dictation, WER and editing time may dominate. Choose metrics from the task and consequence.

How often should the corpus be refreshed?

Add reviewed examples whenever products, vocabulary, channels, or user populations change. Keep a stable regression set for longitudinal comparison and a newer challenge set for emerging failures.

Source Notes — Reviewed 2026-07-30

• Mozilla Foundation, Common Voice Persian: community-led scripted and spontaneous speech context.
• Conneau et al., FLEURS: multilingual evaluation design and dataset scope.
• Sedghiyeh et al., Persian Speech Recognition Benchmark: Persian ASR error variation across speakers and conditions.
• Salehi et al., ManaTTS Persian: Persian TTS corpus construction and informal-speech evaluation material.
• Unicode Consortium, Unicode Normalization Forms: character-normalization definitions; product-specific Persian canonicalization still requires additional rules.

Public corpora do not fully represent production traffic, and published benchmark results are not directly comparable when preprocessing, data splits, or normalization differ. Teams should verify current dataset licenses and datasheets before reuse.