ZharfAI Team
AI can help a language community search recordings, transcribe speech, build teaching materials, and test translation tools. It cannot preserve a language by itself. A language remains living when people can use it at home, in school, in public services, in art, and online—and when its speakers retain authority over how their words and knowledge are recorded or reused.
That distinction matters because the most impressive model demo may solve the wrong problem. A searchable archive can make decades of recordings usable, while a fluent-looking chatbot can normalize the wrong spelling, expose restricted stories, or persuade funders that human teaching is no longer necessary. This article is for community organizations, language technologists, researchers, and public institutions deciding where AI belongs in a preservation or translation program.
UNESCO describes the International Decade of Indigenous Languages, 2022–2032, as a program for language rights, Indigenous-language education, digital inclusion, and transmission between generations—not merely digitization. Its April 2026 overview says nearly 40 percent of the world’s languages are endangered, most of them Indigenous. That figure establishes urgency, but it does not turn every recording into open training data. See UNESCO’s current IDIL overview.
A useful project therefore begins with a community outcome: more children learning the language, easier access to elders’ recordings, reliable terminology for a clinic, or a keyboard that supports the writing system. The AI component is only one means. If the actual obstacle is teacher funding, broadband access, font support, or permission to use an archive, a larger language model will not remove it.
“Language,” “dialect,” and “variety” are also not neutral labels. Boundaries can reflect political history as much as linguistic distance. Project teams should use the names and groupings chosen by speakers, record variation rather than flatten it, and avoid treating one prestige variety as the automatic target.
Before training anything, inventory the material and its rights. A practical register records the speaker, collector, date, place at an appropriate level of precision, genre, variety, writing system, audio quality, transcription status, translation status, and permitted uses. It should also record who can change a permission decision and how a contributor can request correction, withdrawal, or restricted access.
The CARE Principles for Indigenous Data Governance add a crucial layer to familiar open-data practice: Collective Benefit, Authority to Control, Responsibility, and Ethics. “Findable” or “reusable” is not enough when data contains sacred knowledge, names of deceased people, location-sensitive information, or material that a community permits for education but not commercial model training.
Access should be tiered. Public dictionary entries, community-only teaching recordings, researcher-access material, and restricted ceremonial content do not belong in one bucket. Permissions must follow derived artifacts too: transcripts, embeddings, synthetic voices, model checkpoints, evaluation sets, and exported prompts can all reveal source material even when the original audio file is hidden.
“Build AI for our language” is too broad to design or evaluate. Separate tasks include audio segmentation, speaker diarization, speech recognition, spelling normalization, dictionary search, morphological analysis, optical character recognition, text-to-speech, translation, terminology retrieval, and conversational practice. Each needs different data and carries different harms.
For an oral-history archive, timestamped search suggestions with a human editor may be more valuable than full automatic translation. For a clinic, a constrained terminology assistant with escalation to an interpreter is safer than an open-ended speech translator. For a school, aligned sentences and teacher-controlled exercises may matter more than a general chatbot. Readers working across Persian scripts and product interfaces can compare this scoping process with our guide to Persian NLP and language localization.
Write the task as an observable job: “help an authorized archivist find every mention of three place-name variants,” not “understand the language.” This creates a testable output, identifies the responsible reviewer, and prevents an attractive demo from quietly expanding into a higher-risk service.
The old assumption that useful language technology requires millions of translated sentences is too rigid. Multilingual pretraining, transfer learning, lexicons, pronunciation rules, and carefully selected examples can reduce the amount of new data required. But “zero-shot” does not mean that a system has inferred a grammar reliably from a few hours of speech. It means the model applies learned representations to a language or task with little or no task-specific training data; quality may still be poor and uneven.
The primary FLORES-101 benchmark paper illustrates both progress and limits. Its authors created 3,001 professionally translated sentences aligned across 101 languages because existing low-resource evaluation sets often had weak coverage, narrow domains, or semi-automatic construction. FLORES enables controlled comparison, but a Wikipedia-derived benchmark cannot represent every oral genre, dialect, school lesson, medical encounter, or culturally specific expression.
Small, trusted data can outperform a larger, noisy scrape for a defined task. Ten hours of carefully consented, well-transcribed speech from the intended variety may be more actionable than thousands of unlabeled clips from unknown speakers. Report the data composition plainly: speakers, age ranges when appropriate, varieties, genres, recording devices, and exclusions.
Automatic scores are useful for regression testing, not for declaring a language “solved.” A 2024 ACL study of Assamese, Kannada, Maithili, and Punjabi found that even learned zero-shot evaluation metrics correlated only modestly with human annotations: up to 0.32 Kendall correlation and 0.45 Pearson correlation in the study. The authors concluded that synthetic data did not close the evaluation gap consistently. Read the original low-resource MT evaluation study.
A real evaluation panel should include fluent speakers from relevant varieties and domain reviewers when consequences are high. Ask them to label meaning errors, missing information, added information, names, kinship terms, politeness, morphology, terminology, spelling, code-switching, and whether a sentence sounds natural. For speech, test background noise, elder and youth voices, channel quality, pauses, and overlapping speakers.
Average accuracy can hide exclusion. Publish results by variety, genre, speaker group where ethically appropriate, and sentence type. A model that performs well on news text but fails on oral narratives is not a preservation system. Our broader explanation of AI translation and computational linguistics covers architecture choices; here, the decisive requirement is locally defined human evaluation.
Models can cluster signs, propose readings, restore damaged image regions, retrieve parallel passages, or rank candidate segmentations in historical documents. These are research aids. They do not restore a lost speech community, recover unrecorded pronunciation, or prove the meaning of an undeciphered script.
For a language with historical attestations but no living fluent speakers, label every output by evidence level. A direct transcription, a scholarly reconstruction, a model-ranked hypothesis, and generated illustrative text are different objects. Preserve the source image and editorial history beside any proposed reading. Do not let a confident interface remove uncertainty that the underlying scholarship still contains.
For sleeping or reawakening languages, communities may use archival sources to support revitalization. The authority to define new terminology, pronunciation practice, or teaching norms belongs to those communities. A model can surface evidence and speed repetitive work; it should not be presented as the speaker whose absence it is meant to address.
A defensible workflow has explicit gates. First, a governance group approves the use case and data classes. Second, archivists and speakers create a small gold set. Third, the system runs in suggestion mode: it proposes segments, spellings, or translations without publishing them. Fourth, authorized reviewers correct outputs and record error types. Only after the failure pattern is understood should the project expand.
For translation, retain the source, model version, prompt or decoding configuration, reviewer, corrections, and publication decision. For speech, retain time alignment so a reader can return to the original voice. For educational material, teachers should be able to reject generated exercises, lock preferred terms, and see which sentences were machine-generated.
High-consequence settings need a fallback. A clinic or legal service should route uncertainty to a qualified interpreter rather than fabricate a fluent answer. A public archive should block restricted material even if semantic search retrieves a close match. A youth-facing tutor needs reporting and moderation designed with the community, not copied from a dominant-language consumer app.
Technical measures include word or character error rate, translation error categories, retrieval precision, calibration, reviewer agreement, and correction time. Operational measures include the percentage of recordings that become searchable, cost per reviewed minute, terminology coverage, turnaround time, and how often a reviewer overrides the system.
Language-program measures are different: active learners, teacher adoption, frequency of home or community use, new material created by speakers, access to public services, and continued intergenerational transmission. These cannot be attributed to AI alone, but they reveal whether the project advances the reason it exists.
Track harms with equal seriousness: restricted-item exposure, false attribution to a speaker, offensive normalization, missed dialect forms, inappropriate synthetic voice use, contributor withdrawal requests, and benefit-sharing disputes. A launch is not successful if it improves search speed while weakening trust in the archive.
Before signing with a vendor or research partner, ask who owns source data, annotations, fine-tuned weights, and corrected outputs. Can the provider train unrelated products on them? Can community administrators export everything in usable formats? What happens when a contributor changes permission? Where is the data stored, and which subcontractors can access it?
Require model and dataset documentation at the level needed for audit. Insist on per-variety evaluation, not a marketing claim about “hundreds of languages.” Define incident response for leaks and harmful outputs. Specify whether the system may synthesize a recognizable voice and who authorizes that use. If the project creates commercial value, define benefits rather than relying on vague promises of exposure.
The strongest exit clause is technical as well as legal: open formats, documented schemas, exportable corrections, and no dependency on a proprietary identifier. Preservation infrastructure must outlive a grant cycle, a startup, and a model generation.
During the first month, form a community-led governance group, choose one narrow task, classify permissions, and select a representative evaluation set. Document what will not be attempted. In the second month, run two or three baseline approaches, including a non-AI baseline, and have paid reviewers annotate errors. Measure both accuracy and review effort.
In the third month, deploy only to a small authorized group. Hold weekly error reviews, test access controls, rehearse withdrawal and incident procedures, and compare results with the original goal. A decision to stop, redesign, or fund human work instead is a valid pilot outcome.
Scale only when speakers judge the output useful, rights remain enforceable, subgroup performance is visible, and the organization can maintain the system. Preservation is a long relationship. The responsible role for AI is to make language work easier without taking ownership of the language or overstating what the evidence can support.
Substantively reviewed on 2026-07-30. The review used UNESCO’s current International Decade of Indigenous Languages framing, the GIDA CARE governance principles, the primary FLORES-101 benchmark paper, and the 2024 ACL study of low-resource machine-translation evaluation. These sources support program design and evaluation; they do not certify any particular model or authorize use of a community’s language data.
AI can improve access to archives while provenance, custody, integrity, and evidential status remain explicit—and every generated description stays attributable.
Read More
Font-generation AI can explore glyphs and spacing, but usable releases require correct encoding, script shaping, legibility, accessibility, rights, and testing.
Read More
How museums and art-market specialists can use AI to organize provenance, imaging, materials, and brushwork evidence without automating attribution.
Read MoreIf this note maps to a real system in your organization, start with the services page or a shipped case study.