How 1.6 million couplets on Ganjoor got new meanings

How 1.6 million couplets on Ganjoor got new meanings

Ganjoor, the free online library of classical Persian poetry, now has a fresh plain-Persian note under almost every couplet. How it was built, what it cost, and where to read it with care.

ZharfAI ResearchNovember 20, 202620 minutesSource: Arman Jafarnezhad

As of this week, almost every couplet on Ganjoor has a new explanation under it. Ganjoor is a free online library of classical Persian poetry, and about 1.6 million of its couplets, by 245 poets from Rudaki to Shahriar, now carry a plain modern-Persian meaning written by Claude Opus 5.5. The work is Arman Jafarnezhad's, the engineer who earlier built Ganjoorjoor, a semantic search engine over Ganjoor. The idea came on a sleepless night, November 9, 2026, and every meaning was written by November 16. On November 19, Hamid Reza Mohammadi, who built and runs Ganjoor, merged the changes into Ganjoor's public data repository, and on November 20 (29 Aban 1405) Ganjoor's blog announced that the new meanings were live on the site.

ZharfAI was one of two sponsors: ZharfAI and mana.space provided the AI credits, and Arman paid part of the cost. Arman tells the story in a blog post and gives the technical details in the pull request sent to Ganjoor.

What changed on Ganjoor

In December 2024 Ganjoor added machine-written meanings and summaries to every couplet and poem it had at the time that lacked one, paid for with credits from a support program run by an Iranian AI company. Mohammadi said in reply to a reader that most were written by GPT-4o mini and some by GPT-4o. Even today, only about 1.5% of couplets have a meaning written by a person. Mohammadi wrote at the time that every machine text began with the label «هوش مصنوعی:» ("AI:") so that, if a better model became available, those texts could be found and replaced. That day has now come. Arman's work was built on Ganjoor's data as of November 8, and by Arman's count the changes are:

Count
Couplet meanings that replaced an earlier AI meaning1,500,863
Couplet meanings added where there was none86,846
Human couplet meanings left untouched24,678
Poem summaries that replaced an earlier AI summary128,442
Poem summaries added where there was none645
Human poem summaries left untouched1,677

Three rules governed the swap. A new text replaced only a text that began with the AI label. Anything a person had written or edited stayed exactly as it was. And every new text carries the same label, so readers can still tell machine from human. The exception is long sections, such as those of the Shahnameh: they were sent to the model in windows, and while their couplet meanings were replaced, their old summaries were kept. The whole run used about 566 million tokens.

Arman calls the result "the largest commentary in the history of Persian poetry." The comparison Arman draws is with Sudi, who finished a verse-by-verse commentary on Hafez's divan in Istanbul in 1594, some 430 years ago. As Arman tells it, commentators have always gone to the famous five: Hafez, Rumi, Saadi, Ferdowsi and Nezami. By Arman's count, none of the 12,312 couplets by Waqif Lahori on Ganjoor had any explanation, not even a machine one, until this month.

What was wrong with the old meanings

The machine meanings drew criticism from the start. Many readers found them weak, and some laughable. The best-known case is a couplet from the story of Alexander in the Shahnameh:

دلارای برداشت چندان جهیزکه شد در جهان روی بازار تیز

Delaray gathered so rich a dowry that markets across the world were busy.

Delaray is the name of Roshanak's mother. The model saw «دلار», dollar, in the name and explained that the dollar had risen so far in value that it was being traded briskly around the world. That reading became a joke on Persian Twitter. Ferdowsi is saying something else: Roshanak's dowry was so large that it set the world's markets bustling.

Roshanak's dowry leaves the palace gate and sets the bazaar bustling: the couplet where the old note saw a dollar. Illustration: ZharfAI, with AI.

In March 2025 Mohammadi answered the critics in a tongue-in-cheek post with a practical request: correct the wrong meanings yourselves, and when you find one that is right, delete the "AI" label from it, both to show a person has checked it and so that the next round of replacement leaves it alone. The Delaray meaning no longer carries the AI label, which means someone has edited it, and the new run did not touch it.

Arman's diagnosis is that the old errors came less from the model's ability than from what it was given. It saw each couplet alone, with no idea which story it was in or what the previous line said. A model that knew this line belonged to the story of Alexander and Roshanak would never have reached for the dollar. GPT-4o also predates reasoning models.

Four couplets, before and after

Before the method, here is what the difference looks like. The notes are exactly what Ganjoor showed before and after this work; the English lines under them are glosses.

Majnun's khāl

In Nezami's Leyli and Majnun, an old man named Salim Ameri brings Majnun bread and clothes and finds him on a mountainside among wild animals. Nezami calls him Majnun's khāl (خال), his maternal uncle. Khāl also means a mole or beauty mark, and the old meaning read it that way.

Salim Ameri, Majnun's uncle, brings a bundle of bread and clothes to Majnun, who sits among gazelles, deer, a lion, a wolf and a fox. Illustration: ZharfAI, with AI.

کز نقد کنان حال مجنونپیری سره بود خال مجنون

Ganjoor’s old note

The new note

آنکه در بین معامله‌کنندگان با اندیشه‌هایی پاک (نقد کنان) به حال مجنون نگریسته، در حقیقت پیر زکات و درستکاری است که نشانه‌اش (خال مجنون) بر پیشانی اوست.

از میان کسانی که احوال مجنون را می‌شناختند و از او خبر داشتند، پیرمردی نیک و پاک‌نهاد بود که دایی مجنون بود.

Among the traders who looked on Majnun with pure thoughts is an elder of charity and honesty, whose sign (Majnun's mole) is on his forehead.

Among those who knew how Majnun was doing was a good, pure-hearted old man who was Majnun's maternal uncle.

Twelve couplets later, Nezami plays on the word. Salim introduces himself:

خال تو ولی ز روی تو فردروی تو به خال نیست در خورد

Ganjoor’s old note

The new note

خال تو، اما از روی تو جداست و به تنهایی، زیبایی‌ات را نمی‌تواند وصف کند.

با بازی با واژهٔ «خال» می‌گوید: من خال (دایی) تو هستم، ولی از چهرهٔ تو جدا مانده‌ام؛ و چهرهٔ تو اکنون دیگر جای خال (نقطهٔ زیبایی) نیست.

Your mole, but it is separate from your face, and alone it cannot describe your beauty.

Playing on khāl, he says: I am your uncle, but I have been parted from your face; and your face is no longer a place for a beauty mark.

The new meaning catches both senses. The whole section reads well from the start.

A lion with a bow

In Nezami's Haft Peykar, King Bahram Gur is out hunting with his slave girl Fitna and rides after a herd of onagers, gur in Persian.

Bahram Gur, bow drawn, rides after the onagers while Fitna watches with her harp. Illustration: ZharfAI, with AI.

چون درآمد به گور تیز آهنگتند شیری کمان گرفته به چنگ

Ganjoor’s old note

The new note

وقتی به قبر وارد شد، شیر تند و تیز که کمان را در دست داشت به سرعت به او نزدیک شد.

شاه مانند شیری تند و کمان به دست، با شتاب به سوی گورخرها درآمد.

When he entered the grave, a fierce lion holding a bow quickly approached him.

The king, fierce as a lion, bow in hand, rushed at the onagers.

The old model read gur as its other meaning, grave, and the lion as a separate animal. The new one noticed the onagers a line earlier and that the lion is Bahram himself. The new meaning is on the live Ganjoor page now.

The jewel box

In his autumn mosammat, Manuchehri walks through a garden after the harvest: the gardener bites his finger in astonishment that no rose or pomegranate blossom is left, and the fruit is described one by one. A couplet earlier, the old meaning had read nār, pomegranate, as fire; by this couplet the pomegranate had vanished altogether.

Manuchehri's autumn garden: the gardener raises a finger to his lips in astonishment, and a split pomegranate lies open like a jewel box beside citrons, oranges and grapes. Illustration: ZharfAI, with AI.

لختی گهر سرخ در آن حقه نهادهتوتو سلب زرد بر آن روی فتاده

Ganjoor’s old note

The new note

یک دانه مروارید سرخ در آن پارچهٔ رنگی گذاشته‌اند و نقاشی زردی بر آن درخشیدگی می‌افکند.

در آن جعبه مقداری گوهر سرخ (دانه‌های انار) نهاده‌اند و پرده‌های زرد لایه‌لایه بر روی آن‌ها افتاده است.

A single red pearl has been laid in that coloured cloth, and a yellow painting casts a shine on it.

In that box lie red jewels (the pomegranate seeds), with yellow membranes folded over them layer on layer.

The same poem shows one more thing: the meaning of its third couplet was written or edited by a person, and the new run left it as it was.

Sour grapes and raisins

In one of the quatrains Sadegh Hedayat selected from Khayyam, Khayyam speaks of people who spent their lives arguing about being and nothingness. The last couplet:

رو با خبرا، تو آب انگور گُزینکآن بی‌خبران به غوره مِیْویز شدند!

Ganjoor’s old note

The new note

با افرادی که از حقیقت آگاهند معاشرت کن و از لذت‌های زودگذر دوری کن، زیرا دیگران که بی‌خبرند، تنها در کارهای سطحی و ناپایدار غرق شده‌اند.

ای آگاه، برو و شراب را برگزین، زیرا آن بی‌خبران هنوز غوره بودند که مویز شدند. یعنی پیش از آنکه پخته شوند و از زندگی بهره ببرند، خشکیدند و پیر شدند. انگور اگر به شراب بدل شود به کمال می‌رسد، ولی غوره‌ای که مویز شود نه شراب شده و نه انگور شیرین.

Keep company with those who know the truth and stay away from fleeting pleasures, for the unaware are drowning in shallow, passing things.

You who know, go and choose wine, for those unaware ones turned to raisins while still sour grapes: they dried up and grew old before they ripened or enjoyed life. A grape that becomes wine is fulfilled; a sour grape that becomes a raisin was never wine and never sweet.

The old meaning split bā-khabarā into bā, "with", and khabarā, "those who know", and turned Khayyam into a preacher against pleasure. It is one word, a vocative: you who know.

Which model reads Persian poetry best?

Because the whole library would be annotated once, the choice of model was the decision that mattered most. Arman found no benchmark for Persian literature and built one: Konkour-Bench, which tests models on this year's Konkour, Iran's national university entrance exam. According to Mehr News, 1,070,241 people registered for it this year, and the humanities exam was held on the morning of Friday, August 21, 2026 (30 Mordad 1405). It suits the job: the questions are in Persian, written fresh each year, range from easy to hard, and include a literature section on many topics. Arman's reasoning is that a test run days after the exam cannot have its answer key in any model's training data, which is the condition for trusting any benchmark score. The published results come from runs between October 18 and 31, about two months after the exam, so that protection holds for models whose training data ends before the exam.

The test uses the 80 questions of the humanities track's first booklet: 20 in mathematics, 30 in Persian language and literature, 15 in social sciences and 15 in psychology. Every model receives the same Persian text and the same short instruction and must return only the number of its chosen option. No model had web search or other tools. Each was tested at three reasoning-effort levels: minimal, medium and max. The Claude models ran through the Claude Code command-line tool on a Claude subscription and the rest through OpenRouter. The repository's own guide calls these distinct evaluation protocols: Claude Code, for example, may take several internal steps before it answers. So Claude and the other models were not tested under identical conditions.

Across all 80 questions, Claude Opus 5.5 at medium effort and Gemini 3.8 Flash at max each answered 74 correctly. On the 30 literature questions, Opus 5.5 at medium came first with 27, followed by Opus 5.5 at minimal and Gemini 4 Argon at medium with 26. Opus 5.5 at max effort scored only 25. Arman chose Opus 5.5 at medium effort.

Correct answers to the 30 Persian literature questions of Konkour 1405

Each row is one model at one reasoning-effort level; the dot is the share correct and the line its 95% confidence interval.

ModelEffortCorrect of 30
Claude Opus 5.5medium27
Claude Opus 5.5minimal26
Gemini 4 Argonmedium26
Claude Opus 5.5max25
Gemini 3.8 Flashmax25
Gemini 4 Argonmax25
GPT-6.1 Solmedium24
GPT-6.1 Solmax24
Gemini 3.8 Flashmedium24
Gemini 3.8 Flashminimal23
GPT-6.1 Solminimal22
Gemini 4 Argonminimal22
DeepSeek V4.1 Flashmax21
Grok 4.7minimal21
DeepSeek V4.1 Flashminimal20
Claude Sonnet 5.5medium19
GPT-6 Lunamedium19
Qwen3.8 Maxminimal19
Qwen3.8 Maxmedium19
GPT-6 Lunaminimal17
GPT-6 Lunamax16
Claude Sonnet 5.5minimal13
Qwen3.8 27Bminimal13
Qwen3.8 27Bmedium10

Data: Arman Jafarnezhad’s Konkour-Bench results file (humanities track 1405, booklet 1). Wilson intervals. Incomplete or ineligible runs are left out. Claude models ran through Claude Code, the others through OpenRouter. The answer key has not been checked against the official one. Chart: ZharfAI.

Read this chart with care. With only 30 questions, first and second place are one question apart, and the 95% interval of the top row overlaps those of most rows below it. The test does not show which model is best; it shows which models are in the leading group. The repository also notes that its answer key has not been checked against the official one. Even so, the final choice was sound. Opus 5.5 stayed in the leading group at all three levels and did best at medium effort, which, at this volume, means fewer reasoning tokens and a smaller bill.

Choosing the prompt against human meanings

To choose a prompt Arman needed an answer key, and found one in Ganjoor's 1.5% of human-written meanings. If a prompt makes the model's meanings agree with what people wrote, it is a good prompt. Arman picked 160 poems that had human meanings, tested and compared the prompts on 80 of them, and locked the other 80 away to test the winner at the end.

Two models from two different companies judged the results: Claude Opus 5.5 and GPT-6.1 Sol. For each couplet, a judge marked the model's meaning correct, partial or wrong against the human one. A prompt's score was the share correct, with half credit for partial, averaged across the two judges.

Twenty-three prompts entered. Some were Arman's: more context, worked examples, a lesson in literary devices, a short version, instructions written in Persian. Arman then described all of them to Gemini and GPT and asked them to do better. A first round kept 15. Before the main round, Arman wrote down the decision rule: the most accurate prompt wins, but only if its lead is real; otherwise the cheapest one wins.

All 15 scored between 97.4% and 97.9%. So the rule fell back to cost, and the winner was one of the first prompts written. It simply gives the model the poem with its surroundings: the poet, the book, the section, the couplets before and after. Every cleverer idea layered on top made the output longer and none made it better. On the 80 locked-away poems the winner scored 97.8%, the same as before.

The big difference was in poem summaries. Judged by the same two models, Ganjoor's existing summaries scored about 43% and the new ones between 96% and 99%. Arman's conclusion is simple: you do not need to teach the model how to read Persian poetry, only to tell it what it is reading. It holds for prompt writing in general: telling the model where the text comes from does more than clever instructions.

What it cost, and why the cheaper prompt mattered

The prompt test took two days and cost about $160. It also produced real token counts, from which Arman re-estimated the total. Every request has a fixed cost, for the instructions, the poet's details and the summary, and each couplet adds a little. About 155,400 requests and 1.61 million couplets at batch prices of $2 per million input tokens and $10 per million output tokens come, by Arman's estimate, to about $3,100. The napkin estimate on that first sleepless night had been $3,000.

The estimated cost of annotating all of Ganjoor, by part

Batch prices: $2 per million input tokens and $10 per million output tokens

ZharfAI’s arithmetic from the formula and figures in Arman Jafarnezhad’s write-up: about 155,400 requests of about 1,700 input and 600 output tokens each, and about 1.61 million couplets of 40 input and 91 output tokens each. The total is $3,055, which Arman rounds to about $3,100.

About 80% of that figure is output, because each output token costs five times as much as an input token. That is why "longer, but not better" was enough to rule out every clever prompt: each extra sentence in the model's answer would have been paid for some 1.6 million times.

How the poems were sent

Short poems were sent whole. Long sections, such as those of the Shahnameh, went in windows of up to 30 couplets, each with up to three earlier couplets attached for context only, not to be explained. With the text went the poet's name, era, book and section, the verse form and metre, a short biography of the poet and the titles of the previous and next sections. Arman's example is from the fight between Rostam and Esfandiyar. The window opens with "why did you resort to tricks and deceit?" and nothing in that couplet says who is speaking to whom. The three context couplets do, and the model's meaning begins "Esfandiyar says to Rostam".

The single combat of Rostam and Esfandiyar on the bank of the Hirmand: lances, swords and maces broken, the two heroes grip each other by the belt. Illustration: ZharfAI, with AI.

For each couplet the model was asked for three things: a meaning of one to three sentences in plain Persian, a flag saying whether it was confident of its reading, and, for the whole window or poem, a summary of two to four sentences. Every response was checked before it was accepted: each numbered couplet covered exactly once, a summary of at least 40 characters, Persian script only, and a normal finish. A response that failed was requested again, at most three times. It is the standing rule of batch processing: nothing enters the main data unchecked. The run processed 155,651 poems or windows and produced 1,613,346 couplet meanings. According to Arman, no fallback model was needed and the model refused nothing.

Old against new on 500 random couplets

To measure how much better the new meanings are, Arman drew 500 couplets at random, in five separate draws of 100. The reviewers read each couplet with the ones before and after it, set the new meaning beside the old one and said which was better and by how much: much better, slightly better, or about the same.

New meaning against the old one, on 500 random couplets

Five separate draws of 100; the reviewers read each couplet with the ones before and after it.

Data: Arman Jafarnezhad’s write-up and pull request #4 to Ganjoor’s data repository. The reviewers were AI models and knew which meaning was new. Chart: ZharfAI.

The new meaning was better for 413 of the 500 couplets, or 83% (95% confidence interval 79–86%), and every draw landed between 76% and 89%. The old meaning was better in only four cases, and never much better. One caveat: by Arman's own account, the reviewers in this comparison were AI models too, and they knew which meaning was new.

The limits of this work

Arman lists several limits plainly, and anyone who uses these meanings should know them:

  • Most of the test poems come from the five best-known poets. Accuracy on less-studied poets, satire, dialect and prose is probably lower.
  • The model was unsure of its reading for 20.6% of couplets, 332,738 in all. In testing, flagged meanings were about eight times as likely to have a problem: 22% against 2.8%. The flag was not written into Ganjoor's data, so readers do not see it; the full list is available as a review queue.
  • The judges were not people. Both the prompt test and the 500-couplet comparison were judged by models. A Persian reader checked a sample by hand and, according to Arman, it passed.
  • 985 poems or windows are in languages or dialects other than standard Persian, including Mazandarani, Arabic, Azerbaijani Turkish, Kurdish, Isfahani and Gilaki. They were annotated too, but the quality tests were mostly on Persian poems.
  • Summaries of long sections were not refreshed. The page for Bahram and the slave girl in Haft Peykar, for example, has new couplet meanings under an old summary.

If that flag were shown beside the meanings, or those 330,000 couplets went first to Ganjoor's editors, the review would start where most of the errors are.

For teams working with Persian text

None of this is specific to poetry. Any organization that wants a language model to process a large volume of Persian text, such as contracts, letters or invoices, faces the same decisions. These lessons carry over:

  1. Build the answer key from your own data. There was no public benchmark for this job, so Arman made a test from the 1.5% of human meanings. Most organizations have a few hundred examples that someone has already done carefully by hand.
  2. Lock half the examples away and write the decision rule before you run. When 15 prompts sit within half a percentage point of each other, it is tempting to crown a "best" one without a rule set in advance.
  3. Count output length. About 80% of the cost here was output. A prompt that lengthens answers gets expensive across millions of requests.
  4. Tell the model where the text comes from; don't teach it. The winning prompt taught the model nothing. It only said where the text came from and what surrounded it.
  5. Check every response before storing it, and record doubt. The "not sure" flag separated out a fifth of the couplets, and that is where most of the errors were.

Ganjoor is one of the largest free collections of Persian text, and better meanings on it help anyone building search, teaching or reading tools in Persian. Arman's next step, according to the write-up, is to use these meanings in Ganjoorjoor.

Sources

Last checked November 20, 2026 (29 Aban 1405).

TagsGanjoorPersian PoetryPersian NLPLLM EvaluationBatch InferenceClaude Opus 5.5

Related Posts

Name one process for a discovery call

If this note maps to a real system in your organisation, start with the services page or a shipped case study.