ZharfAI Team
AI search changes more than the shape of a results page. It moves part of the reader’s work—finding, comparing, and synthesizing sources—into an answer assembled by a platform. That can be genuinely useful. It can also move attention and bargaining power away from the organizations that paid for reporting, research, photography, databases, or expert analysis.
The useful question for a publisher is therefore not “Will AI search kill traffic?” No single number can answer that across query types, countries, brands, or products. The operational question is: which uses of our work create discoverability, which substitute for the visit, what controls exist, and how will we measure the exchange?
As of July 30, 2026, those controls and measurements are still uneven. Google says AI Overviews and AI Mode can use “query fan-out” to retrieve pages across subtopics and that ordinary Search eligibility and snippet controls apply. In the UK, the Competition and Markets Authority has gone further: its June 2026 publisher conduct requirement obliges Google to provide effective controls over generative-AI use in search, clear attribution, routes to the source, and detailed engagement metrics. The requirement is UK-specific and implementation is not instantaneous, but it establishes an important benchmark for a fair answer product.
An answer has at least five economic stages:
Classic web analytics observe stage five mainly through clicks. AI answers can create value at stages three and four without producing a visit. That does not prove that every use requires payment, nor does it settle copyright or competition questions. It does mean that “we sent some high-quality clicks” is an incomplete accounting system.
Segment queries before drawing conclusions. A two-sentence factual lookup, a breaking-news query, a product comparison, a specialist method, and a branded subscription query have different substitution risks. The same answer interface may send useful discovery traffic for one class and satisfy the user completely for another.
For each class, estimate:
publisher value = qualified visits + attributable brand exposure + conversions + licensing value - substituted sessions - access costs
This is a management model, not a legal formula. Its purpose is to force the team to name benefits and losses separately instead of hiding both inside total organic traffic.
A row of source logos beneath a long response is not enough. A functional citation should let the reader identify which source supports which claim, open the relevant page, and distinguish direct evidence from the answer engine’s inference. When sources disagree, the interface should preserve the disagreement rather than merge it into false consensus.
Publishers can test attribution with a small, repeatable audit:
Useful rates include claim-support rate, correct-attribution rate, source-diversity per answer, fresh-source rate, and citation prominence. These metrics evaluate the answer experience; they do not imply that citation alone resolves licensing or compensation.
For a deeper treatment of traceable claims, read our guide to content provenance and watermarking. The related guide on RAG knowledge quality explains why retrieval quality, passage support, and freshness need separate tests.
The IETF’s Robots Exclusion Protocol, RFC 9309, standardizes how site owners express crawler preferences. It also says robots.txt is not access authorization. A compliant crawler may honor it; a malicious or misconfigured client can ignore it. Sensitive or paid material still needs authentication, authorization, rate limits, and contractual controls.
For Google Search specifically, the company’s AI features and website documentation says Googlebot access governs eligibility for Search AI features, while nosnippet, data-nosnippet, max-snippet, and noindex limit what Search may show. Google-Extended applies to some other training and grounding uses, not the Search controls described on that page. Publishers should verify current documentation instead of assuming that one crawler token controls every product.
Maintain a versioned matrix with:
| Content class | Indexing | Snippet policy | AI-search use | Training use | Access method | Owner |
|---|---|---|---|---|---|---|
| Public service information | Allowed | Generous | Discovery preferred | Policy decision | Public | Editorial |
| Original investigations | Allowed | Limited excerpt | Attribution required | Licensed or blocked | Public | Editor in chief |
| Paid archive | Landing page only | Minimal | No full answer | Blocked | Authentication | Product |
| Licensed database | No public indexing | None | Contract only | Contract only | API/auth | Data lead |
This matrix makes trade-offs explicit. Blocking all discovery may protect extraction but reduce audience growth. Allowing everything may increase reach while weakening subscription or licensing value. The correct choice is a portfolio decision, not a universal SEO setting.
Google announced dedicated generative-AI performance reports in Search Console in June 2026, initially for a subset of sites. Where available, publishers should export the dedicated view rather than infer AI exposure only from aggregate “Web” search data. Where it is unavailable, mark the visibility dataset as incomplete.
A practical scorecard has four layers:
Answer visibility
Referral quality
Substitution signals
Economic outcome
Do not call a traffic change causal merely because an AI feature appeared at the same time. Ranking updates, seasonality, news intensity, interface changes, and measurement gaps can move the same metrics. Use matched page groups, annotated timelines, and, where possible, geographic or query-level comparisons.
Treat content according to how it creates durable value.
Definitions, public-interest explainers, documentation, and useful summaries can be designed to travel. Make the entity, author, date, evidence, and canonical page easy to identify. The goal is accurate reach and a credible path to deeper material.
Original reporting, expert interpretation, proprietary benchmarks, and tested methods should expose enough to be discoverable without making the source unnecessary. Strong author identity, transparent methods, update history, primary evidence, and distinctive data make both readers and answer systems more likely to recognize the source’s unique contribution.
Databases, premium archives, alerts, tools, and high-cost research may require authentication, metered access, licensing, or an API. Do not depend on robots.txt to protect them. Decide what public landing page supports discovery while the underlying value remains access-controlled.
This portfolio is stronger than chasing a new “AI optimization” checklist. Google’s own documentation says no special schema or AI text file is required for inclusion in its AI Search features. Invest first in original value, technically sound indexing, accurate structured data, internal linking, and a direct reader relationship.
Imagine a publisher sells a weekly energy-market briefing. Its public article contains a chart, three findings, and a summary; subscribers receive the dataset, methodology, alerts, and analyst call.
The team can:
If impressions rise while qualified visits and branded demand remain stable, the answer surface may be expanding discovery. If the system repeatedly reproduces the briefing’s conclusions without a usable citation and clicks collapse for that query class, the publisher has a concrete control, product, and policy problem—not merely an SEO mood.
Publisher data will remain partial. Search platforms see answer impressions and interactions that site analytics cannot. Referral headers may not identify the feature. Controlled experiments are difficult because products and rankings change continuously.
Regulatory obligations also differ by jurisdiction. The UK CMA publisher conduct requirement applies to Google’s designated UK general-search activity; it is not a worldwide licensing rule and does not by itself decide copyright disputes. Its practical value is that it names controls, transparency, engagement metrics, attribution, and source access as connected requirements.
Finally, blocking is not bargaining power unless the publisher has something users or platforms cannot cheaply replace. The long-term defense is a combination of enforceable access controls, collective or direct licensing where appropriate, recognizable authorship, proprietary evidence, useful products, and channels the publisher owns.
Not by default. Separate search discovery, model training, commercial retrieval, archiving, and unknown bots. Decide by content tier and business goal, then enforce protected access with authentication rather than crawler etiquette alone.
No. Citation supports traceability and may produce discovery or brand value, but it does not automatically answer licensing, copyright, competition, or revenue-sharing questions.
Start with query-class cohorts that combine AI-feature visibility, citations, clicks, engaged visits, conversions, and direct or branded demand. A single total-organic-traffic chart hides the mechanism.
No. Accurate structured data can help search systems understand visible content, but Google says there is no special AI schema and does not guarantee crawling, indexing, inclusion, or ranking.
The platform documentation describes Google’s own product and should be read as a first-party account, not independent proof of publisher impact. The CMA requirement is authoritative for its UK scope, while the IAB workshop report records areas of disagreement rather than technical consensus.
An operational view of AI in Iranian banking: fraud detection, credit scoring, Persian customer assistants, and document automation, with governance requirements and a low-risk pilot path.
Read More
An evidence-first checklist for selecting an AI company in Iran: define the workflow, test Persian performance, examine security, measure a pilot, and negotiate an exit.
Read More
Iranian AI is more than a label. A decision guide for weighing local platforms against foreign tools: data residency, access resilience, Persian quality, and a fair evaluation checklist.
Read MoreSee the daily briefing and the operational guides. This page is an archive note, not an invitation to start a project.