Synthetic Data With a Birth Certificate

Synthetic data can unblock software testing, simulate rare events, augment training, and reduce some uses of sensitive records. But “synthetic” describes how data was produced; it is not a guarantee that the result is private, representative, statistically useful, or safe to release.

A governed synthetic dataset needs a birth certificate: who made it, from what source population, for which purpose, under which privacy controls, how it was tested, who may use it, and when it must be reviewed or retired.

This guide provides a practical governance model for tabular, text, image, audio, and simulated synthetic data. It focuses on the decisions a data owner must make before a generated dataset enters software development, model training, evaluation, research, or external sharing.

Begin With Purpose, Not a Generator

Different purposes require different kinds of realism:

Intended use	What the data must preserve	What it does not need to preserve
Schema and pipeline testing	fields, types, constraints, nulls, error cases, volume	real population statistics
User-interface testing	plausible values, formatting, edge-case lengths, locale behavior	analytical fidelity
Statistical analysis	relationships required by the named analysis	unrelated distributions
Model training	task-relevant conditional patterns and difficult slices	direct resemblance to individuals
Rare-event augmentation	validated rare mechanisms and prevalence assumptions	artificial balance presented as real prevalence
Privacy-preserving sharing	approved utility plus measured disclosure risk	unsupported claims of anonymity
Evaluation	independent, representative cases and protected answers	generator patterns seen during training

The UK's Office for National Statistics synthetic-data policy makes this purpose dependence explicit: synthetic data should be expected not to preserve every property of real data, and suitability depends on how it was produced and what it will be used for.

Write the intended use and prohibited uses before choosing the method. “Improve access to health data” is too broad. “Allow external developers to test schema validation without analytical claims” is reviewable.

The Synthetic Data Birth Certificate

Every released dataset should have a versioned data card containing:

Identity and ownership

dataset ID, version, creation date, owner, steward, and contact;
intended users, approved environments, access tier, and license;
intended uses, prohibited uses, review date, and retirement triggers.

Source lineage

source dataset versions and collection periods;
population, sampling method, geography, and known missing groups;
sensitive fields and legal basis for processing real data;
cleaning, imputation, filtering, labeling, or transformation steps;
whether source records, public data, simulations, or foundation models influenced generation.

Generation record

generator name, version, code revision, configuration, random seed, and privacy parameters;
training environment and access controls;
constraints applied before or after generation;
rejected batches and reasons for rejection;
human or automated post-editing.

Validation record

utility tests tied to the intended use;
privacy and disclosure tests tied to an explicit attacker model;
subgroup, conditional, temporal, and rare-event analyses;
domain-expert review;
comparison with real and simple baseline data;
unresolved limitations and acceptance decision.

The record should let another qualified team reproduce the generation and understand why the owner allowed this version to be used.

Privacy: Artificial Does Not Mean Anonymous

A generator can memorize and reproduce source records or preserve combinations that identify a person when linked with outside information. Partially synthetic data may retain untouched personal fields. Even fully synthetic output can reveal sensitive attributes about small groups.

The UK Information Commissioner's guidance on privacy-enhancing technologies treats synthetic data as a privacy technique with residual risks, not an automatic exemption from data-protection duties.

Define an attacker model before testing:

Does the attacker know some real records?
Can they query the generator or only see a released dataset?
Do they know that a person was in the training population?
Can they combine the output with public or commercial datasets?
Are rare combinations, free text, images, or timestamps present?

Then evaluate exact or near-record matches, membership inference, attribute inference, singling out rare records, linkage using quasi-identifiers, memorized media, and leakage through metadata or generation logs.

If a formal privacy claim is required, consider mechanisms with a defined guarantee. NIST's SP 800-226 guidance explains how to evaluate differential-privacy claims and their assumptions. A privacy budget is not a decorative number: document adjacency, accounting, composition, clipping, randomness, and implementation tests.

Do not publish a dataset merely because common attacks failed. Attack tooling is incomplete, and future auxiliary data may change risk. Use access controls, contractual limits, secure environments, output review, and retention rules in addition to generation.

Utility: Test the Decision the Data Will Support

Marginal distribution similarity is an inadequate quality claim. Two datasets can share column histograms and disagree on the relationships that matter.

Use a purpose-specific test plan:

Structural validity: types, constraints, uniqueness, referential integrity, null behavior, and realistic error cases.
Univariate fidelity: ranges, frequencies, missingness, and tails.
Multivariate fidelity: correlations, conditional distributions, interactions, sequences, and causal constraints relevant to the use.
Subgroup fidelity: performance for small, protected, high-risk, and operationally important groups.
Temporal and geographic fidelity: seasonality, drift, events, regional differences, and policy periods.
Downstream utility: repeat the intended analysis or train the intended model, then compare results with real-data and simple-baseline results.
Expert plausibility: domain specialists inspect cases that are statistically ordinary but operationally impossible.

The ONS synthetic-data methodology paper describes a useful spectrum: data made only for structural testing has different analytical value and disclosure risk from data designed to preserve detailed multivariate relationships. Greater realism can increase privacy risk.

Report uncertainty and failure, not only a similarity score. A dataset may be approved for load testing but rejected for training, approved for aggregate research but rejected for individual-level decisions.

Bias, Rare Events, and Impossible Records

Synthetic generation can:

smooth away rare but consequential cases;
amplify historical bias in source data;
rebalance classes and accidentally imply false prevalence;
invent combinations that violate physical, legal, or business constraints;
underrepresent language varieties or sensor conditions;
reproduce label errors at scale;
make minority groups look statistically cleaner than reality.

Build invariant checks from domain rules. A transaction date cannot precede account creation; a medical procedure may require a preceding diagnosis; a machine state may exclude another state. Review difficult slices separately, and preserve the difference between “rare” and “invalid.”

For augmentation, record how many synthetic examples were added per class and test the final model against a real, untouched evaluation set. A gain on synthetic validation data may only show that the model learned the generator's artifacts.

Keep Training, Evaluation, and Benchmark Generation Separate

Synthetic data can contaminate evaluation when the same generator and prompt create training and test cases, a foundation model used to generate questions has seen benchmark answers, paraphrases preserve answer cues, or synthetic cases are easier than production inputs.

Use independent generation pipelines, private holdouts, deduplication, and contamination tests. Keep generation prompts, seeds, source pools, and model versions. Where possible, validate with real post-deployment outcomes.

Our guide to evaluation datasets and synthetic data covers release-set design; data quality observability explains how to monitor the upstream data that feeds generation.

Release Decisions and Access Tiers

Do not treat release as binary. Define tiers:

Internal sandbox: controlled users, no production decisions.
Restricted research: approved purpose, trained users, monitored exports.
Partner access: contract, audit rights, retention and onward-sharing limits.
Public release: disclosure review, durable documentation, licensing, and no reliance on later deletion.

For each tier, document who approved utility, privacy, legal basis, security, and communication. If a version changes, repeat the decision; approval does not automatically transfer.

Monitor Drift and Retire Stale Data

Synthetic data becomes stale when the real population, workflow, product, sensors, coding rules, law, or adversary changes.

Set a scheduled review date, source-drift thresholds, schema-change triggers, incident and re-identification triggers, generator-change triggers, and an owner for regeneration and downstream notification.

Maintain a registry of which models, analyses, tests, and products used each version. When a defect is discovered, this lineage defines the blast radius. Preserve old versions and their environments for reproducibility, but remove access when their approved use ends.

A Practical Approval Checklist

Before release, require evidence that:

the purpose and prohibited uses are precise;
the source data was lawfully and securely processed;
generation is reproducible and versioned;
utility tests match the actual downstream decision;
privacy tests match a stated attacker model;
important subgroups and rare events were reviewed;
training and evaluation pipelines are independent;
access, sharing, retention, and deletion rules are enforceable;
limitations are visible to downstream users;
review and retirement triggers have owners.

The evidence-first automation guide provides a reusable structure for this approval packet.

Frequently Asked Questions

Is synthetic data outside privacy law?

Not automatically. Creating it from personal data is itself processing, and the output may remain linkable or reveal information. Legal status depends on the data, method, attacker capabilities, jurisdiction, and controls. Obtain qualified privacy and legal review for the actual use.

Can synthetic data replace real data?

Sometimes for a narrow purpose. Structural test data can replace real records in development. Synthetic data may support analysis or training when validated for those tasks. It should not be presented as a complete substitute for a population or process it cannot faithfully represent.

Does differential privacy make all synthetic data safe?

Differential privacy can provide a formal, bounded guarantee when correctly implemented and documented. Utility may fall, configuration matters, and non-private components or later joins can still create risk. Evaluate the complete release pipeline.

How often should a synthetic dataset be regenerated?

Use risk- and drift-based triggers rather than an arbitrary schedule alone. Regenerate when source distributions, schemas, policies, products, sensors, labels, generators, or approved purposes materially change.

Source Notes

This guide was substantially reviewed on July 30, 2026 against:

Synthetic data is a designed artifact, not a free substitute for reality. Its value comes from knowing where it came from, what it can represent, and where it must not be trusted.