The Open Model Factory: Operations for Open-Source AI

Downloading weights is not an operating model. A production team must know what it received, under which terms, from whom, through which build, with which dependencies, evaluation results, security controls, and rollback path.

The first discipline is vocabulary. “Open source,” “open weights,” “source available,” and “downloadable” are not synonyms. The Open Source AI Definition 1.0 from the Open Source Initiative describes freedoms to use, study, modify, and share and defines the preferred form for modification as data information, code, and parameters under qualifying terms. A release that publishes weights under use restrictions or omits the required code and data information may still be useful, but it should be classified accurately.

The OSI definition also does not certify that a model is safe, unbiased, lawful for a use case, or fit for production; the OSAID FAQ explicitly separates those questions. Operations begins after classification.

Build a model dossier before running the artifact

Treat every external model as an acquired software-and-data system. Create an immutable dossier keyed by the exact model digest, not a mutable tag such as latest.

The dossier should contain:

upstream project, publisher identity, repository and release URL;
exact weight, tokenizer, configuration, adapter, and code digests;
architecture, parameter count, context limits, modalities, languages, and chat template;
license or terms for code, weights, data, tokenizer, and included assets;
acceptable-use restrictions and jurisdictional or sector constraints;
training-data information, known exclusions, and provenance disclosures;
base-model and adapter lineage;
required runtime, drivers, kernels, libraries, container image, and hardware;
model card, evaluation methods, raw result artifacts, and known limitations;
security advisories, remote-code requirements, vulnerability scans, and review decisions;
importing reviewer, approval scope, expiry or review date, and operational owner.

Do not collapse licensing into one field. Code can use one license, weights another instrument, and a tokenizer or dataset another. Restrictions may affect modification, redistribution, hosted service, field of use, or generated output. Legal interpretation belongs to qualified counsel; the engineering requirement is to preserve the exact terms and prevent deployment beyond the approved scope.

SPDX 3.0.1 includes an AI Profile compliance point for inventorying software components and dependencies associated with AI/ML models and systems, while its Dataset Profile covers dataset metadata. These profiles can help exchange structured inventory; conformance to one profile does not imply conformance to all others or prove that the inventory is complete.

Model files are executable supply-chain inputs

Do not load an untrusted artifact on a production host simply to see whether it works. Some serialization formats can execute code during deserialization. Hugging Face’s pickle security documentation warns that malicious pickle content can enable arbitrary code execution.

Use an isolated acquisition lane:

resolve a release to immutable digests;
download through a controlled egress path;
verify expected hashes and, where available, signatures and attestations;
scan archives, dependencies, serialized files, and included code without executing them;
inspect configuration for remote code, custom kernels, network access, and dynamic imports;
convert unsafe formats only inside an ephemeral, network-restricted sandbox;
emit a new internal artifact with its own digest, provenance, and review record;
publish only to a quarantined internal registry;
promote after evaluation and security gates pass.

Prefer data-only formats such as safetensors where supported, but do not treat a file extension as proof of safety. The surrounding repository, tokenizer code, custom operators, container, and loader can still be malicious or vulnerable. Disable trust_remote_code by default. If a use case genuinely requires it, pin the exact code revision, inspect it, run with least privilege and no unnecessary network or secrets, and record the exception.

Sigstore’s Cosign documentation is one implementation option for verifying signed containers and blobs. SLSA provenance describes verifiable information about where, when, and how software artifacts were produced. These mechanisms establish artifact identity and build claims; they do not prove model quality or benign behavior.

NIST SP 800-218A augments the Secure Software Development Framework with practices for generative AI and dual-use foundation models. It is relevant to producers, acquirers, and users of AI models and should be applied with the base SSDF rather than treated as a model-specific checklist badge.

Reproduce the serving bundle, not only the weights

The same weights can behave differently with a changed tokenizer, chat template, system prompt, generation defaults, quantization method, runtime, kernel, adapter order, or tool schema. The deployable unit must pin the full serving bundle.

A release manifest should include:

model_digest
tokenizer_digest
configuration_digest
chat_template_digest
adapter_digests_and_order
quantization_recipe_and_calibration_set
runtime_and_container_digest
driver_and_kernel_compatibility
prompt_policy_version
tool_schema_version
safety_filter_versions
evaluation_suite_digest

Build the bundle in a controlled pipeline and generate provenance. Recreating an environment from a loose notebook is not reproducibility. A rollback needs the old artifact, configuration, runtime, dependency images, and data migrations—not merely the previous model name.

For higher-risk uses, test reproducibility at two levels. Deployment reproducibility means an approved bundle can be rebuilt or restored byte-for-byte where feasible. Behavioral reproducibility means the fixed evaluation set remains within defined tolerances despite nondeterministic generation. Record seeds and sampling settings, but do not promise identical text across different hardware or kernels unless verified.

Evaluate the workflow, languages, and failure modes

Public leaderboards are discovery tools. They rarely match the production prompt, tool permissions, languages, document noise, latency budget, or harm model. Build a versioned release set from authorized, representative data and keep a locked holdout.

Score:

task correctness and critical-field accuracy;
Persian, English, code-switching, dialect, and script variants used in production;
groundedness, citation support, and abstention;
structured-output and tool-call validity;
jailbreak, prompt-injection, data-exfiltration, and privilege-boundary tests;
unsafe-content and protected-group slices relevant to the use case;
long-context behavior, context-position effects, and conflicting evidence;
p50/p95 latency, time to first token, throughput, memory, energy, and cost;
failure under concurrency, dependency loss, and degraded hardware.

Report confidence intervals or sample sizes for proportions. A 100% pass rate on twelve cases is not equivalent to 99.8% on ten thousand. Keep raw outputs and evaluator versions. If an LLM judge is used, calibrate it against blinded human labels and prevent it from being the only arbiter of high-impact safety.

The original Model Cards paper proposed documenting intended uses, evaluation conditions, and performance across relevant groups and conditions. A model card is useful evidence, not a substitute for the acquirer’s own release evaluation.

Our frontier-model evaluation guide provides a fuller treatment of release sets and uncertainty. Open availability does not lower the evidence bar.

Choose a serving strategy from measured demand

Self-hosting creates control over data boundary, runtime, scaling, and version timing. It also transfers capacity planning, patching, observability, incident response, and on-call responsibility to the operator.

Benchmark at the real distribution of prompt length, output length, concurrency, and service-level objectives. Measure:

replica startup and model-loading time;
warm and cold time to first token;
decode throughput and p95 latency;
memory headroom and out-of-memory recovery;
scheduler fairness across tenants and long requests;
saturation behavior and queue abandonment;
quality and speed after quantization;
failover, scale-up, scale-down, and rollback time;
successful-work cost and energy, including idle capacity.

Batch processing, interactive chat, retrieval, and agentic tool use are different workloads. A configuration that maximizes offline throughput may fail interactive latency. A large shared endpoint may be efficient but create a data-isolation or noisy-neighbor risk.

Use model routing when one model is not the best fit for every case. Route from tested task and risk signals, retain fallbacks, and version the policy. Do not let a model choose itself solely from self-confidence.

Fine-tuning creates a new product lineage

An adapter or fine-tuned checkpoint is not a small configuration change. It creates a new artifact with new data obligations, behavior, and attack surface.

For every training run, record:

approved purpose and data-owner authorization;
dataset snapshot, inclusion and exclusion rules, deduplication, labels, and consent or legal basis;
sensitive-data handling, redaction, retention, and deletion propagation;
base-model digest and all adapter ancestors;
training code, hyperparameters, seeds, framework and hardware;
checkpoints, optimizer state where retained, metrics, and failure logs;
data contamination and benchmark leakage checks;
safety and regression evaluation against the base model;
reviewer decisions and final promoted digest.

Separate training, validation, and test entities to avoid leakage. When users can request deletion, document whether the response is data deletion from future runs, adapter retraining, machine unlearning, or another mechanism; do not imply that deleting a source row edits an already trained weight automatically.

Synthetic data needs lineage too. Record its generator, source prompts, filtering, and ratio. It can expand coverage but can also amplify model artifacts or leak benchmark answers.

Upgrades should be migrations with a canary

A new upstream revision can alter license terms, tokenizer, architecture, chat template, safety behavior, or hardware requirements. Never auto-promote a mutable upstream branch.

An upgrade process should:

open a new dossier and diff terms, artifacts, dependencies, and disclosures;
acquire and scan through the quarantine lane;
run compatibility and full regression suites;
benchmark the new serving bundle at realistic load;
run shadow traffic without affecting users;
canary a small authorized segment with stop conditions;
compare quality, safety, latency, capacity, cost, and energy;
expand gradually while the previous digest stays deployable;
close only after the observation window and incident review.

Do not compare only average thumbs-up rate. Watch critical failure slices, refusal changes, tool-call behavior, unsupported claims, and correction burden. An upstream security fix may justify expedited rollout, but it still needs scoped compatibility checks and a rollback.

Concrete example: a private bilingual support assistant

Imagine a company selecting an open model for an English-Persian support assistant with retrieval and read-only account tools.

Three candidates enter the acquisition lane. Candidate A has strong multilingual benchmarks but restrictive use terms. Candidate B publishes weights and code but requires remote custom code and has weak Persian evidence. Candidate C qualifies under the team’s open-source classification and has a complete artifact set, but needs quantization to fit available hardware.

The decision is not a beauty contest. The team creates one dossier per exact revision, obtains a legal classification, scans and converts artifacts, then evaluates a representative set:

Persian and English tickets, mixed numerals, dates, and product names;
answer support against authorized knowledge;
correct abstention for missing account facts;
valid read-only tool calls and resistance to injected tool instructions;
privacy and cross-tenant isolation;
p95 latency at peak concurrency;
memory and energy per accepted resolution;
human correction and escalation rate.

Candidate C may win only if its quantized bundle meets the quality floor and serving objectives. The team publishes that internal digest, not the upstream tag. Every answer trace records the model bundle, retrieval sources, tool results, policy version, and final disposition. A later upgrade replays the same cases and uses a canary.

This concrete workflow also supports on-device and private AI decisions: “self-hosted” is meaningful only when the full data path, logs, support access, and update mechanism respect the claimed boundary.

Release gates and production controls

No bundle reaches production until these gates pass:

Legal and classification: terms are preserved; intended use and redistribution are approved; “open source” or “open weights” labeling is accurate.

Integrity and provenance: all artifacts are pinned by digest; signatures or attestations are verified where provided; the internal build and promotion trail is complete.

Security: unsafe serialization and remote code are blocked or explicitly sandboxed; dependencies and containers are scanned; secrets and network access are least-privilege.

Quality and safety: task, language, grounding, structured output, abuse, and tool-boundary thresholds pass with adequate sample sizes.

Performance and capacity: latency, throughput, memory, failure recovery, energy, and cost meet the service envelope at realistic load.

Operations: an owner, alerting, runbook, incident path, previous bundle, tested rollback, and deprecation plan exist.

Monitor model and bundle digest, route, prompt and policy versions, error slices, unsafe-output signals, citation support, tool denials, p95 latency, queue depth, GPU memory, accepted-work cost, energy, patch lag, and rollback time. Alert on configuration drift from the approved manifest.

Frequently asked questions

Are downloadable weights open source?

Not necessarily. Review the code, data information, parameters, and terms against the classification you use. “Open weights” is often the more accurate label.

Is a model safe because its files use safetensors?

No. A data-only weight format reduces one deserialization risk. Repository code, loaders, custom kernels, containers, dependencies, and model behavior still require review.

Can we trust the upstream model card?

Use it as supplier evidence. Reproduce relevant tests where possible and run a deployment-specific evaluation on your languages, data conditions, tools, and risks.

Does self-hosting guarantee privacy?

No. Telemetry, logs, crash reports, model downloads, support access, vector stores, and tools can still send or expose data. Map and test the complete data path.

When should we fine-tune?

Only after prompt, retrieval, deterministic validation, and routing have been evaluated. Fine-tune when a stable behavioral gap remains and you can govern the data and new lineage.

What is the minimum rollback unit?

The complete serving bundle: weights, tokenizer, configuration, adapters, runtime, prompt and policy, tool schema, and compatible infrastructure. Rolling back weights alone may not restore behavior.

Open models can reduce dependency on one provider and enable inspection, localization, and private deployment. They do not eliminate product engineering. The teams that gain durable control operate a signed, evaluated, reproducible model factory—not a folder of downloaded checkpoints.

Source notes

Sources reviewed and current as of July 30, 2026: