Designing a Compliance Artifact Metadata Schema

This guide is part of the Compliance Metadata Standards section within the broader Core Architecture & Compliance Mapping framework, and it answers one narrow question: what metadata must every compliance artifact carry so that a filing produced today still validates, replays, and reconciles against its exact original contract three years from now?

An artifact — a budget reconciliation, an eligibility determination, an expenditure report — is only as auditable as the metadata wrapped around it. A payload with an ad-hoc dictionary of keys drifts silently: one service writes retention where another expects retention_class, a reviewer bolts on a note field that never gets stored, and two years later nobody can prove which schema the numbers were validated against. This guide fixes the metadata into a single closed contract: nine required fields, a pydantic v2 model that rejects anything extra, a semver version pinned per artifact so old filings replay against the exact contract that produced them, a controlled vocabulary for framework tags, and a canonical-JSON serializer whose output is byte-stable.

When to Use This Approach #

Design the schema this way when all three conditions hold:

• The artifact outlives the code that made it. Federal award records fall under a retention window set by 2 CFR §200.334 — three years from final report submission, longer under a litigation hold. A metadata format that only the current deployment can parse is worthless the moment you refactor; pinning schema_version per artifact is what lets a 2026 filing replay unchanged in 2029.
• The metadata is itself audit evidence. 2 CFR §200.302 requires financial records to identify the source and application of funds and to be traceable, and a SOC 2 engagement (Common Criteria CC7.2) expects that security-relevant events produce complete, tamper-evident records. If a reviewer can add or omit fields at will, the metadata cannot serve as evidence. The extra="forbid" contract in Step 2 is what closes that gap.
• Multiple producers write the same artifact type. A budget service, a batch reconciler, and a manual review tool all emit expenditure metadata. Without one enforced contract they drift on field names and types. A single frozen model is the only place that drift can be caught at write time.

What is explicitly out of scope here: this page defines the metadata contract, not the storage that makes it immutable, not the cryptographic signature over signer_id, and not the retention clock that acts on retention_class. The append-only ledger that persists these records lives in Audit Trail & Evidence Automation; the detached signature that binds an artifact to its signer is covered in Cryptographically Signing Grant Compliance Artifacts; and the retention scheduler that enforces the window lives in Automating 2 CFR 200 Record Retention Schedules. Any personally identifiable fields the artifact points at are governed by Securing PII in Nonprofit Grant Databases — this schema stores a digest of the subject, never the subject itself.

Step-by-Step Implementation #

The reference implementation targets Python 3.10+ and pydantic v2. Install exact pins so a replayed artifact validates against the same validator build that wrote it:

pydantic==2.7.1

One contract to internalize before any code: the metadata never carries the subject’s bytes, only a digest of them. subject_digest is a SHA-256 of whatever the artifact describes — a PDF, a JSON reconciliation, a CSV — so the record stays small, non-sensitive, and independently verifiable.

Step 1: Enumerate the required metadata fields #

Fix the field inventory first, before touching validation. Every compliance artifact carries exactly these nine fields, and the enumerations that constrain them are closed on purpose: an artifact type or retention class that is not on the list is a design decision, not a runtime surprise. Configure the standard-library logger here too — never print — because registration and replay both emit audit lines.

import logging
from enum import Enum
from typing import Final

logger: Final = logging.getLogger("compliance.metadata")
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
)

# The nine fields every compliance artifact must carry — no more, no fewer.
REQUIRED_FIELDS: Final[frozenset[str]] = frozenset({
    "artifact_id", "subject_digest", "artifact_type", "schema_version",
    "framework_tags", "donor_restriction", "retention_class",
    "signer_id", "created_at",
})


class ArtifactType(str, Enum):
    """Closed set of artifact kinds the pipeline is allowed to emit."""
    BUDGET_RECONCILIATION = "budget_reconciliation"
    ELIGIBILITY_DETERMINATION = "eligibility_determination"
    EXPENDITURE_REPORT = "expenditure_report"
    RETENTION_CERTIFICATE = "retention_certificate"


class RetentionClass(str, Enum):
    """Retention windows keyed directly to 2 CFR 200.334 categories."""
    FEDERAL_3YR = "federal_3yr"                 # default award record
    FEDERAL_LITIGATION_HOLD = "federal_hold"    # indefinite until released
    REAL_PROPERTY_3YR = "real_property_3yr"     # 3 yr after final disposition

retention_class maps to the specific categories in 2 CFR §200.334 — the default three-year window, the litigation-hold exception that suspends the clock, and the real-property rule — rather than storing a raw number of days, so the downstream retention scheduler reads intent, not an opaque integer.

Step 2: Model the contract as a strict pydantic v2 model #

Now bind those fields into one model. Three model_config choices carry the compliance weight: extra="forbid" rejects any key not on the contract, frozen=True makes an instance immutable once built so the record cannot mutate after it is signed, and str_strip_whitespace=True removes an entire class of “looks equal but isn’t” bugs. The semver format check on schema_version lives here too, because a malformed version can never pin to a real contract.

import re
from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field, field_validator

SEMVER_RE: Final = re.compile(r"^\d+\.\d+\.\d+$")


class ComplianceArtifactMetadata(BaseModel):
    """Immutable, closed metadata contract for one compliance artifact."""

    model_config = ConfigDict(extra="forbid", frozen=True, str_strip_whitespace=True)

    artifact_id: str = Field(..., min_length=1, description="UUIDv4 identifying this artifact")
    subject_digest: str = Field(..., min_length=64, max_length=64,
                                description="SHA-256 hex of the bytes this metadata describes")
    artifact_type: ArtifactType
    schema_version: str = Field(..., description="semver of the contract this record validates against")
    framework_tags: tuple[str, ...] = Field(..., min_length=1)
    donor_restriction: bool = Field(..., description="True when funds carry a donor-imposed restriction")
    retention_class: RetentionClass
    signer_id: str = Field(..., min_length=1)
    created_at: datetime

    @field_validator("schema_version")
    @classmethod
    def _validate_semver(cls, value: str) -> str:
        if not SEMVER_RE.match(value):
            raise ValueError("schema_version must be semver MAJOR.MINOR.PATCH")
        return value

Using a tuple rather than a list for framework_tags is deliberate: a frozen model needs hashable fields, and a tuple signals that the tag set is fixed at creation. Passing an unexpected key such as reviewer_note now raises a ValidationError at write time instead of being silently dropped — which is exactly what an auditor needs, since a dropped field is an invisible loss of evidence.

Step 3: Pin the schema version with semver for deterministic replay #

extra="forbid" protects one version. Replay across versions needs a registry: every contract that ships is registered under its semver string and is then immutable — you never edit 1.1.0 in place, you release 1.2.0. An archived filing carries the version it was written under, and replay resolves that string back to the exact model class.

CURRENT_SCHEMA_VERSION: Final[str] = "1.1.0"

# Each shipped contract version pins to the model class that defined it.
CONTRACT_REGISTRY: Final[dict[str, type[BaseModel]]] = {}


class ContractNotFound(LookupError):
    """Raised when a filing pins a schema_version absent from the registry."""


def register_contract(version: str, model: type[BaseModel]) -> None:
    """Register an immutable contract version; re-registration is rejected."""
    if not SEMVER_RE.match(version):
        raise ValueError(f"schema_version must be semver, got {version!r}")
    if version in CONTRACT_REGISTRY:
        raise ValueError(f"contract {version} already registered; shipped versions are immutable")
    CONTRACT_REGISTRY[version] = model
    logger.info("registered compliance contract version=%s model=%s", version, model.__name__)


def replay_artifact(raw: dict[str, object]) -> BaseModel:
    """Validate an archived payload against the exact contract it was written under."""
    version = raw.get("schema_version")
    if not isinstance(version, str):
        raise ValueError("payload is missing a string schema_version; cannot select a contract")
    model = CONTRACT_REGISTRY.get(version)
    if model is None:
        logger.error("no registered contract for schema_version=%s", version)
        raise ContractNotFound(f"schema_version {version!r} is not registered")
    return model.model_validate(raw)


register_contract(CURRENT_SCHEMA_VERSION, ComplianceArtifactMetadata)

The ContractNotFound raise is the important guardrail: if a filing pins a version you no longer ship, replay fails loudly rather than quietly validating against today’s contract and reporting numbers under rules that did not exist when the artifact was created. Keeping retired model classes registered for the full 2 CFR §200.334 retention window is what makes a three-year-old expenditure report replayable.

Step 4: Validate framework_tags against a controlled vocabulary #

framework_tags says which regulatory regimes an artifact answers to, and free text there is worthless for later filtering — SOC2, soc 2, and SOC-2 would fragment your evidence. Pin the vocabulary to a closed set and reject anything outside it, along with duplicates.

FRAMEWORK_VOCABULARY: Final[frozenset[str]] = frozenset({
    "2CFR200",   # Uniform Guidance financial management
    "IRS990",    # annual information return
    "SOC2",      # SOC 2 Common Criteria
    "FASB958",   # not-for-profit financial statements
    "STATE_AG",  # state charity registration
})


@field_validator("framework_tags")
@classmethod
def _validate_framework_tags(cls, tags: tuple[str, ...]) -> tuple[str, ...]:
    unknown = sorted(t for t in tags if t not in FRAMEWORK_VOCABULARY)
    if unknown:
        raise ValueError(
            f"framework_tags outside controlled vocabulary: {unknown}; "
            f"allowed: {sorted(FRAMEWORK_VOCABULARY)}"
        )
    if len(set(tags)) != len(tags):
        raise ValueError("framework_tags must be unique")
    return tags

Add this validator inside ComplianceArtifactMetadata alongside the semver check from Step 2. Growing the regime list is a real schema change — a new tag means a new registered schema_version, not an in-place edit to a shipped contract, which keeps the vocabulary itself replayable.

Step 5: Emit the artifact as canonical JSON #

Two byte-for-byte identical records must serialize to identical bytes on any host, in any process, or the fingerprint you store in the ledger is meaningless. Canonical JSON gets you there: sort keys, drop insignificant whitespace, and keep Unicode intact. pydantic’s model_dump(mode="json") renders the datetime to ISO-8601 and enums to their string values first.

import hashlib
import json


def to_canonical_json(record: ComplianceArtifactMetadata) -> bytes:
    """Serialize to deterministic, whitespace-free, key-sorted JSON bytes."""
    payload = record.model_dump(mode="json")
    return json.dumps(
        payload,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    ).encode("utf-8")


def artifact_fingerprint(record: ComplianceArtifactMetadata) -> str:
    """SHA-256 over canonical JSON — the value an append-only ledger stores."""
    return hashlib.sha256(to_canonical_json(record)).hexdigest()

sort_keys=True and separators=(",", ":") are the two settings that make the output canonical; ensure_ascii=False keeps accented funder names as real UTF-8 rather than \uXXXX escapes, so the encoding is stable regardless of locale. The artifact_fingerprint is what a downstream ledger stores and what a replayed filing must reproduce exactly.

Verification #

Confirm the contract behaves deterministically with four checks:

1. The closed contract rejects extra fields. Add a stray key to a valid payload and assert model_validate raises ValidationError. This proves extra="forbid" is enforcing the field inventory rather than silently dropping the key.
2. The framework vocabulary is enforced. Pass a tag outside FRAMEWORK_VOCABULARY and assert the raised error names the offending tag, so an operator sees why the write failed.
3. Unknown versions fail replay loudly. Call replay_artifact with a schema_version that is not registered and assert it raises ContractNotFound — never a silent fallback to the current contract.
4. Canonical JSON round-trips byte-for-byte. Dump a record, replay it from that dump, and assert the two canonical serializations are identical bytes. That equality is the proof an auditor uses to bind a stored fingerprint to exactly one record.

from pydantic import ValidationError


def _valid_payload() -> dict[str, object]:
    return {
        "artifact_id": "8f14e45f-ceea-467a-9d0b-1c7a2f3e4b5c",
        "subject_digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
        "artifact_type": "expenditure_report",
        "schema_version": "1.1.0",
        "framework_tags": ["2CFR200", "SOC2"],
        "donor_restriction": True,
        "retention_class": "federal_3yr",
        "signer_id": "svc-reconciler-01",
        "created_at": "2026-06-16T14:03:00Z",
    }


def test_extra_field_is_rejected() -> None:
    payload = _valid_payload()
    payload["reviewer_note"] = "looks fine"
    try:
        ComplianceArtifactMetadata.model_validate(payload)
        raise AssertionError("extra field should have raised ValidationError")
    except ValidationError:
        pass


# End-to-end: build, replay from its own dump, and prove byte equality.
record = ComplianceArtifactMetadata.model_validate(_valid_payload())
replayed = replay_artifact(record.model_dump(mode="json"))
assert to_canonical_json(record) == to_canonical_json(replayed)
assert artifact_fingerprint(record) == artifact_fingerprint(replayed)
assert record.schema_version in CONTRACT_REGISTRY

A clean run emits one INFO line at registration and nothing else; a rejected write raises before any bytes are persisted. Ship the fingerprint and canonical JSON to a write-once tier so the record satisfies the retention period under 2 CFR §200.334.

Common Errors & Fixes #

Related #

• Parent section: Compliance Metadata Standards
• Where these records are persisted immutably and replayed as evidence: Audit Trail & Evidence Automation
• How the subject a digest points at stays protected: Securing PII in Nonprofit Grant Databases