Ports & protocols

The kernel is a mediator over five ports plus a KindPort. Each is a typing.Protocol — a host wires concrete adapters (filesystem, SQLite, Postgres, …) into the kernel that satisfy these contracts. What every source adapter must implement is walked through narratively in How to write a source adapter; the exhaustive contract is below.

SourcePort

dna.kernel.protocols.SourcePort

Bases: Protocol

WHERE — load instances from storage.

supports_readers property

supports_readers: bool

Whether this source uses ReaderPort plugins to detect bundles.

Filesystem sources return True (they walk directories). Database sources return False (instances are self-contained JSON). Used by Kernel.auto() to decide whether to wire cache + resolvers.

load_bootstrap_docs async

load_bootstrap_docs(
    scope: str, *, tenant: str | None = None
) -> list[dict[str, Any]]

Return the docs the kernel needs registered/parsed BEFORE load_all fires.

Phase 16 — replaces the cardinality-1 load_manifest contract with a list of bootstrap-priority docs. Generalizes for multiple "system" Kinds:

• KindDefinition (dna.kind/v1) — custom Kind schemas. Registered first so subsequent parsing has the types available.
• LayerPolicy (github.com/ruinosus/dna/policy/v1) — overlay rules read by the kernel at write-time enforcement.
• Genome (github.com/ruinosus/dna/v1) — scope-root identity Kind. Used by mi.root and dependency resolution (Genome.spec.dependencies).

Adapters that can filter cheaply (SQL WHERE kind IN (...)) SHOULD do so. Adapters that scan everything (filesystem) MAY return a superset — the kernel filters defensively.

Tenant semantics: when tenant is set, the tenant-published Genome SHADOWS the platform Genome (Phase 9 multi-tenant publishing). KindDefinition + LayerPolicy stay platform-only (non-overlayable per Phase 16).

list_doc_refs async

list_doc_refs(
    scope: str,
    *,
    kind: str | None = None,
    tenant: str | None = None
) -> list[tuple[str, str]]

Lista (kind, name) de todos os docs do scope. Filtrável por kind. Retorna metadata only — sem bundle entries, sem parse.

Custo esperado: - Postgres: 1 SELECT indexed → 10-20ms para 1510 rows. - Filesystem: directory walk sem parse → 30-50ms. - SQLite: 1 SELECT indexed → 5-10ms.

Tenant: quando tenant é passado, retorna a união do base layer com o overlay (tenant-published shadows platform). None (default) = base only.

load_one async

load_one(
    scope: str,
    kind: str,
    name: str,
    *,
    readers: list[ReaderPort] | None = None,
    tenant: str | None = None
) -> dict[str, Any] | None

Carrega UM doc específico com seu bundle (se aplicável). Retorna o raw dict (kind, name, spec, metadata) ou None se não encontrado.

Custo esperado: - Postgres: 1-2 SELECTs (content + bundle_entries) → 5-10ms. - Filesystem: parse de 1 bundle → 10-30ms. - SQLite: 1-2 SELECTs → 3-8ms.

Tenant: idem list_doc_refs. Overlay shadows base quando ambos existem.

api_version resolves the Kind EXACTLY. A Kind is identified by (apiVersion, kind), and a store whose row key carries that identity (SourceCapabilities.api_version_identity) can really hold two workspaces' Deal instances under one name — a bare lookup then answers with whichever row comes back first. Optional, and declared through that capability: an adapter whose identity is mediated otherwise (the filesystem's <container>/<name> path, resolved through the StorageDescriptor registry) does not accept it, and the kernel does not pass it.

query async

query(
    scope: str,
    kind: str,
    *,
    filter: QueryFilter | None = None,
    projection: QueryProjection | None = None,
    limit: int | None = None,
    offset: int | None = None,
    order_by: QueryOrder | None = None,
    tenant: str | None = None
) -> AsyncIterator[dict[str, Any]]

Push-down query sobre o storage do scope.

Args: scope: identificador do scope (ex: dna-development). kind: nome do Kind (ex: Story). Multi-kind queries não são suportadas em v1 — chame múltiplas vezes em asyncio.gather (ou use o endpoint composto HTTP). filter: QueryFilter — dict de field_path → value. Veja a docstring de QueryFilter para shape + operadores. None = sem filtro. projection: QueryProjection — lista de field_paths para incluir em cada row. None = retorna raw dict completo (mesmo shape de load_one). Quando projection é dada, rows são objects com apenas os campos pedidos (mais name sempre presente). limit: máximo de rows a retornar. None = sem limite. Recomendado SEMPRE passar limit em hot paths para proteger contra scopes que crescem. offset: paginação. Usado com limit. Default 0. order_by: QueryOrder. None = ordem natural do adapter (insertion order para FS; índice padrão para SQL). Use "-spec.field" para descending. tenant: overlay aware. None = base layer only. Slug = união base + overlay com overlay shadowing base (mesma semântica de load_one + load_layer).

Returns: AsyncIterator[dict] — itera rows. Caller pode materializar via [r async for r in source.query(...)] ou consumir preguiçosamente.

Raises: QueryError — filter/projection/order_by malformado em forma detectável estaticamente (operador inválido, path com sintaxe quebrada). Adapters podem levantar erros runtime (timeout, conexão) — esses propagam como exceções nativas do backend.

Custo esperado (1500 docs no scope, filter+limit típicos): - Postgres: 1 SELECT indexed → 5-30ms. - Filesystem: glob + parse + filter em Python → 30-150ms. - SQLite: 1 SELECT indexed → 3-15ms.

Examples: # List view: 50 Stories in-progress mais recentes async for row in source.query( "dna-development", "Story", filter={"status": "in-progress"}, projection=["name", "spec.title", "spec.feature"], order_by=["-spec.updated_at"], limit=50, ): ...

# Cross-tenant: union base + acme overlay
async for row in source.query(
    "hr-screening", "Agent",
    tenant="acme",
):
    ...

# Single-field count helper (materialized)
rows = [r async for r in source.query(
    scope, "Issue", filter={"status": "open"},
    projection=["name"],
)]
assert len(rows) == open_count

Implementations: Concrete adapters (Postgres / SQLite / Filesystem) ship impls in parity and declare query_pushdown=True in their SourceCapabilities. Sources without a native impl are served by the kernel via the load_all fallback in dna.kernel.query.fallback.query_via_load_all (s-sourceport-contract-cleanup: the fallback used to be a ~60-line concrete body HERE, reaching back into the mediator via getattr(self, "_kernel") — the Protocol now declares the signature only).

count async

count(
    scope: str,
    kind: str,
    *,
    filter: QueryFilter | None = None,
    group_by: str | None = None,
    tenant: str | None = None
) -> dict[str, Any]

Aggregation push-down (two-planes F2, spec D2): total de docs que casam o filter, opcionalmente agrupados por um field_path (group_by, mesma convenção do QueryFilter — ex.: spec.status).

Returns: {"total": int, "groups": list[{"key", "count"}] | None} — groups ordenados por count DESC; key None agrupa docs sem o campo.

Fallbacks vivem em dna.kernel.query.fallback (count_via_query ride o query do adapter; count_via_load_all é o caminho kernel-side p/ sources sem query nativo). Adapters SQL fazem override com SELECT count(*) … GROUP BY nativo.

WritableSourcePort

dna.kernel.protocols.WritableSourcePort

Bases: SourcePort, Protocol

SourcePort with write + versioning capabilities.

Phase 2a (tenant first-class): tenant is now a first-class parameter on save/delete. Adapters route tenant-scoped writes to physically isolated storage (e.g. tenants/<X>/scopes/<S>/); layer is reserved for non-tenant overlays (branch, region, user) — when both are passed the adapter combines them.

save_instance async

save_instance(
    scope: str,
    kind: str,
    name: str,
    raw: dict,
    author: str | None = None,
    *,
    tenant: str | None = None,
    layer: tuple[str, str] | None = None,
    write_class: str = "substantive",
    version_retention: int | None = None,
    if_absent: bool = False,
    if_match: str | None = None,
    edges: list | None = None
) -> str

Persist one instance (an UPSERT by default).

if_absent=True makes it an ATOMIC CREATE: the write claims the name or raises dna.kernel.errors.InstanceNameTaken, never overwrites. Read-then-write cannot give that guarantee — two callers can both read "free" in the same instant — and the two shipped adapters can: a composite primary key on the SQL side, O_CREAT|O_EXCL (files) or mkdir (bundles) on the filesystem. Optional, declared through SourceCapabilities.write_kwargs.

if_match is the same guarantee for an UPDATE (i-083): the write proceeds only if the STORED instance's spec still hashes to the given token (:func:dna.kernel.etag.spec_etag), else :class:dna.kernel.errors.StaleInstanceWrite, and nothing is written. The adapter must read the stored instance ITSELF — reading it back through a cache the caller already consulted would compare a stale value against itself and agree, which is exactly the lost update i-083 measured. Also optional and declared the same way. if_absent and if_match are mutually exclusive: one asserts the instance is absent, the other that it is present and unchanged.

edges carries the DERIVED reference graph of this instance — the x-dna-ref targets the write path already resolved while validating them (:func:dna.kernel.query.references.resolve_declared_edges). An adapter that declares the kwarg replaces this instance's outgoing edges INSIDE the same transaction as the instance itself, so the two become true together; one that does not declare it is never handed them, and the graph face answers unsupported for that store rather than an empty edge list.

Three values, three meanings, and the middle one is load-bearing: a LIST replaces the stored edges (an empty list therefore REMOVES them, which is how an instance that stopped referencing anything stops having relations); None means the kernel has nothing trustworthy to say — the producer is off, or a read failed part-way — and the stored edges must be LEFT ALONE, because an old known-good edge set beats a fresh partial one.

delete_instance async

delete_instance(
    scope: str,
    kind: str,
    name: str,
    *,
    tenant: str | None = None,
    layer: tuple[str, str] | None = None,
    api_version: str | None = None
) -> None

Delete one instance.

api_version resolves the Kind EXACTLY. A delete carries no instance, so it used to route by bare Kind NAME — and two workspaces may each declare a Deal under their own namespace, which is precisely what namespacing the apiVersion is for. A bare lookup then resolves whichever port the registry returns first, so the delete can look inside the OTHER Kind's container. The blast radius is a delete that MISSES, never one that reaches another workspace (the scope, not the container, is the isolation boundary) — but a delete that silently does nothing is its own failure, and the caller is told it succeeded.

Optional, and declared through SourceCapabilities.delete_kwargs: an adapter that does not accept it keeps working, and the kernel simply does not pass it.

CachePort

dna.kernel.protocols.CachePort

Bases: Protocol

WHERE — store/retrieve installed deps.

ResolverPort

dna.kernel.protocols.ResolverPort

Bases: Protocol

FROM — fetch external deps.

ReaderPort

dna.kernel.protocols.ReaderPort

Bases: Protocol

Reads a bundle and produces a raw dict.

Phase 8 (PR1) — detect and read now receive a BundleHandle instead of a pathlib.Path. The handle abstracts over filesystem, Postgres, S3, in-memory dict — same reader works regardless of where the bundle lives. See dna.kernel.bundle.handle.

Backward-compat: BundleHandle.path returns the underlying filesystem Path when FS-backed (None otherwise) — escape hatch for code that genuinely needs Path semantics. New readers SHOULD use handle.read_text(...) / handle.iter_entries(...).

Implementations MUST inherit this Protocol explicitly (class MyReader(ReaderPort)) — the same convention source adapters follow (s-dna-source-conformance-kit). Inheriting also provides the _owner_container default below.

WriterPort

dna.kernel.protocols.WriterPort

Bases: Protocol

Writes a raw dict back to a bundle. Inverse of ReaderPort.

Phase 8 (PR1) — write receives a BundleHandle instead of Path; same source-agnostic contract.

s-dna-rw-roundtrip-suite — serialize is part of the contract (it was load-bearing but informal: kernel.serialize_instance consumed it via hasattr, so a Protocol-conforming writer could silently miss it and only fail at emission time).

Implementations MUST inherit this Protocol explicitly (class MyWriter(WriterPort)) and keep write and serialize COHERENT: write(bundle, raw) must produce exactly the entries serialize(raw) returns (the canonical implementation is write_entries_to_handle(bundle, self.serialize(raw)) from dna.kernel.write.helpers). The round-trip conformance suite (dna.testing.reader_writer_conformance_suite) enforces this equivalence for every registered pair.

serialize

serialize(raw: dict) -> list[dict[str, Any]]

Return the bundle entries write would emit, WITHOUT writing.

Each entry is {"relativePath": str, "content": str} for text payloads or {"relativePath": str, "content_bytes": bytes} for binary ones (see writer_helpers.pop_source_files_as_entries). TS twin: WriterPort.serialize returning SerializedFile[].

The Protocol body raises on purpose: a writer that inherits WriterPort without overriding serialize would otherwise silently return None and only break at emission time — the exact failure mode this member was formalized to kill.

KindPort

dna.kernel.protocols.KindPort

Bases: Protocol

WHO — identity + composition role.

This runtime_checkable Protocol lists ONLY the core contract every Kind must provide — it is exactly what the H1 registration gate (kernel.kind → isinstance(k, KindPort)) enforces.

The optional presentation/UX surface (docs, ui_schema, graph_style, ascii_icon, display_label, presentation, description_fallback_field, visible_in_backend, preview(), graph_meta()) lives on the separate KindPresentation capability Protocol below — declared there so it is typed + documented, but NEVER required by the isinstance check (s-dna-kindport-descriptor-schema).

.. warning:: do NOT add optional members to THIS Protocol body. runtime_checkable isinstance checks member PRESENCE — a new member here silently breaks registration of every third-party Kind that doesn't declare it (the is_runtime_artifact precedent — see test_port_contract.py). Optional surface goes on KindPresentation (or a new capability Protocol) instead.

dependencies

dependencies() -> dict[str, str] | None

Which spec fields reference other kinds by alias.

Clearer name for dep_filters(). Default implementation delegates to dep_filters() so existing extensions stay compatible.

schema

schema() -> dict[str, Any] | None

JSON Schema for this kind's spec.

Supporting types

dna.kernel.protocols.Extension

Bases: Protocol

Registers kinds, readers, and writers on the Kernel.

kernel.load(ext) fail-loud validates the whole contract before calling register(): name must be a non-empty str, version a str, register callable (ExtensionLoadError otherwise). register() receives the registration-time host slice — see :class:ExtensionHost for the exact vocabulary.

Optional capabilities (feature-tested, NOT required Protocol members so legacy extensions predating Phase 0 keep working):

• :class:TemplateProvider — def templates(self) -> list[Template]. When present, Kernel.list_templates() aggregates entries from every loaded extension so UIs (Tauri Studio, CLI) can offer scaffold() for any extension-shipped file tree. See dna.kernel.compose.templates.Template for the payload shape.
• :class:ManifestActivator — def activate_manifest(self, mi, kernel). When present, Kernel.run_manifest_activators(mi) calls it so the extension can turn its OWN declared instances into runtime behaviour (a Hook onto the HookRegistry, a SafetyPolicy into a masking middleware). This is where the schema of an extension's Kind gets read; it must not be read in the kernel.

dna.kernel.protocols.Template dataclass

Template(
    id: str,
    label: str,
    kind: str,
    description: str,
    files_root: Path,
    owner_extension: str,
    post_init_hint: str | None = None,
    upstream_ref: str | None = None,
)

A scaffoldable file tree declared by an Extension.

id Namespaced identifier: "/". label Human-friendly name shown in UIs. kind Primary Kind this template scaffolds (may span multiple kinds in the file tree, but this is the headline one for filtering/grouping). description One-line description. files_root Absolute Path to the root of the template tree on disk (resolved via importlib.resources by the Extension). owner_extension Name of the Extension that owns this template. post_init_hint Optional shell/cli snippet shown after scaffold (e.g. "cd .dna//programs/research && uv sync"). upstream_ref Optional upstream pin (e.g. a git sha of the source repo the template was cloned from).

dna.kernel.protocols.ToolDefinition dataclass

ToolDefinition(
    name: str,
    group: str | None = None,
    description: str = "",
    summary: str = "",
    args_schema: dict[str, Any] = dict(),
    hitl: bool = False,
    scope: str | None = None,
    source: str = "",
    _callable: Any = None,
)

Concrete ToolPort implementation. Stored in kernel._tools.

The _callable field holds the langchain StructuredTool; never serialize or wrap-replace it — downstream langgraph / deepagents consume it directly. get_callable() is the canonical accessor.

dna.kernel.protocols.StorageDescriptor dataclass

StorageDescriptor(
    container: str,
    pattern: StoragePattern,
    marker: str | None = None,
    body_as: BodyMode | None = None,
    body_field: str | None = None,
    body_parser: (
        Callable[[str], dict[str, Any]] | None
    ) = None,
)

Declares how a kind's instances are stored on the filesystem.

Used by writers, readers, and the Studio UI to understand the canonical layout for each kind without hardcoded maps.

bundle classmethod

bundle(
    container: str,
    marker: str,
    body_as: BodyMode = BodyMode.TEXT,
    body_field: str = "instruction",
) -> "StorageDescriptor"

Bundle directory containing a marker file (e.g. SKILL.md).

yaml classmethod

yaml(container: str) -> 'StorageDescriptor'

Plain YAML files inside a container directory.

root classmethod

root(
    filename: str = "manifest.yaml",
) -> "StorageDescriptor"

Single root file at the module root.

standalone classmethod

standalone(
    filename: str,
    body_as: BodyMode = BodyMode.TEXT,
    body_field: str = "content",
) -> "StorageDescriptor"

Standalone file at the module root (e.g. AGENTS.md).

dna.kernel.protocols.StoragePattern

Bases: StrEnum

How a kind's instances are laid out on the filesystem.