Colloq GitHub

Runtime experiment

The first Colloq runtime tests one claim: a global conversation can remain semantically identical while its projected endpoints execute through different transport plans.

It is deliberately a reference implementation rather than a performance architecture.

Execution model

The runtime compiles a Conversation v0 graph into one verified Colloq Plan containing both projected endpoints. Each role creates a lightweight session over its shared immutable endpoint graph. No central coordinator advances the conversation.

demo compiles once before starting either role. compile and run-plan make that boundary explicit and reusable across executions; see Colloq Plan v0.

For every transition, the sending endpoint:

  1. Checks that its projected state permits the action.
  2. Creates a versioned Colloq Wire envelope.
  3. Records the conversation name, experimental semantic hash, state, and sequence.
  4. Advances only its local endpoint machine.

The receiving endpoint independently checks all envelope fields and the permitted receive action before advancing. It rejects stale, reordered, graph-incompatible, or protocol-invalid frames.

Reference mode serializes that complete semantic envelope. Compact mode resolves the checked frame through the plan's deterministic transition dictionary and sends {t, q, p?}: transition ID, sequence, and optional data payload. The receiver reconstructs the complete envelope from its verified plan before running the same checks. See Colloq Wire v0.

Before TCP, QUIC, or Iroh frame zero, both roles exchange an Colloq Session Preface containing session version, conversation and plan identity, role, and exact encoding. Either side aborts on any mismatch; network transports refuse frame I/O until the preface succeeds. Iroh sessions additionally bind the authenticated endpoint IDs and a TLS exporter from the concrete connection.

The SHA-256 value is an experimental semantic identity. It excludes the schema path and annotations, but it is not yet an Colloq ContentId; canonicalization fixtures and normalization rules must exist before that name is justified.

Transport plans

Memory

The memory plan executes client and server endpoints on separate threads connected by channels. Envelopes cross a real serialization boundary rather than being passed as Rust objects.

cargo run -- demo --transport memory --tokens 5 --cancel-after 2

TCP

The TCP plan uses the selected reference or compact JSON envelope with a four-byte big-endian length prefix. Loopback mode makes transport equivalence easy to test:

cargo run -- demo --transport tcp --tokens 3

The roles can also run as separate operating-system processes:

# Process 1
cargo run -- serve --wire compact \
  --listen 127.0.0.1:7878 --tokens 4

# Process 2
cargo run -- connect --wire compact \
  --server 127.0.0.1:7878 --cancel-after 2

Both sides compile the supplied conversation deterministically, exchange plan identities, roles, and exact wire encodings, and reject a mismatch before sending a semantic frame. This TCP check is not authenticated against an active attacker.

QUIC

The QUIC plan carries the selected reference or compact envelope over one bidirectional QUIC stream. TLS encrypts transport data, and the client pins the exact public certificate generated by the server.

cargo run -- demo --transport quic --tokens 3

Separate processes use the public certificate as their explicit trust handoff:

# Process 1
cargo run -- serve-quic --listen 127.0.0.1:7879 \
  --wire compact --certificate-out build/colloq-quic-cert.der --tokens 4

# Process 2
cargo run -- connect-quic --server 127.0.0.1:7879 \
  --wire compact --certificate build/colloq-quic-cert.der --cancel-after 2

The generated private key remains in the server process. The client pins that server certificate, then verifies the session preface inside the authenticated TLS channel. This binds the server's declared plan, role, and encoding to the pinned key. The public certificate is regenerated for each server invocation; persistent identity, client authentication, authorization, and certificate rotation are intentionally deferred.

After the Colloq conversation reaches end, the QUIC plan performs a role-ordered close handshake outside the semantic trace. This ensures each process observes the peer's completion before either endpoint releases its QUIC connection.

Iroh

The Iroh plan carries the same reference or compact envelope over a bidirectional QUIC stream with ALPN colloq/0.1. Each node is created from a persistent Ed25519 secret key, and the caller supplies the exact remote Endpoint ID it expects. A different peer is rejected before Colloq session setup.

cargo run -- demo --transport iroh --tokens 3
cargo run -- run-plan build/generate.colloqplan.json \
  --transport iroh --wire compact --tokens 3

The reference demo uses direct loopback addresses and disables relays, matching a routable data center network. The library also exposes Iroh's discovery, NAT-traversal, and relay-capable default endpoint. During session setup, each peer checks that the preface's endpoint identity matches the key authenticated by Iroh and that its channel_binding matches a TLS exporter derived with the plan identity. A copied preface therefore cannot authenticate a different connection.

Persistent *.colloqnode.json files keep the private key local, public *.colloqendpoint.json tickets carry direct addresses, and *.authorization.json policies map authenticated peer IDs to exact roles and plan identities. serve-iroh and connect-iroh run the endpoints as independent processes; verify-session compares their saved reports. A plan change is fail-closed until each operator grants its new identity explicitly. See the two-node runbook.

Each report includes a semantic trace hash. Equivalent client and server hashes mean both independently projected endpoints observed the same ordered protocol transitions. Tests require reference and compact memory, TCP, QUIC, and Iroh sessions to produce the same hash for identical successful inputs.

Typed failure and deterministic faults

Conversation v0 declares failure IDs separately from states. A communication state maps an applicable failure ID to a terminal fail state. The v0 checker rejects an unknown failure ID or an edge whose target does not terminate with that same failure. Recovery and retry transitions are intentionally deferred; this slice first makes failure meaning explicit.

The example declares transport.closed, transport.timeout, transport.reset, transport.unreachable, and transport.uncertain. The reference runtime classifies applicable I/O errors into the first four categories. uncertain represents a local knowledge state: an endpoint cannot prove how far its peer progressed.

The memory fault plan makes this behavior reproducible. It fails one exact transport operation selected by endpoint role, send or receive, and a one-based occurrence:

cargo run -- fault-demo --fault-role server \
  --fault-operation send --fault-at 2 \
  --failure transport.timeout \
  --peer-failure transport.uncertain

Injecting the server's second send records transport.timeout at the server and closes the simulated link. The client records transport.uncertain because it cannot know whether the peer advanced before the loss. Both traces are valid, but their terminal states and hashes differ. Reports distinguish semantic_trace_equivalent from outcome_equivalent rather than inventing agreement that the transport cannot provide.

The report includes the exact fault plan. The command fails if the selected occurrence is never reached, preventing a silent successful run from masquerading as an injected counterexample.

This does not solve distributed agreement. The deterministic memory injector assigns the selected local observations so asymmetric semantics can be tested repeatably. It does not yet model time, packet-level partial delivery, recovery, or a real network partition.

Reference benchmark

The benchmark command compares the whole Colloq memory path with a hand-written request/token/done protocol using the same basic threads, channels, and JSON boundary. It currently exposes rather than hides the cost of the reference machinery:

cargo run --release --locked -- benchmark \
  --iterations 500 --warmup 50 --tokens 3

See the benchmark protocol, initial result, and limitations.

Security and correctness boundary

The prototype currently guarantees only:

  • local action checking against the projected endpoint;
  • conversation-name and semantic-hash agreement;
  • exact state and sequence agreement;
  • verified reusable plan identity and structurally valid projected state references;
  • deterministic, plan-bound compact transition IDs;
  • fail-closed session-version, conversation, plan, role, and encoding agreement before network frame zero;
  • a maximum envelope size;
  • explicit selection and cancellation transitions;
  • declared terminal transport-failure transitions;
  • deterministic role/operation/occurrence fault injection;
  • equivalent successful semantic traces across both encodings and all four implemented transports;
  • explicit asymmetric timeout/uncertainty observations under deterministic faults;
  • QUIC transport encryption and pinned server authentication;
  • binding of the QUIC server's plan preface to that authenticated connection;
  • rejection of a QUIC server presenting an untrusted certificate;
  • persistent mutually authenticated and exact role/plan-authorized Iroh endpoint identities;
  • Iroh session-preface binding to the concrete TLS connection and plan identity.

It does not yet provide:

  • common identity provisioning, rotation, or revocation distribution across transports;
  • asynchronous multiplexing or flow control;
  • retries, reconnects, recovery branches, or distributed failure agreement;
  • enforcement of declared deadlines;
  • structural validation of payloads from Colloq type definitions;
  • canonical graph normalization;
  • bulk tensor transfer or zero-copy buffers;
  • replay-resistant session nonces, resumption, or persistent connection pooling;
  • a binary payload codec or zero-copy performance architecture.

The TCP server listens on loopback by default because that wire plan is plaintext and unauthenticated. QUIC is encrypted, but its generated certificate is suitable only for this explicit pinning experiment. Iroh authenticates both endpoint keys; the local Colloq authorization policy controls admission and roles.

Current result and next experiment

Memory, TCP, QUIC, and Iroh preserve the same successful semantic trace across reference and compact encodings. Independent TCP, QUIC, and Iroh processes exchange the compact representation after a strict plan-bound preface. QUIC authenticates the server and binds its declared plan to the pinned TLS connection; Iroh authenticates both persistent endpoint keys, applies exact role/plan authorization, and binds the session to its concrete TLS connection. Conversations compile into reusable, identified endpoint plans with deterministic transition dictionaries. Incremental Automerge synchronization also runs as a projected Colloq conversation over authenticated Iroh. Compact encoding improved the checked-transition median by 1.32× and the complete warm exchange by 1.11×, but remains 1.51× the hand-written baseline. Deterministic failures preserve different local timeout and uncertainty observations.

The next runtime experiment should:

  1. Define endpoint-key rotation, revocation distribution, and replay-resistant freshness across QUIC and Iroh.
  2. Measure preface exchange latency, protocol-machine checks, transition lookup, allocation, encoding, channel transfer, and scheduling separately.
  3. Add deadline enforcement and recovery transitions rather than terminal failures only.
  4. Exercise asymmetric faults and the conventional baseline across independent processes and a controlled multi-node testbed.

Edit this page on GitHub