# UFN — USE Frame Notation (full specification)

Standards Track · kova ecosystem · Apache 2.0 · v2 (stable, locked)
Canonical HTML: https://ufn.kovanex.dev/

This is the complete UFN specification in plain text, intended for single-fetch
ingestion by LLM agents. UFN's producers and consumers are agents; this file IS the
standard, written for them.

================================================================================
ABSTRACT
================================================================================

UFN (USE Frame Notation) is a one-line, dictionary-driven notation for structured
output produced by Large Language Models. Where JSON spends a per-token tax on
{ } [ ] " , on every structured reply, UFN moves structure into named, self-delimiting
tags and a tight separator, so a model spends its tokens on content, recovers cleanly
from truncation, and streams its header before its prose.

UFN is for the EDGE between a model and a consumer — not for storage. Storage stays
JSON; UFN is the token-economical wire format an LLM emits and a parser ingests. On a
representative task list it is ~59% smaller than compact JSON on the structural
portion, turns parse-failure-on-truncation from total loss into a usable partial
frame, and cuts time-to-first-content from tens of seconds to ~1.5 s by streaming the
header ahead of the prose.

================================================================================
1. INTRODUCTION
================================================================================

UFN is the smallest legal notation that gives a producer tagged records plus a free
prose tail on a single line. The conceptual model is 1990s SMS / ICQ / IRC
compression: sender and receiver share a tiny convention dictionary, every byte
counts, and a partial message still makes sense.

1.1. Motivation — three costs of JSON for LLM output

- Token tax on syntax. A four-field header can cost ~110 tokens in JSON against ~38
  in UFN — 30-65% of a short reply spent on braces, quotes, commas, not content.
- Truncation = total failure. A single missing } makes the whole reply unparseable;
  ~20% of measured benchmark runs hit the fallback path purely on a missing brace.
- Hostile to streaming. JSON closes at the end; nothing renders until the last
  bracket balances.

1.2. Requirements language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHOULD, RECOMMENDED, MAY, OPTIONAL are
to be interpreted as in BCP 14 (RFC 2119) when, and only when, in all capitals.

1.3. Design scope (NORMATIVE)

UFN is a wire/edge format, not a storage format. Producers and consumers SHOULD keep
their system of record in JSON and emit UFN only at the LLM edge. UFN saves STRUCTURE,
never content: a UUID or long title is incompressible.

================================================================================
2. NOTATION MODEL
================================================================================

Everything is ONE LINE, ;-separated. A token is classified by its SHAPE alone — three
kinds:

  SHAPE                                KIND          MEANING
  key:value (contains ':')             field         scalar record; split on the
                                                     FIRST ':'; value may contain ':'
  acr (no ':', no leading '/')         scope open    opens a container of class acr
  /                                    scope close   closes nearest open scope (stack);
                                                     '/acr' is optional resync mode

2.2. Fields — key:value. The key MUST come from the shared dictionary; the value is
free text to the next separator. Field separator is ';' (tight) at the general edge;
the v1 game profile uses ' ; ' (with spaces).

2.3. Scopes and nesting — a bare non-field token OPENS a scope; a bare '/' CLOSES the
top of the stack. An unclosed scope at end-of-line is closed implicitly, so a
truncated object is still usable.

2.4. Arrays vs objects — a class is array- or object-typed BY THE DICTIONARY, not by
punctuation. An array scope holds repeated item scopes.

2.5. Prose tail — everything after the FIRST top-level ' ::: ' (triple colon with
surrounding spaces) is VERBATIM prose to end-of-line: no escaping, no structure. This
is the streaming seam: emit the header, then stream prose token by token.

================================================================================
3. GRAMMAR (ABNF)
================================================================================

  line     = element *( ";" element ) [ " ::: " prose ]
  element  = field / open / close
  field    = key ":" value
  open     = acr                  ; bare token, no ":", not starting "/"
  close    = "/" [ acr ]          ; bare "/" closes top-of-stack;
                                  ; named "/acr" = robustness/resync mode
  key      = token                ; from the shared dictionary
  acr      = token                ; class name from the shared dictionary
  value    = *( vchar / escape )  ; "\;" "\/" "\\" escaped (Section 4)
  prose    = *VCHAR               ; verbatim to end-of-line

================================================================================
4. LITERALS AND ESCAPING (NORMATIVE, MANDATORY)
================================================================================

4.1. Reserved control tokens (everything else is data):
  ;        field separator
  :        key/value separator — the FIRST ':' in a field only
  /        scope close, when it is a token's leading character
  \        escape character
  " ::: "  prose-tail separator (with surrounding spaces)

4.2. Escape sequences — exactly three; implementations MUST NOT define others:
  \;   -> literal ';'   (value contains a semicolon)
  \/   -> literal '/'   (value would otherwise read as a scope close)
  \\   -> literal '\'   (value contains a backslash)

4.3. What does NOT require escaping:
  - Spaces are free everywhere (significant inside values and prose).
  - A ':' after the first in a field is data (timestamps, URLs).
  - A '/' that is not leading is data (paths, dates).
  - The prose tail is fully verbatim — no character inside it is ever escaped.
  (Contrast: JSON forces \" on every quote and \\n on every newline — the exact
  boundary where models trip on long prose. UFN removes that failure mode.)

4.4. Scalar lists — a short list is carried inside ONE value, comma-separated:
  lb:bug,infra   (no scope opened for the common small-list case)

4.5. Display names — a reference value MAY carry a human name after a literal '|':
  @:player|Grond   If omitted, the consumer resolves from its catalogue; for
  improvised/unknown entities the producer MUST include the name.

================================================================================
5. FAIL-SOFT PROCESSING
================================================================================

A conforming parser MUST:
  - Unknown key or class            -> skip the token, continue.
  - Scope still open at end-of-line -> close it implicitly (partial object usable).
  - '/acr' with no matching open    -> ignore.
  - No tokens recognized at all     -> treat the whole line as prose.
  - Empty value after a key         -> skip the field.

Principle: a partial frame is always better than a parse error. A syntactic failure
MUST NOT trigger a consumer's hard fallback; only genuine semantic failure should.

================================================================================
6. DICTIONARIES
================================================================================

UFN is dictionary-driven: senders and receivers share a token dictionary verbatim —
no inference, no fuzzy matching. Object/array classes and field keys are per-domain
and declared by the producer.
  - A FIXED receiver shares the dictionary ahead of time.
  - A GENERIC receiver self-orients: the producer SHOULD emit the dictionary it used
    (keys, enum codes, ligatures) so any consumer can expand the output.
Machine-readable starter dictionary: https://ufn.kovanex.dev/ufn.dict.json

================================================================================
7. ENCODING ECONOMY
================================================================================

UFN saves structure, not content. Levers ranked by MEASURED impact on a 50-task list
(vs compact JSON at 8201 chars):

  1. Short ids, not UUIDs            -37%     8-hex prefix (ce3760ed), not 36-char UUID
  2. Enum codes + 1-char keys        -11..16% s:in_progress->s:i, ty:feature->y:f
  3. Anonymous close '/'             -16%*    on nested output; '/acr' only for resync
  4. Omit defaults                   --       drop a field equal to its default
  (* on the nested portion). Combined ~ -59% vs compact JSON, ~ -44% vs naive UFN.

Measured results:
  Header tokens (4-field frame):      JSON ~110   UFN ~38    (65% fewer)
  Short response (60-token prose):    JSON 170    UFN 98     (42% smaller)
  Long response (600-token prose):    JSON 710    UFN 638    (10% smaller)
  Parse failure on truncation:        JSON total loss  UFN partial frame
  Time to first visible content:      JSON 30-60s  UFN ~1.5s (20-40x faster)

The shorter the response, the bigger the win — structural overhead amortizes across
less content.

7.4. Ligatures (tier 2) — one dictionary token that expands to a COMBINATION (the way
ROFL or :-) carried whole phrases/states). Value ligature (a frequent value-combo),
skeleton ligature (a frequent field pattern), prose ligature (recurring domain
phrases). Encode only Pareto-frequent combos; declare those you used. Because both
ends are LLMs, the shorthand vocabulary can be far richer than IRC ever was.

Honest baseline: the fair comparison is COMPACT JSON. Against bloated JSON (pretty +
all empty fields) UFN looks like -89%, but the real structural advantage is ~-59%.

================================================================================
8. RECOMMENDATIONS
================================================================================

Use UFN when: output is LLM-generated (esp. local models); you mix metadata with long
prose; you stream and want low time-to-first-content; truncation is a real risk.
Use JSON when: data is machine-generated and machine-consumed; purely structured /
deeply nested; one-shot document with a fixed schema; storing the system of record.

Operational guidance:
  - Keep storage in JSON; emit UFN only at the LLM edge.
  - Share the dictionary verbatim, or emit it so a generic receiver self-orients.
  - Prefer short ids, enum codes, omit defaults, use the anonymous close.
  - Encode only Pareto-frequent ligatures; keep the set learnable.
  - Carry long text in the prose tail, never as an escaped field.
  - Make the parser fail-soft; a partial frame beats an error.

================================================================================
9. EXAMPLES
================================================================================

9.1. Flat record (v1-style):
  t:241;s:done;ty:feature;ti:kovasite skill

9.2. Nested — a task with a repos array:
  t;id:9c92;n:241;s:done;ti:kovasite skill;rp;r;id:ce37;nm:pr;/r;r;id:94be;nm:kovanex-server;/r;/rp;/t
  (t..../t object task; rp..../rp array of repos; each r..../r an item object.)

9.3. Header plus prose tail (game frame):
  f:a ; @:player|Grond ; ^:shadow_figure|dark figure ; ~:tavern|Rusty Mug ; s:steel clash, muffled gasp ; l:torchlit ; c:clear ; >:atk shadow_figure ::: The air freezes for one long second...

9.4. The same frame in JSON (for comparison) — ~110 structural tokens vs UFN's ~38,
and a single missing brace renders all of it unparseable:
  {"frame":"action","roles":[{"type":"agent","entity_id":"player","name":"Grond"},{"type":"patient","entity_id":"shadow_figure","name":"dark figure"},{"type":"location","entity_id":"tavern","name":"Rusty Mug"}],"atmosphere":{"sounds":["steel clash","muffled gasp"],"light":"torchlit","weather":"clear"},"state_changes":[{"type":"attack","target":"shadow_figure"}],"prose":"The air freezes..."}

================================================================================
10. VERSIONING
================================================================================

                  UFN v1 (game profile)              UFN v2 (general)
  Origin          Tormund GM output (replaces JSON)  generalized for the agent edge
  Shape           flat: records + prose, no nesting  nesting-capable (scope open/close)
  Field sep       ' ; ' (spaces)                     ';' (tight)
  Dictionary      fixed game dictionary              per-domain, producer-declared
  Status          locked (v1.x adds only)            stable (locked)

A flat v1 line is a valid v2 line with no open scopes. v2 adds nesting, the tight
separator, enum/ligature economy, and the domain-agnostic dictionary model.

================================================================================
11. IMPLEMENTATIONS
================================================================================

  kovanex-server/internal/ufn     v2 encoder + parser, nesting, ligatures (reference)
  kovanex-server/internal/ufnpb   protobuf -> UFN, shared by ctl + MCP
  tormund/internal/game           v1 game-frame (ufn.go, ufn_bridge.go, bundle_ufn.go)
  prompt-a-porte/internal/ufn     a second independent implementation

Shipped surfaces: `kovanex-ctl --ufn` (list economy + nested get); MCP tool output in
UFN v2 (with compact-JSON fallback).

================================================================================
12. CONFORMANCE (FOR AGENTS)
================================================================================

A conforming PRODUCER:
  - emits one line per frame; uses ';' (general) or ' ; ' (v1) as the field separator;
  - uses only \; \/ \\ as escapes; puts long text in the prose tail after ' ::: ';
  - draws keys/classes/enums from a shared dictionary and SHOULD declare the
    dictionary (and any ligatures) it used;
  - SHOULD apply the economy levers (short ids, enum codes, omit defaults, anon close).

A conforming CONSUMER (parser):
  - classifies tokens by shape (field / open / close); splits a field on the FIRST ':';
  - maintains a scope stack; closes unclosed scopes implicitly at end-of-line;
  - is fail-soft per Section 5 (skip unknowns, never hard-fail on syntax);
  - treats everything after the first top-level ' ::: ' as verbatim prose.

================================================================================
13. REFERENCES
================================================================================

Normative:
  [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels",
            BCP 14, RFC 2119, March 1997.
  [UFN-v2]  Podtsebnev, I., "UFN v2 — USE Frame Notation, generalized",
            kovanex-server docs/UFN_v2.md, 2026.
Informative:
  [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange
            Format", RFC 8259, December 2017.
  [UFN-v1]  Podtsebnev, I., "USE Frame Notation (UFN) v1 — spec",
            tormund docs/UFN_SPEC.md, 2026.

Author: I. Podtsebnev — Kovanex — devops@kovanex.com — https://kovanex.dev