flowrs-engine/spec/flowrs-document-1.md

# The flowRS document format

**Specification version 1.0.0, for document format 1.** The JSON Schema is published at
<https://flowrs.openie.dev/schemas/flowrs-document-1.json> (`spec/flowrs-document-1.schema.json`
in flowRS's repository). This page is published at
<https://flowrs.openie.dev/docs/document-format/>.

This text is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (`spec/LICENSE`
in flowRS's repository). Its schema and conformance corpus are under the licence of flowRS's code,
Apache-2.0.

A flowRS graph is saved in one of two forms. A **`.flowrs` document** is JSON: the whole graph
with its layout, the app laid over it and who wrote each part. This is what the editor saves
and opens. The **graph language** (`.fdsl`) is text: one statement per node, the wires written
as the expressions that feed each input. It is what an agent reads and edits, and what
flowRS's own example corpus is written in. Both describe the same graph, and flowRS reads both
anywhere it reads a graph: the editor, `flowrsc`, and the MCP server.

This specification exists so that a graph never depends on flowRS's editor. With it, a person
or a program can read, write, check and run a graph without the IDE. `flowrsc check` says
whether a document is a valid graph, and `flowrsc run` computes it, both without the editor.
The specification, its schema and its conformance corpus are kept in step with the code that
reads them. A test fails when the schema is not what the code generates, when the schema and
the reader disagree about a document, or when a table here names a member, a type or a keyword
the code does not have.

The key words MUST, MUST NOT, SHOULD and MAY are used as RFC 2119 and RFC 8174 describe, when
they appear in capitals.

## The graph

A graph is a set of **nodes**, a set of **wires** and a set of **groups**.

- A **node** is one instance of a node type (`math.add`, `dsp.filter.iir`, `FloatLiteral`),
  with its parameter values. The node types are flowRS's node library: what each one computes,
  its ports and its parameters are listed in the [node reference](/docs/widgets/).
- A **wire** connects one node's output port to another node's input port. It says how the value
  crosses (its *mode*: moved, borrowed, cloned, copied) and whether it is *held*: a held wire
  delivers what its source produced on the previous step, which is how a loop gets an order of
  evaluation.
- A **group** is a collapsed subgraph: nodes name the group they belong to. Groups organise the
  canvas and never change what the graph computes.

Every node, wire and group has an id, a non-negative integer, unique among items of its kind.
Positions on the canvas, groups and the record of who wrote each part are layout and history:
two graphs that differ only in them compute the same thing.

## The `.flowrs` document

### Encoding

A document is one JSON value (RFC 8259) in UTF-8: an object. A reader MUST accept any valid JSON
layout. A writer SHOULD write the **canonical form**, which is what flowRS writes:

- two-space indentation, one member per line, members in the order of the tables below;
- `nodes`, `wires` and `groups` sorted by id, ascending; the keys of every object that maps names
  to values (`properties`, `widgets`, an origin's `params` and `inputs`) sorted by name;
- a member that is optional and holds its default (an empty list or map, `false`, an empty
  string, `0`, an unknown author) left out, except these, which are always written: the
  document's `generator`, `groups` and `widgets`; a node's `position` and `group`; a wire's
  `mode`; a group's `parent` and `position`; an app's `source`.

The canonical form makes the same graph the same bytes, so a document can be diffed, reviewed
and hashed.

Integers (ids, columns, rows, steps) are written as JSON integers, with no fraction and no
exponent: a reader refuses `3.0` where an integer belongs, although JSON Schema's `integer`
accepts it. Ids are unsigned 64-bit integers. A writer SHOULD keep them at or below
9007199254740991 (2^53 − 1), the largest integer a JavaScript reader holds exactly. Positions
are finite numbers.

### The document

<!-- schema: # -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `format` | 1 | yes | The document format. This specification is format 1. |
| `generator` | string | no | What wrote the file (`"flowrs 0.1.0"`). Informational: no reader depends on it. |
| `nodes` | array of node | yes | The nodes. |
| `wires` | array of wire | yes | The wires. |
| `groups` | array of group | no | The groups. Absent: none. |
| `widgets` | map of string | no | The interface fingerprint of each node type the graph uses, by type id (see *Fingerprints*). Absent: none recorded. |
| `app` | app | no | The app laid over the graph. Absent: the graph has no app of its own. |

### A node

<!-- schema: #/$defs/node -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `id` | id | yes | The node's id. |
| `type_id` | string | yes | The node type, by its id in flowRS's node library. |
| `properties` | map of string | yes | Parameter and literal values by name. Every value is a string as written: `"2"` and `"2.0"` are different values. |
| `position` | position | no | Where it sits on the canvas, `[x, y]`. Absent: `[0, 0]`. |
| `group` | id or null | no | The group it belongs to. `null` or absent: the top level. |
| `origin` | node_origin | no | Who wrote the node, its parameters and its inputs. Absent: not recorded. |

### A wire

<!-- schema: #/$defs/wire -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `id` | id | yes | The wire's id. |
| `from_node` | id | yes | The node whose output it reads. |
| `from_port` | string | yes | That node's output port. |
| `to_node` | id | yes | The node whose input it feeds. |
| `to_port` | string | yes | That node's input port. |
| `mode` | enum | no | How the value crosses, one of the modes below. Absent: `"Move"`. |
| `feedback` | boolean | no | `true`: the wire is held. Absent: `false`, the wire is instant. |
| `origin` | author | no | Who drew it. Absent: not recorded. |

The modes:

<!-- modes -->
| Mode | Meaning |
|---|---|
| `Move` | the value moves; its source may be consumed once |
| `Borrow` | a shared borrow, `&T`; any number may borrow one source |
| `BorrowMut` | an exclusive borrow, `&mut T`; no other borrow of that source |
| `Clone` | an explicit `.clone()` of the value |
| `Copy` | a `Copy` value, duplicated freely |

When more than one wire feeds the same input port, the wire with the lowest id is the one a
single input reads, and a variadic input reads them all in id order.

A held wire delivers, on each step, what its source produced on the previous step, and on the
first step the zero of its source port's type (`0`, `0.0`, `false`). A loop of wires
none of which is held cannot be evaluated, and is not a valid graph.

### A group

<!-- schema: #/$defs/group -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `id` | id | yes | The group's id. |
| `name` | string | yes | Its name, shown on the collapsed node. |
| `parent` | id or null | no | The group it sits in. `null` or absent: the top level. |
| `position` | position | no | Where its collapsed node sits. Absent: `[0, 0]`. |

### Who wrote it

An author is an object whose `by` member says which kind it is:

<!-- authors -->
| `by` | Other members | Meaning |
|---|---|---|
| `unknown` | none | not recorded |
| `human` | none | a person, by hand |
| `agent` | `model`, `conversation` | an agent: its model (`provider/model`) and the conversation or proposal it came from |
| `imported` | `dialect` | read from another tool's file, by the importer named |
| `mixed` | none | more than one of the above |

A node's origin:

<!-- schema: #/$defs/node_origin -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `node` | author | no | Who created the node, or last changed its type. |
| `params` | map of author | no | Who last wrote each parameter, by name. |
| `inputs` | map of author | no | Who last decided what feeds each input, by port. |

The origin is history, not meaning: it never changes what the graph computes. A writer that
records nothing leaves it out.

### The app

The app is how the graph's ports are presented when it runs: a program in OpenUI Lang bound
to the graph's ports by reference (`n7.mean` reads node 7's `mean` output), and how it is drawn.

<!-- schema: #/$defs/app -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `version` | string | yes | The OpenUI Lang version of `source`: `"1.0"`. |
| `library` | string | no | The component library it was written against: `"flowrs-app@1"`. |
| `source` | string | no | The program. Empty: the graph's inferred app, which follows the graph as it changes. |
| `skin` | enum | no | `"instrument"` (dark) or `"paper"` (light). Absent: instrument. |
| `panel` | panel | no | The Front Panel's tabs and where each card sits. |
| `title` | string | no | The app's title. Absent: one is found from the graph or the file's name. |
| `description` | string | no | One line under the title. |
| `steps` | u32 | no | How many passes the step loop runs when the app's program runs outside the editor. Absent: the default. |
| `turns` | array of app_turn | no | The acceptance checks each accepted turn of the app's making left, in turn order. Each is run again after every later edit, and an edit that breaks one is named. |
| `prefer` | array of string | no | The layouts a person picked, as `<rule>=<component>` (`R06=Scope`), in the order picked. One that names no rule, or a component its rule does not offer, is ignored. |

<!-- schema: #/$defs/app_turn -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `request` | string | no | The request, in the words it came in. Absent: a person's own edit. |
| `checks` | array of string | no | Its checks, in flowrs-app's check language: `shows n4.magnitude as spectrum`, `unit n2.voltage`, `controls $n3`. |

<!-- schema: #/$defs/panel -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `tabs` | array of string | no | The tabs, by name, in order. Absent: one page with every card. |
| `cards` | array of card | no | Where each card sits. |

<!-- schema: #/$defs/card -->
| Member | Type | Required | Meaning |
|---|---|---|---|
| `card` | string | yes | Which card: `ctl:3` for the control with id 3, `ind:7` for the Indicator with id 7, `out:5.values` for the `values` output of the node with id 5. |
| `col` | u32 | yes | The column of its top-left corner, from 0. |
| `row` | u32 | yes | The row of its top-left corner, from 0. |
| `tab` | u32 | no | The tab it is on, as an index into `tabs`. Absent: every tab. |

### Fingerprints

`widgets` maps each node type the graph uses to the fingerprint of that type's interface when
the graph was saved: 16 lowercase hexadecimal digits. The fingerprint is FNV-1a with 64 bits
(offset basis `0xcbf29ce484222325`, prime `0x100000001b3`) over a sequence of strings, each
followed by the byte `0xff`: the type's id; `in`, then each input port's name, type and
`variadic` or `single`; `out`, then each output port's the same way; then each parameter's
name, type and default. A reader that has flowRS's node library compares them with its own and
reports each node type that changed or no longer exists. It MUST still open the graph: drift
is reported, never refused. A reader without the library ignores `widgets`.

### Reading a document

A reader:

- MUST read `format` before anything else. A document with no `format`, or with `format` 0, is
  malformed. A document whose `format` is greater than the reader's MUST be refused as written
  by a newer version, not read as though it were this one.
- MUST refuse a document that is not valid JSON, whose members have the wrong types, that
  lacks a required member, or in which a node, wire or group id appears more than once (a
  repeated id would keep one item and lose the other).
- MUST accept the arrays in any order, and MUST ignore members it does not know, so that a
  later version of this specification can add a member that older readers skip.
- MAY take an object written as a JSON array of its members' values in the order of the tables
  (the sequence form serde reads a struct from), as flowRS's reader does: `"origin": []` is an
  empty origin there. No writer writes it, a document that holds it does not conform, and
  another reader need not read it.
- MUST accept a wire whose node does not exist and a node whose type is unknown. These are
  defects of the graph, not of the document: `flowrsc check` reports them, with the editor's
  own diagnostics.

A format-1 reader also accepts two older spellings and reads them as what they always meant:

<!-- legacy -->
| Written | Read as |
|---|---|
| node type `ParSquare` | `MapSquare` with `schedule` = `parallel` |
| node type `ParSqrt` | `MapSqrt` with `schedule` = `parallel` |
| node type `ParSum` | `Sum` with `schedule` = `parallel` |
| node type `GpuSquare` | `MapSquare` with `schedule` = `gpu` |
| node type `GpuSqrt` | `MapSqrt` with `schedule` = `gpu` |
| node type `GpuScale` | `MapScale` with `schedule` = `gpu` |

and a `tab` property on a node (`"tab": "1"`, from before the app had a panel layout) is taken off
the node. On a control or an Indicator whose `tab` is a whole number, the node's card is placed
on that tab, an index from 0 (at most 64), in the app's `panel`; tabs named `Tab 1`, `Tab 2`, …
are added up to it, and an app is created when the document has none.

### An example

A graph that adds two numbers and prints the sum, in canonical form:

```json flowrs
{
  "format": 1,
  "generator": "flowrs 0.1.0",
  "nodes": [
    {
      "id": 0,
      "type_id": "FloatLiteral",
      "properties": {
        "value": "2"
      },
      "position": [
        0.0,
        0.0
      ],
      "group": null
    },
    {
      "id": 1,
      "type_id": "FloatLiteral",
      "properties": {
        "value": "3"
      },
      "position": [
        0.0,
        120.0
      ],
      "group": null
    },
    {
      "id": 2,
      "type_id": "math.add",
      "properties": {},
      "position": [
        260.0,
        60.0
      ],
      "group": null
    },
    {
      "id": 3,
      "type_id": "io.print",
      "properties": {},
      "position": [
        520.0,
        60.0
      ],
      "group": null
    }
  ],
  "wires": [
    {
      "id": 0,
      "from_node": 0,
      "from_port": "value",
      "to_node": 2,
      "to_port": "a",
      "mode": "Move"
    },
    {
      "id": 1,
      "from_node": 1,
      "from_port": "value",
      "to_node": 2,
      "to_port": "b",
      "mode": "Move"
    },
    {
      "id": 2,
      "from_node": 2,
      "from_port": "sum",
      "to_node": 3,
      "to_port": "value",
      "mode": "Move"
    }
  ],
  "groups": [],
  "widgets": {}
}
```

## The graph language

The graph language writes the same graph as statements. The example above is:

```fdsl
let n0 = FloatLiteral { value: 2 };
let n1 = FloatLiteral { value: 3 };
let n2 = math.add(a: n0.value, b: n1.value);
let n3 = io.print(value: n2.sum);
```

A node is a `let` statement: its handle, its type, then the wires into its inputs as
`port: source`, then its parameters as `{ key: value }`. A source is a node's handle and an
output port, `n0.value`, and its mode is how the expression reads.

### Lexical structure

A text is UTF-8. Positions in it (in error messages) are a line and a column, both from 1,
counted in characters.

- **Space**: spaces, tabs and carriage returns separate tokens. A **line break** is a token: it
  ends a statement, except inside `( )`, `[ ]` or `{ }`, where it is only space.
- **Comments** run from `//` to the end of the line, and mean nothing.
- An **identifier** starts with a letter or `_` and goes on with letters, digits and `_`
  (letters and digits in the Unicode sense).
- A **number** starts with an ASCII digit, or `-` before one, and goes on with ASCII letters,
  digits, `_`, a `.` before a digit, and `+` or `-` after `e` or `E`. It is kept exactly as
  written: `48_000`, `1e-9` and `-3.5` are parameter values as they stand.
- A **string** is `"…"`, and may run over lines. Its escapes are:

<!-- escapes -->
| Escape | Means |
|---|---|
| `\n` | line feed |
| `\t` | tab |
| `\r` | carriage return |
| `\0` | the NUL character |
| `\\` | a backslash |
| `\"` | a double quote |
| `\'` | a single quote |
| `\u{…}` | the Unicode character with that hexadecimal code |

  Any other escape is an error. A **raw string**, `r"…"`, `r#"…"#`, `r##"…"##` and so on, is
  taken as written, with no escapes, up to the quote followed by as many `#` as opened it.
- **Punctuation**: `=` `+=` `(` `)` `{` `}` `[` `]` `,` `:` `;` `.` `&`. Any other character is an
  error.

These words are keywords, and a handle cannot be one of them:

<!-- keywords -->
`let` `replace` `remove` `set` `unset` `group` `in`

`_` stands for "nothing feeds this input", and `mut` follows `&` in an exclusive borrow, so
neither should name a node.

### Grammar

In EBNF (ISO 14977 style: `{ }` repeats, `[ ]` is optional, `|` separates choices). Line breaks
inside brackets are left out, as the lexical rules say.

```ebnf
graph      = { statement-end } , [ statement , { statement-end , { statement-end } , statement } ] , { statement-end } ;
statement-end = ";" | line-break ;
statement  = let | group | wire ;

let        = "let" , handle , "=" , decl ;
decl       = type , [ "(" , [ arg , { "," , arg } , [ "," ] ] , ")" ]
                  , [ "{" , [ prop , { "," , prop } , [ "," ] ] , "}" ]
                  , [ "in" , handle ] ;
type       = dotted | string ;
arg        = name , ":" , sources ;
prop       = ( dotted | string ) , ":" , value ;
value      = string | number | dotted ;

group      = "group" , handle , "=" , string , [ "in" , handle ] ;

wire       = handle , "." , name , ( "=" | "+=" ) , sources ;

sources    = "_" | source | "[" , [ source , { "," , source } , [ "," ] ] , "]" ;
source     = [ "&" , [ "mut" ] ] , handle , "." , name , { "." , modifier , "(" , ")" } ;
modifier   = "clone" | "copy" | "prev" ;

handle     = identifier ;   (* not a keyword *)
name       = identifier | string ;
dotted     = identifier , { "." , identifier } ;
```

A parameter block may begin on the line after a node's inputs: a line break before its `{` is
space.

### What a text means

- **Handles.** A handle of the form `n<k>`, where `<k>` is digits with no leading zero (`n0`,
  `n12`, not `n01`), is node `k`. Any other handle (`lowpass`) is given the next id after the
  largest `n<k>` in the text, in the order the statements are written. Groups are the same with
  `g<k>`. A node may name a group declared after it.
- **Wires.** Each source in a node's inputs is a wire into that input, and so is each source of
  a wire statement. Wires take ids from 0 in the order they are written: every `let`'s inputs in
  statement order, then the wire statements. In a graph text `=` and `+=` both add their wires
  (each is one more wire into the input), and `_` adds none.
- **Modes.** `n0.value` moves the value; `&n0.value` borrows it; `&mut n0.value` borrows it
  exclusively; `n0.value.clone()` and `n0.value.copy()` clone and copy it. A wire has one mode,
  so `.clone()` or `.copy()` after a borrow, or both together, is an error. `.prev()` holds the
  wire, in any mode.
- **Parameters.** A value is text, exactly as written: a string's contents, a number's
  spelling, or a word (`window: hann`, `zero_phase: on`). A key given twice is an error.
- **Groups.** `group g0 = "Filters";` declares a group, and `in g0` after a node or a group puts
  it there.
- **Positions** are not written. A reader lays the nodes out from left to right by dependency.
- **The app.** An `app { … }` block holds the app's OpenUI Lang source. It begins only where a
  statement may begin, with `app` followed by `{`, and ends at the brace that closes it, read
  with OpenUI Lang's own rules for strings and comments. It is cut out before the graph is read,
  so every position after it is where it was written. Several blocks are one app, in order.

A text is refused when it does not parse, when a handle or a group is declared twice, when a
source or a wire statement names a handle nothing declares, or when it holds an edit
statement. `replace`, `remove`, `set` and `unset` belong to the patch language an agent edits a
graph with, which this specification does not cover. As in a document, an unknown node type or a
port its type does not have is a defect of the graph, which `flowrsc check` reports, not of the
text.

Some texts that are refused, with where:

```fdsl refused 1:13
let a = T(x n0.y)
```

```fdsl refused 1:21
let a = math.add(a: 2)
```

```fdsl refused 2:31
let n0 = FloatLiteral { value: 2 };
let n1 = io.print(v: &n0.value.clone());
```

```fdsl refused 1:6
n1.a -> n2.b
```

### Comments, types and a held wire

flowRS prints a graph in dependency order with each output's type, and each input left open,
in a comment. The comments mean nothing, so a reader may write them or leave them out:

```fdsl
// flowRS graph: a counter.
let n0 = FloatLiteral { value: 1 };  // → value: f64
let n1 = math.add(a: n1.sum.copy().prev(), b: n0.value);  // → sum: f64
let n2 = io.print(value: n1.sum);
```

Here `n1.sum.copy().prev()` feeds `n1` its own sum from the previous step, copied, so the graph
counts up by one each step.

## Conformance

A document **conforms** when it validates against the schema and meets every rule above that
the schema cannot state: integers without fractions, ids once each. flowRS writes conforming
documents in the canonical form. A text conforms when it parses as the grammar says.

Conformance says a document is well formed, not that its graph is valid. A **valid graph** also
uses only node types the library has, wires only ports they have, gives each parameter a value
its type accepts, and has no loop without a held wire. `flowrsc check` says whether a document or
a text holds a valid graph, with the diagnostics the editor shows.

The conformance corpus is in flowRS's repository under `spec/conformance/`, with a manifest that
says, for each file, whether a reader accepts it, whether it validates against the schema,
whether it is in canonical form, and why. A reader that agrees with the manifest on every file
reads documents as flowRS does.

## Versions

The document's `format` changes only when an older reader could misread a newer file. A member
added with a default that older readers can skip does not change it, and moves this
specification's minor version. A change of wording alone moves its patch version. A format-1
reader keeps reading every format-1 document ever written, including the older spellings above.