open-standards/sense/SPEC.md

# Sense v0.1 — A Standard for Ambient Context

| | |
|---|---|
| **Standard** | sense |
| **Status** | v0.1.0 draft for review |
| **Licence** | Specification text CC-BY-4.0 ([`LICENSE-SPEC`](LICENSE-SPEC)); reference code Apache-2.0 ([`LICENSE`](LICENSE)) |
| **Reference** | [`sense-rs/`](sense-rs/) |

A vendor-neutral standard for the **ambient context substrate** — the always-on,
deterministic, energy-accounted record of *what is happening around an agent*, and
a principled account of *when that is worth a human's or another agent's
attention*. Where the tool protocols (MCP, A2A, ACP) standardize an agent
**reaching for** capabilities, Sense standardizes the **afferent** half: the
stream of observations flowing in, and the governed decision to surface them.

---

## 1. Scope and doctrine

1. **Afferent, not efferent.** Sense records the *world the agent is in* — file
   changes, focus, logs, thermals, shell activity — not the agent's own reasoning
   trace. The reasoning trace is observability's domain (OpenTelemetry GenAI), is
   non-deterministic by its own admission, and is out of scope here.
2. **Deterministic by construction.** Every observation is content-addressed and
   totally ordered under a hybrid logical clock. Replaying a log yields a
   byte-identical projection on any machine (§3). Determinism is what makes an
   ambient record *auditable* rather than merely *logged*.
3. **Perception ⊗ attention-budget.** Observing is cheap; interrupting a human is
   not. Sense pairs a salience projection (§4) with an **attention budget** (§5)
   that decides whether an observation surfaces, scoring its value against a
   modelled interruption cost and pricing the surfacing in joules. This is the
   afferent dual of JCP's *capability ⊗ joule-budget*.
4. **Inference is the last resort.** A question about the observed world is
   answered from the log by a deterministic recall cascade; a model is called only
   when nothing cheaper can answer (§6). Cheapest sufficient answer wins.
5. **Provenance and trust are declared, never promoted.** Every datum carries its
   source class and a **trust level** (§2); every energy figure carries its
   provenance (§7). A host may downgrade what it is unsure of; it must never
   silently upgrade.
6. **Non-goals.** Sense does not mandate a sensor set, a UI, a model, a transport,
   or a storage engine. It standardizes the *observed datum*, the *determinism*,
   the *attention decision*, the *cascade accounting*, and the *receipt* — not the
   daemon.

## 2. The observed event (REQUIRED)

An **Event** is one datum on the log:

- `hlc` — a [hybrid logical clock](#3-canonical-serialization-and-determinism)
  stamp; the event's place in the total order.
- `source` — a **SourceClass**: the vendor-neutral class of the signal
  (`file_change`, `app_focus`, `system_log`, `process_tree`, `thermal_power`,
  `shell`, `accessibility`, `network`, `endpoint`, `salience`). A host maps its
  native sensors onto these classes — macOS FSEvents, Linux inotify and Windows
  `ReadDirectoryChangesW` all surface as `file_change` — so a log means the same
  thing on every platform. Codes are stable and append-only; vendor-specific
  classes use `vendor:<code>` with `code ≥ 1024`.
- `valid_time_ns` — optional wall-clock time at observation, captured **as data**,
  not as the clock (so replay stays deterministic when the wall clock jumps).
- `trust` — a **TrustLevel**: `Attested` (cryptographically vouched, e.g. a
  Sandbox attested capture), `FirstParty` (a first-party OS API the host called),
  `Inferred` (parsed from an untrusted byte stream), or `Untrusted` (the default).
- `payload` — the content address (§3) of the raw bytes; the bytes live in the
  store's content-addressed table, recoverable from the address alone.
- `line` — the textual projection the attention and synthesis tiers read.

An event's **content id** is `blake3` over a fixed-order, length-prefixed
concatenation of its fields. The id binds the source class and trust level, so an
event cannot be re-classified or re-trusted without changing its id (conformance
C2).

## 3. Canonical serialization and determinism (REQUIRED)

- **Content address.** A `Cid` is `blake3(payload_bytes)`, lowercase-hex on the
  wire. Identical bytes always yield the identical address; storage *is* the index.
- **Total order.** Events are keyed by an `Hlc = (physical_ns, logical)`. The
  physical component is a monotonic anchor; `logical` breaks ties at the same
  instant. The clock is totally ordered within a daemon run; on restart a host
  seeds it past the maximum stored stamp so new stamps strictly exceed all stored
  ones.
- **Replay determinism.** `events_since(Hlc::ZERO)` replays the whole log in clock
  order. Two hosts fed the same event sequence produce a byte-identical
  **replay digest** — `blake3` over each event's content id in order (conformance
  C1). A production host MAY persist to an MVCC B-tree keyed by the HLC
  big-endian bytes; the wire and the order are fixed here, not the engine.
- **Receipt core (§8)** uses canonical det-CBOR (RFC 8949 §4.2.1: length-first
  then bytewise key ordering, shortest integer form, no floats/bools/nulls), the
  same encoding the family's JCR-1 receipt signs over.

## 4. The salience projection (REQUIRED)

A **bounded-state observer** consumes the event stream and projects three scalars
in `[0, 1]`:

- **salience** — is the current source *bursting*? (the fraction of a recency
  window occupied by the current source class).
- **novelty** — is the current source *rare*? (one minus its share of all traffic
  seen).
- **uncertainty** — is the recent window *low-trust*? (the fraction of the window
  at `Inferred`/`Untrusted`). An honesty signal: "the system isn't sure what's
  going on."

The reference observer computes these from **exact integer counts**, so two runs
over the same events project byte-identically. A host MAY substitute a model
observer (e.g. a recurrent bounded-state network); the standard requires only
that a *conforming* observer be deterministic under replay, and that the three
scalars carry these meanings.

## 5. The attention budget (REQUIRED)

A continuous projection is not yet a decision. A **detector** turns it into
discrete **Highlights** by an EMA-baselined *k*-sigma rule with an absolute floor
and a debounce — it fires on *anomalies relative to a probe's own recent
baseline*, above an absolute floor, not on every reading. The decision is made
against the *prior* baseline (then the baseline updates) so a single large spike
does not suppress its own fire.

A highlight is then **gated** by the attention budget before it reaches a human or
agent. Following the proactive-interruption literature, the gate scores:

- **value** — the highlight's probe score, with `uncertainty` highlights
  discounted (an honesty signal is usually less worth an interruption than a true
  salience or novelty spike).
- **interruption cost** — *displacement* (proportional to how engaged the user is)
  plus *interference* (a penalty when the surfacing would occupy the user's
  currently-active modality — a visual alert while the user reads).

A highlight **surfaces** only when `value > cost + margin` **and** the energy
budget can pay the surfacing's joule price. The three outcomes are `surface`,
`hold:value` (value did not clear cost), and `hold:budget` (no joules left). The
value test comes first: a highlight that is not worth an interruption is
`hold:value` whether or not the budget could pay for it, so `hold:budget` means a
worthy highlight the budget could not afford. The decision is deterministic in its
inputs (conformance C3). Surfacing debits the
budget; a hold spends nothing. This makes attention a **metered resource**: an
ambient agent cannot interrupt without account, and cannot interrupt past its
energy budget at all.

## 6. The synthesis cascade (REQUIRED)

When a human or agent asks a question of the log, the host builds an **Envelope**
— the top-*k* retrieved spans + the salience projection + the query — and walks a
**speculative cascade**:

1. **Deterministic recall first.** A Tier-0 stage recognises a small set of
   factual-recall intents (recent files, last command, what was surfaced, …) and
   answers directly from the envelope's spans, with **no model call**. It is
   conservative: it answers only when query-side and span-side evidence agree, and
   escalates when unsure (a missed answer costs a paid call; a wrong answer is
   worse).
2. **Escalate as a last resort.** Only when no cheap stage answers does the
   request reach the fallback — typically a frontier model.

Retrieval is BM25-lite over exact integer term frequencies, then
**source-stratified recency fill** for any empty slots (natural-language queries
rarely overlap span text by token, and an empty envelope answers nothing). Spans
are content-addressed, so an envelope built from the same log + query is
byte-identical and a provider-side prompt cache hits cleanly. Every answer carries
an honest [`Energy`](#7-energy-and-provenance) figure (a deterministic answer is an
honest zero estimate, never a measurement).

## 7. Energy and provenance (REQUIRED)

Every metered figure — a surfacing's price, a synthesis's cost — is an integer
microjoule pair: `measured_uj` (present only from a hardware interface) and
`estimated_uj` (always present), tagged with a **Provenance** (`HwShunt`,
`ModelBased`, `Estimator`). This is the family's shared energy model: a figure is
reported with the honesty of how it was obtained and **never promoted**. The
honesty invariant is that `HwShunt` requires a measurement and `Estimator`
forbids one. Measurement itself is a **recognized edge**: the sibling **Sandbox**
standard's `PowerMeter` (real RAPL/IOReport) is the reference meter; Sense defines
what an interaction records, the host wires the meter it trusts.

## 8. The sealed, hash-chained surfacing log (REQUIRED)

An ambient agent's audit duty is the inverse of a tool agent's: it must show it did
not observe-and-surface without account, and that no record of what it surfaced
(or chose to hold) was quietly removed. Every gate decision (§5) therefore seals
as the next link of a **hash chain**. The signed core commits to the source
event's content id, its trust level, the highlight, the value/cost the budget
computed, the decision, the joules charged, and the budget left; each link's
`prev` is the BLAKE3 content id of the previous link's canonical core, and `seq`
increments from zero.

**Allows and holds are both chained** — a held highlight is exactly the record an
adversarial host would prefer to forget. Excising, reordering, or editing any link
breaks the chain at that point (conformance C5). The core signs with the host's
Ed25519 key; `verify_chain` checks every signature, the sequence, and every `prev`
link, and reports the first break. A link MAY additionally be expressed as a JCR-1
COSE_Sign1 envelope for verification by the family's shared cross-language tooling
(a recognized edge, not required — the hash chain is the primary audit).

## 9. Conformance

A conforming implementation reproduces, from the published vectors alone
([`conformance/v1/`](conformance/v1/)):

- **C1 — capture determinism.** A fixed event sequence replays to the published
  `replay_digest`; two builds agree.
- **C2 — provenance & trust.** A fixed event's content id matches the published
  `event_id`; re-tagging its source class or trust level changes the id.
- **C3 — attention.** A fixed signal stream yields the published sequence of
  highlights and surface/hold decisions, and the same highlights gated for a busy
  user on a small budget yield every outcome — `surface`, `hold:value`,
  `hold:budget` — with the published budget after each.
- **C4 — synthesis cascade.** A recall query is answered with no model call
  (`deterministic`); an open-ended query escalates.
- **C5 — chain integrity.** A surfacing log verifies (`verify_chain → n`), and
  excising the middle link breaks it at the published index.

## 10. Recognized edges

Sense is one organ in the family; each boundary it leans on is a published sibling
standard, not a dependency it owns:

- **JCP** (capability ⊗ joule-budget) — gates *who* may subscribe to *which*
  source class, for *how many* joules. Sense is the afferent dual.
- **Sandbox** — runs the capture in metered, attested isolation; its `PowerMeter`
  is the reference meter (§7) and its attestation is what earns a datum the
  `Attested` trust level (§2).
- **JouleClaw** — the reference cascade and frontier-model substrate behind the
  synthesis fallback (§6).
- **Proof / JouleContract** — the JCR-1 receipt format the surfacing log seals to
  (§8).
- **joule-ui** — the surface a highlight is rendered on; a surfaced highlight is
  itself a metered, sealed joule-ui interaction.
- **WAI** — the container for multimodal sensor payloads referenced by a `Cid`.