open-standards/sense/SPEC.md

# Sense v0.1 — A Standard for Ambient Context

| | |
|---|---|
| **Standard** | sense |
| **Status** | v0.1.0 draft for review |
| **Licence** | Specification text CC-BY-4.0 ([`LICENSE-SPEC`](LICENSE-SPEC)); reference code Apache-2.0 ([`LICENSE`](LICENSE)) |
| **Reference** | [`sense-rs/`](sense-rs/) |

A vendor-neutral standard for the **ambient context substrate** — the always-on,
deterministic, energy-accounted record of *what is happening around an agent*, and
a principled account of *when that is worth a human's or another agent's
attention*. Where the tool protocols (MCP, A2A, ACP) standardize an agent
**reaching for** capabilities, Sense standardizes the **afferent** half: the
stream of observations flowing in, and the governed decision to surface them.

---

## 1. Scope and doctrine

1. **Afferent, not efferent.** Sense records the *world the agent is in* — file
   changes, focus, logs, thermals, shell activity — not the agent's own reasoning
   trace. The reasoning trace is observability's domain (OpenTelemetry GenAI), is
   non-deterministic by its own admission, and is out of scope here.
2. **Deterministic by construction.** Every observation is content-addressed and
   totally ordered under a hybrid logical clock. Replaying a log yields a
   byte-identical projection on any machine (§3). Determinism is what makes an
   ambient record *auditable* rather than merely *logged*.
3. **Perception ⊗ attention-budget.** Observing is cheap; interrupting a human is
   not. Sense pairs a salience projection (§4) with an **attention budget** (§5)
   that decides whether an observation surfaces, scoring its value against a
   modelled interruption cost and pricing the surfacing in joules. This is the
   afferent dual of JCP's *capability ⊗ joule-budget*.
4. **Inference is the last resort.** A question about the observed world is
   answered from the log by a deterministic recall cascade; a model is called only
   when nothing cheaper can answer (§6). Cheapest sufficient answer wins.
5. **Provenance and trust are declared, never promoted.** Every datum carries its
   source class and a **trust level** (§2); every energy figure carries its
   provenance (§7). A host may downgrade what it is unsure of; it must never
   silently upgrade.
6. **Non-goals.** Sense does not mandate a sensor set, a UI, a model, a transport,
   or a storage engine. It standardizes the *observed datum*, the *determinism*,
   the *attention decision*, the *cascade accounting*, and the *receipt* — not the
   daemon.

## 2. The observed event (REQUIRED)

An **Event** is one datum on the log:

- `hlc` — a [hybrid logical clock](#3-canonical-serialization-and-determinism)
  stamp; the event's place in the total order.
- `source` — a **SourceClass**: the vendor-neutral class of the signal
  (`file_change`, `app_focus`, `system_log`, `process_tree`, `thermal_power`,
  `shell`, `accessibility`, `network`, `endpoint`, `salience`). A host maps its
  native sensors onto these classes — macOS FSEvents, Linux inotify and Windows
  `ReadDirectoryChangesW` all surface as `file_change` — so a log means the same
  thing on every platform. Codes are stable and append-only; vendor-specific
  classes use `vendor:<code>` with `code ≥ 1024`.
- `valid_time_ns` — optional wall-clock time at observation, captured **as data**,
  not as the clock (so replay stays deterministic when the wall clock jumps).
- `trust` — a **TrustLevel**: `Attested` (cryptographically vouched, e.g. a
  Sandbox attested capture), `FirstParty` (a first-party OS API the host called),
  `Inferred` (parsed from an untrusted byte stream), or `Untrusted` (the default).
- `payload` — the content address (§3) of the raw bytes; the bytes live in the
  store's content-addressed table, recoverable from the address alone.
- `line` — the textual projection the attention and synthesis tiers read.

An event's **content id** is `blake3` over a fixed-order, length-prefixed
concatenation of its fields. The id binds the source class and trust level, so an
event cannot be re-classified or re-trusted without changing its id (conformance
C2).

## 3. Canonical serialization and determinism (REQUIRED)

- **Content address.** A `Cid` is `blake3(payload_bytes)`, lowercase-hex on the
  wire. Identical bytes always yield the identical address; storage *is* the index.
- **Total order.** Events are keyed by an `Hlc = (physical_ns, logical)`. The
  physical component is a monotonic anchor; `logical` breaks ties at the same
  instant. The clock is totally ordered within a daemon run; on restart a host
  seeds it past the maximum stored stamp so new stamps strictly exceed all stored
  ones.
- **Replay determinism.** `events_since(Hlc::ZERO)` replays the whole log in clock
  order. Two hosts fed the same event sequence produce a byte-identical
  **replay digest** — `blake3` over each event's content id in order (conformance
  C1). A production host MAY persist to an MVCC B-tree keyed by the HLC
  big-endian bytes; the wire and the order are fixed here, not the engine.
- **Receipt core (§8)** uses canonical det-CBOR (RFC 8949 §4.2.1: length-first
  then bytewise key ordering, shortest integer form, no floats/bools/nulls), the
  same encoding the family's JCR-1 receipt signs over.

## 4. The salience projection (REQUIRED)

A **bounded-state observer** consumes the event stream and projects three scalars
in `[0, 1]`:

- **salience** — is the current source *bursting*? (the fraction of a recency
  window occupied by the current source class).
- **novelty** — is the current source *rare*? (one minus its share of all traffic
  seen).
- **uncertainty** — is the recent window *low-trust*? (the fraction of the window
  at `Inferred`/`Untrusted`). An honesty signal: "the system isn't sure what's
  going on."

The reference observer computes these from **exact integer counts**, so two runs
over the same events project byte-identically. A host MAY substitute a model
observer (e.g. a recurrent bounded-state network); the standard requires only
that a *conforming* observer be deterministic under replay, and that the three
scalars carry these meanings.

## 5. The attention budget (REQUIRED)

A continuous projection is not yet a decision. A **detector** turns it into
discrete **Highlights** by an EMA-baselined *k*-sigma rule with an absolute floor
and a debounce — it fires on *anomalies relative to a probe's own recent
baseline*, above an absolute floor, not on every reading. The decision is made
against the *prior* baseline (then the baseline updates) so a single large spike
does not suppress its own fire.

A highlight is then **gated** by the attention budget before it reaches a human or
agent. Following the proactive-interruption literature, the gate scores:

- **value** — the highlight's probe score, with `uncertainty` highlights
  discounted (an honesty signal is usually less worth an interruption than a true
  salience or novelty spike).
- **interruption cost** — *displacement* (proportional to how engaged the user is)
  plus *interference* (a penalty when the surfacing would occupy the user's
  currently-active modality — a visual alert while the user reads).

A highlight **surfaces** only when `value > cost + margin` **and** the energy
budget can pay the surfacing's joule price. The three outcomes are `surface`,
`hold:value` (value did not clear cost), and `hold:budget` (no joules left). The
value test comes first: a highlight that is not worth an interruption is
`hold:value` whether or not the budget could pay for it, so `hold:budget` means a
worthy highlight the budget could not afford. The decision is deterministic in its
inputs (conformance C3). Surfacing debits the
budget; a hold spends nothing. This makes attention a **metered resource**: an
ambient agent cannot interrupt without account, and cannot interrupt past its
energy budget at all.

## 6. The synthesis cascade (REQUIRED)

When a human or agent asks a question of the log, the host builds an **Envelope**
— the top-*k* retrieved spans + the salience projection + the query — and walks a
**speculative cascade**:

1. **Deterministic recall first.** A Tier-0 stage recognises a small set of
   factual-recall intents (recent files, last command, what was surfaced, …) and
   answers directly from the envelope's spans, with **no model call**. It is
   conservative: it answers only when query-side and span-side evidence agree, and
   escalates when unsure (a missed answer costs a paid call; a wrong answer is
   worse).
2. **Escalate as a last resort.** Only when no cheap stage answers does the
   request reach the fallback — typically a frontier model.

Retrieval is BM25-lite over exact integer term frequencies, then
**source-stratified recency fill** for any empty slots (natural-language queries
rarely overlap span text by token, and an empty envelope answers nothing). Spans
are content-addressed, so an envelope built from the same log + query is
byte-identical and a provider-side prompt cache hits cleanly. Every answer carries
an honest [`Energy`](#7-energy-and-provenance) figure (a deterministic answer is an
honest zero estimate, never a measurement).

## 7. Energy and provenance (REQUIRED)

Every metered figure — a surfacing's price, a synthesis's cost — is an integer
microjoule pair: `measured_uj` (present only from a live hardware interface) and
`estimated_uj` (always present), tagged with a **Provenance** (`HwShunt`,
`ModelBased`, `Estimator`). This is the family's shared energy model: a figure is
reported with the honesty of how it was obtained and **never promoted**. The
honesty invariant is that `HwShunt` requires a measurement and `Estimator`
forbids one.

A reading from a vendor energy counter (RAPL, NVML, Apple IOReport, ...) **MAY**
be graded as a hardware reading (`HwShunt`, with `measured_uj`) only while the
counter is demonstrably live for the domain the figure claims: the implementation
**MUST** have observed the counter covering that domain advance under load (for
Apple IOReport: the CPU energy rail itself advancing in at least 5 sample
intervals, in at least 4 of every 5, over at least 300 ms while the probing
thread burns CPU; an interval runs from when one sample is in hand to when the
next is, and only one no longer than three of the probe's ticks, 75 ms at its
25 ms tick, counts: a longer one counts as nothing, neither among the
intervals, nor as an advance, nor towards the 300 ms, which is the summed
length of the intervals that count). A counter that is present but not live — frozen, or
republished in lumps — **MUST NOT** produce a hardware-graded figure; the
implementation falls back to its estimator or reports the work unmetered, and
**SHOULD** record why. A figure **MUST NOT** claim coverage of a domain its
counter does not read; where the format has a coverage field, such a figure is
Partial.

Measurement itself is a **recognized edge**: the sibling **Sandbox** standard's
`PowerMeter` is the reference meter — it reads RAPL, NVML or IOReport, grades a
reading `HwShunt` only while its counter is live, and falls back to its estimator
when it is not. Sense defines what an interaction records; the host wires the
meter it trusts.

## 8. The sealed, hash-chained surfacing log (REQUIRED)

An ambient agent's audit duty is the inverse of a tool agent's: it must show it did
not observe-and-surface without account, and that no record of what it surfaced
(or chose to hold) was quietly removed. Every gate decision (§5) therefore seals
as the next link of a **hash chain**. The signed core commits to the source
event's content id, its trust level, the highlight, the value/cost the budget
computed, the decision, the joules charged, and the budget left; each link's
`prev` is the BLAKE3 content id of the previous link's canonical core, and `seq`
increments from zero.

**Allows and holds are both chained** — a held highlight is exactly the record an
adversarial host would prefer to forget. Excising, reordering, or editing any link
breaks the chain at that point (conformance C5). The core signs with the host's
Ed25519 key; `verify_chain` checks every signature, the sequence, and every `prev`
link, and reports the first break. A link MAY additionally be expressed as a JCR-1
COSE_Sign1 envelope for verification by the family's shared cross-language tooling
(a recognized edge, not required — the hash chain is the primary audit).

## 9. Conformance

A conforming implementation reproduces, from the published vectors alone
([`conformance/v1/`](conformance/v1/)):

- **C1 — capture determinism.** A fixed event sequence replays to the published
  `replay_digest`; two builds agree.
- **C2 — provenance & trust.** A fixed event's content id matches the published
  `event_id`; re-tagging its source class or trust level changes the id.
- **C3 — attention.** A fixed signal stream yields the published sequence of
  highlights and surface/hold decisions, and the same highlights gated for a busy
  user on a small budget yield every outcome — `surface`, `hold:value`,
  `hold:budget` — with the published budget after each.
- **C4 — synthesis cascade.** A recall query is answered with no model call
  (`deterministic`); an open-ended query escalates.
- **C5 — chain integrity.** A surfacing log verifies (`verify_chain → n`), and
  excising the middle link breaks it at the published index.
- **C6 — injected observations (§11).** A sealed surfacing delivers into a running
  agent's record and the frame carries the link's own facts; a hold, a
  self-authored observation, a wake with no authority, and a delivery priced at
  zero are each refused; and neither the destination nor the presence of a wake
  can change without changing the frame's address.

## 10. Recognized edges

Sense is one organ in the family; each boundary it leans on is a published sibling
standard, not a dependency it owns:

- **JCP** (capability ⊗ joule-budget) — gates *who* may subscribe to *which*
  source class, for *how many* joules. Sense is the afferent dual.
- **Sandbox** — runs the capture in metered, attested isolation; its `PowerMeter`
  is the reference meter (§7: a vendor counter only while live, else its
  estimator) and its attestation is what earns a datum the
  `Attested` trust level (§2).
- **JouleClaw** — the reference cascade and frontier-model substrate behind the
  synthesis fallback (§6).
- **Proof / JouleContract** — the JCR-1 receipt format the surfacing log seals to
  (§8).
- **joule-ui** — the surface a highlight is rendered on; a surfaced highlight is
  itself a metered, sealed joule-ui interaction.
- **WAI** — the container for multimodal sensor payloads referenced by a `Cid`.

---

## 11. Surfacing into a running agent (REQUIRED where the recipient is an agent)

§5 decides whether an observation is worth attention; §8 seals that decision.
Neither says what the recipient *receives*. Where the recipient is a person, that
is right: the form is a UI concern and a non-goal (§1.6). Where the recipient is a
**running agent** it is not, because the delivery enters the agent's own record of
what it has been told — and three plausible forms each corrupt that record.

- **As a result on a call the agent made.** The agent made no such call. A record
  in which every result answers a call cannot hold one that answers nothing
  without ceasing to mean that.
- **As the principal's own words.** The principal did not say it. An auditor
  reading the record afterwards could no longer separate what the agent was
  *asked* from what it was *told*, and that distinction is most of why the record
  is worth keeping.
- **Outside the order.** What an agent did depends on when it learned a thing. An
  observation lifted out of the order silently changes what the record says the
  agent knew at each point before it.

A surfaced observation delivered to an agent is therefore an **injected
observation**: a distinctly-kinded frame occupying one position in the recipient's
order, which **MUST** state

- the **surfacing link** it carries out (§8), by the content address of that
  link's signed core — so the delivery cites a sealed decision instead of
  restating it, and the two cannot drift apart;
- the observed event's content id and its declared **trust level** (§2), bound in
  so a delivery cannot re-tag what it carries;
- its **destination** (below);
- the **authority** under which a parked recipient was woken, where one was
  needed;
- the **joules** the delivery charged — the cost of the recipient reading the
  frame, which is not the cost §5 already charged for deciding to surface it.

A host **MUST NOT** present an injected observation as a result, as the
principal's words, or outside the order. The injected kind **MUST NOT** be offered
to the agent as something it may author, and a host **MUST** refuse an attempt by
the recipient to author one: an agent able to author its own observations could
manufacture a record of having been told something, which is the one thing this
frame exists to witness.

### 11.1 Destination

A delivery states where it landed, because the two destinations say different
things about what the recipient knows.

- **`own`** — into the recipient's own record. Everything the recipient does
  afterwards was done knowing it.
- **`derived`** — into a derived record that inherits the recipient's context. The
  recipient's own record gains that record's *outcome*, not the observation, so
  the recipient never came to know the observation itself.

The destination **MUST** be stated rather than inferred from shape. An auditor
reading a recipient's record has to be able to tell that an observation was
handled elsewhere rather than never delivered.

### 11.2 What is refused

Four conditions are refused rather than recorded.

1. **A hold is not deliverable.** §8 chains holds precisely so a decision not to
   surface cannot be quietly forgotten; delivering one is the same fault in the
   other direction, and **MUST** be refused.
2. **The recipient is not the author.** Author and recipient **MUST** be distinct
   principals, as a steering amendment's amender must differ from its author
   (JCP §4.7).
3. **A wake needs an authority.** Where delivery wakes a parked run, the frame
   **MUST** name the authority that permitted it — the fire of a standing grant
   (JCP §4.6) — and delivery **MUST** be refused where none is held. An
   already-running recipient names none, which is a different fact from a wake
   nobody permitted, and the two **MUST NOT** share a shape.
4. **A delivery that happened is not free.** Something ran: the recipient read the
   frame. Zero is therefore not a measurement, and a delivery priced at zero
   **MUST** be refused. Where a host cannot measure, it estimates and labels
   (§7); it does not report nothing.

Conformance class **C6** exercises the delivery, each of the four refusals, and
that neither the destination nor the presence of a wake can be changed without
changing the frame's address.