# Sense v0.1 — A Standard for Ambient Context | | | |---|---| | **Standard** | sense | | **Status** | v0.1.0 draft for review | | **Licence** | Specification text CC-BY-4.0 ([`LICENSE-SPEC`](LICENSE-SPEC)); reference code Apache-2.0 ([`LICENSE`](LICENSE)) | | **Reference** | [`sense-rs/`](sense-rs/) | A vendor-neutral standard for the **ambient context substrate** — the always-on, deterministic, energy-accounted record of *what is happening around an agent*, and a principled account of *when that is worth a human's or another agent's attention*. Where the tool protocols (MCP, A2A, ACP) standardize an agent **reaching for** capabilities, Sense standardizes the **afferent** half: the stream of observations flowing in, and the governed decision to surface them. --- ## 1. Scope and doctrine 1. **Afferent, not efferent.** Sense records the *world the agent is in* — file changes, focus, logs, thermals, shell activity — not the agent's own reasoning trace. The reasoning trace is observability's domain (OpenTelemetry GenAI), is non-deterministic by its own admission, and is out of scope here. 2. **Deterministic by construction.** Every observation is content-addressed and totally ordered under a hybrid logical clock. Replaying a log yields a byte-identical projection on any machine (§3). Determinism is what makes an ambient record *auditable* rather than merely *logged*. 3. **Perception ⊗ attention-budget.** Observing is cheap; interrupting a human is not. Sense pairs a salience projection (§4) with an **attention budget** (§5) that decides whether an observation surfaces, scoring its value against a modelled interruption cost and pricing the surfacing in joules. This is the afferent dual of JCP's *capability ⊗ joule-budget*. 4. **Inference is the last resort.** A question about the observed world is answered from the log by a deterministic recall cascade; a model is called only when nothing cheaper can answer (§6). Cheapest sufficient answer wins. 5. **Provenance and trust are declared, never promoted.** Every datum carries its source class and a **trust level** (§2); every energy figure carries its provenance (§7). A host may downgrade what it is unsure of; it must never silently upgrade. 6. **Non-goals.** Sense does not mandate a sensor set, a UI, a model, a transport, or a storage engine. It standardizes the *observed datum*, the *determinism*, the *attention decision*, the *cascade accounting*, and the *receipt* — not the daemon. ## 2. The observed event (REQUIRED) An **Event** is one datum on the log: - `hlc` — a [hybrid logical clock](#3-canonical-serialization-and-determinism) stamp; the event's place in the total order. - `source` — a **SourceClass**: the vendor-neutral class of the signal (`file_change`, `app_focus`, `system_log`, `process_tree`, `thermal_power`, `shell`, `accessibility`, `network`, `endpoint`, `salience`). A host maps its native sensors onto these classes — macOS FSEvents, Linux inotify and Windows `ReadDirectoryChangesW` all surface as `file_change` — so a log means the same thing on every platform. Codes are stable and append-only; vendor-specific classes use `vendor:` with `code ≥ 1024`. - `valid_time_ns` — optional wall-clock time at observation, captured **as data**, not as the clock (so replay stays deterministic when the wall clock jumps). - `trust` — a **TrustLevel**: `Attested` (cryptographically vouched, e.g. a Sandbox attested capture), `FirstParty` (a first-party OS API the host called), `Inferred` (parsed from an untrusted byte stream), or `Untrusted` (the default). - `payload` — the content address (§3) of the raw bytes; the bytes live in the store's content-addressed table, recoverable from the address alone. - `line` — the textual projection the attention and synthesis tiers read. An event's **content id** is `blake3` over a fixed-order, length-prefixed concatenation of its fields. The id binds the source class and trust level, so an event cannot be re-classified or re-trusted without changing its id (conformance C2). ## 3. Canonical serialization and determinism (REQUIRED) - **Content address.** A `Cid` is `blake3(payload_bytes)`, lowercase-hex on the wire. Identical bytes always yield the identical address; storage *is* the index. - **Total order.** Events are keyed by an `Hlc = (physical_ns, logical)`. The physical component is a monotonic anchor; `logical` breaks ties at the same instant. The clock is totally ordered within a daemon run; on restart a host seeds it past the maximum stored stamp so new stamps strictly exceed all stored ones. - **Replay determinism.** `events_since(Hlc::ZERO)` replays the whole log in clock order. Two hosts fed the same event sequence produce a byte-identical **replay digest** — `blake3` over each event's content id in order (conformance C1). A production host MAY persist to an MVCC B-tree keyed by the HLC big-endian bytes; the wire and the order are fixed here, not the engine. - **Receipt core (§8)** uses canonical det-CBOR (RFC 8949 §4.2.1: length-first then bytewise key ordering, shortest integer form, no floats/bools/nulls), the same encoding the family's JCR-1 receipt signs over. ## 4. The salience projection (REQUIRED) A **bounded-state observer** consumes the event stream and projects three scalars in `[0, 1]`: - **salience** — is the current source *bursting*? (the fraction of a recency window occupied by the current source class). - **novelty** — is the current source *rare*? (one minus its share of all traffic seen). - **uncertainty** — is the recent window *low-trust*? (the fraction of the window at `Inferred`/`Untrusted`). An honesty signal: "the system isn't sure what's going on." The reference observer computes these from **exact integer counts**, so two runs over the same events project byte-identically. A host MAY substitute a model observer (e.g. a recurrent bounded-state network); the standard requires only that a *conforming* observer be deterministic under replay, and that the three scalars carry these meanings. ## 5. The attention budget (REQUIRED) A continuous projection is not yet a decision. A **detector** turns it into discrete **Highlights** by an EMA-baselined *k*-sigma rule with an absolute floor and a debounce — it fires on *anomalies relative to a probe's own recent baseline*, above an absolute floor, not on every reading. The decision is made against the *prior* baseline (then the baseline updates) so a single large spike does not suppress its own fire. A highlight is then **gated** by the attention budget before it reaches a human or agent. Following the proactive-interruption literature, the gate scores: - **value** — the highlight's probe score, with `uncertainty` highlights discounted (an honesty signal is usually less worth an interruption than a true salience or novelty spike). - **interruption cost** — *displacement* (proportional to how engaged the user is) plus *interference* (a penalty when the surfacing would occupy the user's currently-active modality — a visual alert while the user reads). A highlight **surfaces** only when `value > cost + margin` **and** the energy budget can pay the surfacing's joule price. The three outcomes are `surface`, `hold:value` (value did not clear cost), and `hold:budget` (no joules left). The value test comes first: a highlight that is not worth an interruption is `hold:value` whether or not the budget could pay for it, so `hold:budget` means a worthy highlight the budget could not afford. The decision is deterministic in its inputs (conformance C3). Surfacing debits the budget; a hold spends nothing. This makes attention a **metered resource**: an ambient agent cannot interrupt without account, and cannot interrupt past its energy budget at all. ## 6. The synthesis cascade (REQUIRED) When a human or agent asks a question of the log, the host builds an **Envelope** — the top-*k* retrieved spans + the salience projection + the query — and walks a **speculative cascade**: 1. **Deterministic recall first.** A Tier-0 stage recognises a small set of factual-recall intents (recent files, last command, what was surfaced, …) and answers directly from the envelope's spans, with **no model call**. It is conservative: it answers only when query-side and span-side evidence agree, and escalates when unsure (a missed answer costs a paid call; a wrong answer is worse). 2. **Escalate as a last resort.** Only when no cheap stage answers does the request reach the fallback — typically a frontier model. Retrieval is BM25-lite over exact integer term frequencies, then **source-stratified recency fill** for any empty slots (natural-language queries rarely overlap span text by token, and an empty envelope answers nothing). Spans are content-addressed, so an envelope built from the same log + query is byte-identical and a provider-side prompt cache hits cleanly. Every answer carries an honest [`Energy`](#7-energy-and-provenance) figure (a deterministic answer is an honest zero estimate, never a measurement). ## 7. Energy and provenance (REQUIRED) Every metered figure — a surfacing's price, a synthesis's cost — is an integer microjoule pair: `measured_uj` (present only from a live hardware interface) and `estimated_uj` (always present), tagged with a **Provenance** (`HwShunt`, `ModelBased`, `Estimator`). This is the family's shared energy model: a figure is reported with the honesty of how it was obtained and **never promoted**. The honesty invariant is that `HwShunt` requires a measurement and `Estimator` forbids one. A reading from a vendor energy counter (RAPL, NVML, Apple IOReport, ...) **MAY** be graded as a hardware reading (`HwShunt`, with `measured_uj`) only while the counter is demonstrably live for the domain the figure claims: the implementation **MUST** have observed the counter covering that domain advance under load (for Apple IOReport: the CPU energy rail itself advancing in at least 5 sample intervals, in at least 4 of every 5, over at least 300 ms while the probing thread burns CPU). A counter that is present but not live — frozen, or republished in lumps — **MUST NOT** produce a hardware-graded figure; the implementation falls back to its estimator or reports the work unmetered, and **SHOULD** record why. A figure **MUST NOT** claim coverage of a domain its counter does not read; where the format has a coverage field, such a figure is Partial. Measurement itself is a **recognized edge**: the sibling **Sandbox** standard's `PowerMeter` is the reference meter — it reads RAPL, NVML or IOReport, grades a reading `HwShunt` only while its counter is live, and falls back to its estimator when it is not. Sense defines what an interaction records; the host wires the meter it trusts. ## 8. The sealed, hash-chained surfacing log (REQUIRED) An ambient agent's audit duty is the inverse of a tool agent's: it must show it did not observe-and-surface without account, and that no record of what it surfaced (or chose to hold) was quietly removed. Every gate decision (§5) therefore seals as the next link of a **hash chain**. The signed core commits to the source event's content id, its trust level, the highlight, the value/cost the budget computed, the decision, the joules charged, and the budget left; each link's `prev` is the BLAKE3 content id of the previous link's canonical core, and `seq` increments from zero. **Allows and holds are both chained** — a held highlight is exactly the record an adversarial host would prefer to forget. Excising, reordering, or editing any link breaks the chain at that point (conformance C5). The core signs with the host's Ed25519 key; `verify_chain` checks every signature, the sequence, and every `prev` link, and reports the first break. A link MAY additionally be expressed as a JCR-1 COSE_Sign1 envelope for verification by the family's shared cross-language tooling (a recognized edge, not required — the hash chain is the primary audit). ## 9. Conformance A conforming implementation reproduces, from the published vectors alone ([`conformance/v1/`](conformance/v1/)): - **C1 — capture determinism.** A fixed event sequence replays to the published `replay_digest`; two builds agree. - **C2 — provenance & trust.** A fixed event's content id matches the published `event_id`; re-tagging its source class or trust level changes the id. - **C3 — attention.** A fixed signal stream yields the published sequence of highlights and surface/hold decisions, and the same highlights gated for a busy user on a small budget yield every outcome — `surface`, `hold:value`, `hold:budget` — with the published budget after each. - **C4 — synthesis cascade.** A recall query is answered with no model call (`deterministic`); an open-ended query escalates. - **C5 — chain integrity.** A surfacing log verifies (`verify_chain → n`), and excising the middle link breaks it at the published index. - **C6 — injected observations (§11).** A sealed surfacing delivers into a running agent's record and the frame carries the link's own facts; a hold, a self-authored observation, a wake with no authority, and a delivery priced at zero are each refused; and neither the destination nor the presence of a wake can change without changing the frame's address. ## 10. Recognized edges Sense is one organ in the family; each boundary it leans on is a published sibling standard, not a dependency it owns: - **JCP** (capability ⊗ joule-budget) — gates *who* may subscribe to *which* source class, for *how many* joules. Sense is the afferent dual. - **Sandbox** — runs the capture in metered, attested isolation; its `PowerMeter` is the reference meter (§7: a vendor counter only while live, else its estimator) and its attestation is what earns a datum the `Attested` trust level (§2). - **JouleClaw** — the reference cascade and frontier-model substrate behind the synthesis fallback (§6). - **Proof / JouleContract** — the JCR-1 receipt format the surfacing log seals to (§8). - **joule-ui** — the surface a highlight is rendered on; a surfaced highlight is itself a metered, sealed joule-ui interaction. - **WAI** — the container for multimodal sensor payloads referenced by a `Cid`. --- ## 11. Surfacing into a running agent (REQUIRED where the recipient is an agent) §5 decides whether an observation is worth attention; §8 seals that decision. Neither says what the recipient *receives*. Where the recipient is a person, that is right: the form is a UI concern and a non-goal (§1.6). Where the recipient is a **running agent** it is not, because the delivery enters the agent's own record of what it has been told — and three plausible forms each corrupt that record. - **As a result on a call the agent made.** The agent made no such call. A record in which every result answers a call cannot hold one that answers nothing without ceasing to mean that. - **As the principal's own words.** The principal did not say it. An auditor reading the record afterwards could no longer separate what the agent was *asked* from what it was *told*, and that distinction is most of why the record is worth keeping. - **Outside the order.** What an agent did depends on when it learned a thing. An observation lifted out of the order silently changes what the record says the agent knew at each point before it. A surfaced observation delivered to an agent is therefore an **injected observation**: a distinctly-kinded frame occupying one position in the recipient's order, which **MUST** state - the **surfacing link** it carries out (§8), by the content address of that link's signed core — so the delivery cites a sealed decision instead of restating it, and the two cannot drift apart; - the observed event's content id and its declared **trust level** (§2), bound in so a delivery cannot re-tag what it carries; - its **destination** (below); - the **authority** under which a parked recipient was woken, where one was needed; - the **joules** the delivery charged — the cost of the recipient reading the frame, which is not the cost §5 already charged for deciding to surface it. A host **MUST NOT** present an injected observation as a result, as the principal's words, or outside the order. The injected kind **MUST NOT** be offered to the agent as something it may author, and a host **MUST** refuse an attempt by the recipient to author one: an agent able to author its own observations could manufacture a record of having been told something, which is the one thing this frame exists to witness. ### 11.1 Destination A delivery states where it landed, because the two destinations say different things about what the recipient knows. - **`own`** — into the recipient's own record. Everything the recipient does afterwards was done knowing it. - **`derived`** — into a derived record that inherits the recipient's context. The recipient's own record gains that record's *outcome*, not the observation, so the recipient never came to know the observation itself. The destination **MUST** be stated rather than inferred from shape. An auditor reading a recipient's record has to be able to tell that an observation was handled elsewhere rather than never delivered. ### 11.2 What is refused Four conditions are refused rather than recorded. 1. **A hold is not deliverable.** §8 chains holds precisely so a decision not to surface cannot be quietly forgotten; delivering one is the same fault in the other direction, and **MUST** be refused. 2. **The recipient is not the author.** Author and recipient **MUST** be distinct principals, as a steering amendment's amender must differ from its author (JCP §4.7). 3. **A wake needs an authority.** Where delivery wakes a parked run, the frame **MUST** name the authority that permitted it — the fire of a standing grant (JCP §4.6) — and delivery **MUST** be refused where none is held. An already-running recipient names none, which is a different fact from a wake nobody permitted, and the two **MUST NOT** share a shape. 4. **A delivery that happened is not free.** Something ran: the recipient read the frame. Zero is therefore not a measurement, and a delivery priced at zero **MUST** be refused. Where a host cannot measure, it estimates and labels (§7); it does not report nothing. Conformance class **C6** exercises the delivery, each of the four refusals, and that neither the destination nor the presence of a wake can be changed without changing the frame's address.