open-standards/sense/SPEC.md

Sense v0.1 — A Standard for Ambient Context

Standard	sense
Status	v0.1.0 draft for review
Licence	Specification text CC-BY-4.0 (LICENSE-SPEC); reference code Apache-2.0 (LICENSE)
Reference	sense-rs/

A vendor-neutral standard for the ambient context substrate — the always-on, deterministic, energy-accounted record of what is happening around an agent, and a principled account of when that is worth a human's or another agent's attention. Where the tool protocols (MCP, A2A, ACP) standardize an agent reaching for capabilities, Sense standardizes the afferent half: the stream of observations flowing in, and the governed decision to surface them.

---

1. Scope and doctrine

1. Afferent, not efferent. Sense records the world the agent is in — file changes, focus, logs, thermals, shell activity — not the agent's own reasoning trace. The reasoning trace is observability's domain (OpenTelemetry GenAI), is non-deterministic by its own admission, and is out of scope here.
2. Deterministic by construction. Every observation is content-addressed and totally ordered under a hybrid logical clock. Replaying a log yields a byte-identical projection on any machine (§3). Determinism is what makes an ambient record auditable rather than merely logged.
3. Perception ⊗ attention-budget. Observing is cheap; interrupting a human is not. Sense pairs a salience projection (§4) with an attention budget (§5) that decides whether an observation surfaces, scoring its value against a modelled interruption cost and pricing the surfacing in joules. This is the afferent dual of JCP's capability ⊗ joule-budget.
4. Inference is the last resort. A question about the observed world is answered from the log by a deterministic recall cascade; a model is called only when nothing cheaper can answer (§6). Cheapest sufficient answer wins.
5. Provenance and trust are declared, never promoted. Every datum carries its source class and a trust level (§2); every energy figure carries its provenance (§7). A host may downgrade what it is unsure of; it must never silently upgrade.
6. Non-goals. Sense does not mandate a sensor set, a UI, a model, a transport, or a storage engine. It standardizes the observed datum, the determinism, the attention decision, the cascade accounting, and the receipt — not the daemon.

2. The observed event (REQUIRED)

An Event is one datum on the log:

• hlc — a hybrid logical clock stamp; the event's place in the total order.
• source — a SourceClass: the vendor-neutral class of the signal (file_change, app_focus, system_log, process_tree, thermal_power, shell, accessibility, network, endpoint, salience). A host maps its native sensors onto these classes — macOS FSEvents, Linux inotify and Windows ReadDirectoryChangesW all surface as file_change — so a log means the same thing on every platform. Codes are stable and append-only; vendor-specific classes use vendor:<code> with code ≥ 1024.
• valid_time_ns — optional wall-clock time at observation, captured as data, not as the clock (so replay stays deterministic when the wall clock jumps).
• trust — a TrustLevel: Attested (cryptographically vouched, e.g. a Sandbox attested capture), FirstParty (a first-party OS API the host called), Inferred (parsed from an untrusted byte stream), or Untrusted (the default).
• payload — the content address (§3) of the raw bytes; the bytes live in the store's content-addressed table, recoverable from the address alone.
• line — the textual projection the attention and synthesis tiers read.

An event's content id is blake3 over a fixed-order, length-prefixed concatenation of its fields. The id binds the source class and trust level, so an event cannot be re-classified or re-trusted without changing its id (conformance C2).

3. Canonical serialization and determinism (REQUIRED)

• Content address. A Cid is blake3(payload_bytes), lowercase-hex on the wire. Identical bytes always yield the identical address; storage is the index.
• Total order. Events are keyed by an Hlc = (physical_ns, logical). The physical component is a monotonic anchor; logical breaks ties at the same instant. The clock is totally ordered within a daemon run; on restart a host seeds it past the maximum stored stamp so new stamps strictly exceed all stored ones.
• Replay determinism. events_since(Hlc::ZERO) replays the whole log in clock order. Two hosts fed the same event sequence produce a byte-identical replay digest — blake3 over each event's content id in order (conformance C1). A production host MAY persist to an MVCC B-tree keyed by the HLC big-endian bytes; the wire and the order are fixed here, not the engine.
• Receipt core (§8) uses canonical det-CBOR (RFC 8949 §4.2.1: length-first then bytewise key ordering, shortest integer form, no floats/bools/nulls), the same encoding the family's JCR-1 receipt signs over.

4. The salience projection (REQUIRED)

A bounded-state observer consumes the event stream and projects three scalars in [0, 1]:

• salience — is the current source bursting? (the fraction of a recency window occupied by the current source class).
• novelty — is the current source rare? (one minus its share of all traffic seen).
• uncertainty — is the recent window low-trust? (the fraction of the window at Inferred/Untrusted). An honesty signal: "the system isn't sure what's going on."

The reference observer computes these from exact integer counts, so two runs over the same events project byte-identically. A host MAY substitute a model observer (e.g. a recurrent bounded-state network); the standard requires only that a conforming observer be deterministic under replay, and that the three scalars carry these meanings.

5. The attention budget (REQUIRED)

A continuous projection is not yet a decision. A detector turns it into discrete Highlights by an EMA-baselined k-sigma rule with an absolute floor and a debounce — it fires on anomalies relative to a probe's own recent baseline, above an absolute floor, not on every reading. The decision is made against the prior baseline (then the baseline updates) so a single large spike does not suppress its own fire.

A highlight is then gated by the attention budget before it reaches a human or agent. Following the proactive-interruption literature, the gate scores:

• value — the highlight's probe score, with uncertainty highlights discounted (an honesty signal is usually less worth an interruption than a true salience or novelty spike).
• interruption cost — displacement (proportional to how engaged the user is) plus interference (a penalty when the surfacing would occupy the user's currently-active modality — a visual alert while the user reads).

A highlight surfaces only when value > cost + margin and the energy budget can pay the surfacing's joule price. The three outcomes are surface, hold:value (value did not clear cost), and hold:budget (no joules left). The value test comes first: a highlight that is not worth an interruption is hold:value whether or not the budget could pay for it, so hold:budget means a worthy highlight the budget could not afford. The decision is deterministic in its inputs (conformance C3). Surfacing debits the budget; a hold spends nothing. This makes attention a metered resource: an ambient agent cannot interrupt without account, and cannot interrupt past its energy budget at all.

6. The synthesis cascade (REQUIRED)

When a human or agent asks a question of the log, the host builds an Envelope — the top-k retrieved spans + the salience projection + the query — and walks a speculative cascade:

1. Deterministic recall first. A Tier-0 stage recognises a small set of factual-recall intents (recent files, last command, what was surfaced, …) and answers directly from the envelope's spans, with no model call. It is conservative: it answers only when query-side and span-side evidence agree, and escalates when unsure (a missed answer costs a paid call; a wrong answer is worse).
2. Escalate as a last resort. Only when no cheap stage answers does the request reach the fallback — typically a frontier model.

Retrieval is BM25-lite over exact integer term frequencies, then source-stratified recency fill for any empty slots (natural-language queries rarely overlap span text by token, and an empty envelope answers nothing). Spans are content-addressed, so an envelope built from the same log + query is byte-identical and a provider-side prompt cache hits cleanly. Every answer carries an honest Energy figure (a deterministic answer is an honest zero estimate, never a measurement).

7. Energy and provenance (REQUIRED)

Every metered figure — a surfacing's price, a synthesis's cost — is an integer microjoule pair: measured_uj (present only from a live hardware interface) and estimated_uj (always present), tagged with a Provenance (HwShunt, ModelBased, Estimator). This is the family's shared energy model: a figure is reported with the honesty of how it was obtained and never promoted. The honesty invariant is that HwShunt requires a measurement and Estimator forbids one.

A reading from a vendor energy counter (RAPL, NVML, Apple IOReport, ...) MAY be graded as a hardware reading (HwShunt, with measured_uj) only while the counter is demonstrably live for the domain the figure claims: the implementation MUST have observed the counter covering that domain advance under load (for Apple IOReport: the CPU energy rail itself advancing in at least 5 sample intervals, in at least 4 of every 5, over at least 300 ms while the probing thread burns CPU; an interval runs from when one sample is in hand to when the next is, and only one no longer than three of the probe's ticks, 75 ms at its 25 ms tick, counts: a longer one counts as nothing, neither among the intervals, nor as an advance, nor towards the 300 ms, which is the summed length of the intervals that count). A counter that is present but not live — frozen, or republished in lumps — MUST NOT produce a hardware-graded figure; the implementation falls back to its estimator or reports the work unmetered, and SHOULD record why. A figure MUST NOT claim coverage of a domain its counter does not read; where the format has a coverage field, such a figure is Partial.

Measurement itself is a recognized edge: the sibling Sandbox standard's PowerMeter is the reference meter — it reads RAPL, NVML or IOReport, grades a reading HwShunt only while its counter is live, and falls back to its estimator when it is not. Sense defines what an interaction records; the host wires the meter it trusts.

8. The sealed, hash-chained surfacing log (REQUIRED)

An ambient agent's audit duty is the inverse of a tool agent's: it must show it did not observe-and-surface without account, and that no record of what it surfaced (or chose to hold) was quietly removed. Every gate decision (§5) therefore seals as the next link of a hash chain. The signed core commits to the source event's content id, its trust level, the highlight, the value/cost the budget computed, the decision, the joules charged, and the budget left; each link's prev is the BLAKE3 content id of the previous link's canonical core, and seq increments from zero.

Allows and holds are both chained — a held highlight is exactly the record an adversarial host would prefer to forget. Excising, reordering, or editing any link breaks the chain at that point (conformance C5). The core signs with the host's Ed25519 key; verify_chain checks every signature, the sequence, and every prev link, and reports the first break. A link MAY additionally be expressed as a JCR-1 COSE_Sign1 envelope for verification by the family's shared cross-language tooling (a recognized edge, not required — the hash chain is the primary audit).

9. Conformance

A conforming implementation reproduces, from the published vectors alone (conformance/v1/):

• C1 — capture determinism. A fixed event sequence replays to the published replay_digest; two builds agree.
• C2 — provenance & trust. A fixed event's content id matches the published event_id; re-tagging its source class or trust level changes the id.
• C3 — attention. A fixed signal stream yields the published sequence of highlights and surface/hold decisions, and the same highlights gated for a busy user on a small budget yield every outcome — surface, hold:value, hold:budget — with the published budget after each.
• C4 — synthesis cascade. A recall query is answered with no model call (deterministic); an open-ended query escalates.
• C5 — chain integrity. A surfacing log verifies (verify_chain → n), and excising the middle link breaks it at the published index.
• C6 — injected observations (§11). A sealed surfacing delivers into a running agent's record and the frame carries the link's own facts; a hold, a self-authored observation, a wake with no authority, and a delivery priced at zero are each refused; and neither the destination nor the presence of a wake can change without changing the frame's address.

10. Recognized edges

Sense is one organ in the family; each boundary it leans on is a published sibling standard, not a dependency it owns:

• JCP (capability ⊗ joule-budget) — gates who may subscribe to which source class, for how many joules. Sense is the afferent dual.
• Sandbox — runs the capture in metered, attested isolation; its PowerMeter is the reference meter (§7: a vendor counter only while live, else its estimator) and its attestation is what earns a datum the Attested trust level (§2).
• JouleClaw — the reference cascade and frontier-model substrate behind the synthesis fallback (§6).
• Proof / JouleContract — the JCR-1 receipt format the surfacing log seals to (§8).
• joule-ui — the surface a highlight is rendered on; a surfaced highlight is itself a metered, sealed joule-ui interaction.
• WAI — the container for multimodal sensor payloads referenced by a Cid.

---

11. Surfacing into a running agent (REQUIRED where the recipient is an agent)

§5 decides whether an observation is worth attention; §8 seals that decision. Neither says what the recipient receives. Where the recipient is a person, that is right: the form is a UI concern and a non-goal (§1.6). Where the recipient is a running agent it is not, because the delivery enters the agent's own record of what it has been told — and three plausible forms each corrupt that record.

• As a result on a call the agent made. The agent made no such call. A record in which every result answers a call cannot hold one that answers nothing without ceasing to mean that.
• As the principal's own words. The principal did not say it. An auditor reading the record afterwards could no longer separate what the agent was asked from what it was told, and that distinction is most of why the record is worth keeping.
• Outside the order. What an agent did depends on when it learned a thing. An observation lifted out of the order silently changes what the record says the agent knew at each point before it.

A surfaced observation delivered to an agent is therefore an injected observation: a distinctly-kinded frame occupying one position in the recipient's order, which MUST state

• the surfacing link it carries out (§8), by the content address of that link's signed core — so the delivery cites a sealed decision instead of restating it, and the two cannot drift apart;
• the observed event's content id and its declared trust level (§2), bound in so a delivery cannot re-tag what it carries;
• its destination (below);
• the authority under which a parked recipient was woken, where one was needed;
• the joules the delivery charged — the cost of the recipient reading the frame, which is not the cost §5 already charged for deciding to surface it.

A host MUST NOT present an injected observation as a result, as the principal's words, or outside the order. The injected kind MUST NOT be offered to the agent as something it may author, and a host MUST refuse an attempt by the recipient to author one: an agent able to author its own observations could manufacture a record of having been told something, which is the one thing this frame exists to witness.

11.1 Destination

A delivery states where it landed, because the two destinations say different things about what the recipient knows.

• own — into the recipient's own record. Everything the recipient does afterwards was done knowing it.
• derived — into a derived record that inherits the recipient's context. The recipient's own record gains that record's outcome, not the observation, so the recipient never came to know the observation itself.

The destination MUST be stated rather than inferred from shape. An auditor reading a recipient's record has to be able to tell that an observation was handled elsewhere rather than never delivered.

11.2 What is refused

Four conditions are refused rather than recorded.

1. A hold is not deliverable. §8 chains holds precisely so a decision not to surface cannot be quietly forgotten; delivering one is the same fault in the other direction, and MUST be refused.
2. The recipient is not the author. Author and recipient MUST be distinct principals, as a steering amendment's amender must differ from its author (JCP §4.7).
3. A wake needs an authority. Where delivery wakes a parked run, the frame MUST name the authority that permitted it — the fire of a standing grant (JCP §4.6) — and delivery MUST be refused where none is held. An already-running recipient names none, which is a different fact from a wake nobody permitted, and the two MUST NOT share a shape.
4. A delivery that happened is not free. Something ran: the recipient read the frame. Zero is therefore not a measurement, and a delivery priced at zero MUST be refused. Where a host cannot measure, it estimates and labels (§7); it does not report nothing.

Conformance class C6 exercises the delivery, each of the four refusals, and that neither the destination nor the presence of a wake can be changed without changing the frame's address.