open-standards/jouleclaw/jouleclaw-rs/crates/jouleclaw-stack/examples/wave47_streaming_pipeline.rs
Transaction Science e3ed89cd95 jouleclaw: wave-47 — streaming-cascade substrate
Today's research confirmed streaming / continuous-interaction is the
2026 load-bearing AI shape: Thinking Machines shipped Interaction
Models (May 11, 2026) with parallel continuous streams + 200ms
micro-turns + 0.40s response latency; the τ-Voice ICASSP-2026
benchmark canonical-quantizes at 200ms ticks; production voice
agents (OpenAI gpt-realtime, Google gemini-live, xAI grok-voice)
all converge on <150ms barge-in. Today's JouleClaw cascade assumes
request → walk-tiers → answer; that shape doesn't fit.

This wave is the substrate for the streaming shape, typed up so any
downstream model / network / codec slots in without rewriting.

1. jouleclaw-stream (new crate) — pure types.
   - Tick { window_id, channel_id, kind: TickKind { Audio | Text |
     Video | Marker { TickMarker { Start | End | BargeIn | Silence
     | EndOfTurn | Custom } } }, bytes, ts_ns }
   - StreamSpec { tick_window_ms, max_chunk_bytes, max_ticks_per_
     window, channels: Vec<ChannelSpec { id, direction, kind_tag,
     mime }> } + duplex_voice_default()
   - StreamWindow with push validation + has_barge_in + content_hash
   - RollingConservationLedger: streaming analog of the per-request
     DeterministicVerifier. Tracks inbound + outbound + substrate
     bytes per window over a rolling buffer; surfaces a
     WindowConservationFinding { Conforming | Hallucination |
     BargeInIgnored } per tick. BargeInIgnored is the load-bearing
     check: if a barge-in fires and the cascade is still emitting
     outbound past the deadline, the ledger flags it on the spot.
   - DEFAULT_TICK_WINDOW_MS = 200; BARGE_IN_DEADLINE_MS = 150 —
     both lifted from production voice-agent guidance.
   - 16 tests.

2. jouleclaw-stream-oracle (new crate) — gate-per-window Oracle.
   - StreamOracle trait + WindowVerdict { Pass | Fail | Yield |
     NeedMore }. Yield is the load-bearing new variant — the
     cascade MUST stop emitting outbound within
     BARGE_IN_DEADLINE_MS. NeedMore lets the Oracle defer commitment
     across windows.
   - Reference backends: AlwaysPassStream, BargeInDetector,
     PerTickAdapter<O: Oracle> (lifts the per-request Oracle into
     the streaming surface so the existing fuzz + future Lean
     backends compose).
   - CompositeStreamOracle: unlike the per-request CompositeOracle
     that short-circuits on first Fail, the streaming composite
     ALWAYS runs every backend and combines by dominance rank
     (Fail > Yield > NeedMore > Pass). A window can carry a
     barge-in AND a content failure; both signals matter.
   - 10 tests.

3. jouleclaw-jcr1 — stream-window/1 schema.
   - StreamWindowReceipt: jc_receipt + id + closed_at + steward +
     stream_id + window_id + window_hash (BLAKE3 of canonical
     StreamWindow JSON) + tick_count + inbound/outbound bytes +
     joules_uj + energy_provenance + WindowVerdictWire +
     WindowConservationWire.
   - Canonical JSON with sorted keys + content-addressed via
     BLAKE3, ready for COSE_Sign1 over the canonical bytes via the
     existing v1 signing path.
   - 3 conformance vectors (Pass / Yield / Hallucination) with
     embedded content_hash the Rust verifier asserts byte-equal at
     test time. Other-language verifiers must round-trip under
     canonical encoding and match the hash.
   - 4 stream_window tests on top of the existing jcr1 surface.

4. Demo binary — examples/wave47_streaming_pipeline.rs in
   jouleclaw-stack. Simulates a duplex-voice stream over 13 windows
   (~2.6 s at 200 ms each). Builds a CompositeStreamOracle (always-
   pass + barge-in detector + per-tick adapter wrapping a per-
   request Oracle), runs each window through the oracle + the
   conservation ledger, mints a stream-window/1 receipt per window.
   The yield correctly fires on the barge-in window; hallucinations
   flag when outbound bursts past the rolling budget.

The triangle is the type system; wave-46 filled per-request backend
slots; wave-47 extends the same triangle to streaming windows
without rewriting the triangle itself. Stream Oracles compose with
per-request Oracles via PerTickAdapter; the rolling conservation
ledger surfaces the streaming analog of the same Conforming /
Hallucination findings the per-request verifier emits; the JCR-1
receipt schema mirrors v1's wire shape so existing infrastructure
consumes it without surgery.

Workspace check clean.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-06-05 04:12:16 -04:00

372 lines
12 KiB
Rust

//! `wave47_streaming_pipeline` — streaming-cascade substrate demo.
//!
//! Simulates a duplex-voice stream of audio + marker ticks over
//! 13 windows (~2.6 s at 200 ms/window). For each window the
//! pipeline runs:
//!
//! 1. **Stream Oracle** (CompositeStreamOracle = always-pass +
//! barge-in detector + per-tick adapter wrapping the no-`unsafe`
//! fuzz property on the text channel) — emits a [`WindowVerdict`].
//! Dominance order: Fail > Yield > NeedMore > Pass.
//! 2. **Rolling conservation ledger** (4-window buffer, 256 B
//! tolerance) — emits a [`WindowConservationFinding`]. The
//! barge-in deadline is 1 window; if the agent keeps emitting
//! outbound past it, the ledger flags `BargeInIgnored`.
//! 3. **JCR-1 `stream-window/1` receipt** — minted per window with
//! verdict + conservation + joules. Canonical JSON, BLAKE3-
//! addressed, ready for COSE_Sign1.
//!
//! Run:
//! cargo run -p jouleclaw-stack --example wave47_streaming_pipeline
use jouleclaw_jcr1::{
EnergyProvenanceWire, StreamWindowReceipt, WindowConservationWire, WindowVerdictWire,
};
use jouleclaw_oracle::AlwaysPassOracle;
use jouleclaw_stream::{
BARGE_IN_DEADLINE_MS, ChannelDirection, ChannelSpec, DEFAULT_TICK_WINDOW_MS,
RollingConservationLedger, StreamSpec, StreamWindow, Tick, TickMarker, WindowConservationFinding,
WindowId,
};
use jouleclaw_stream_oracle::{
AlwaysPassStream, BargeInDetector, CompositeStreamOracle, PerTickAdapter, StreamOracle,
WindowVerdict,
};
const STEWARD: &str = "did:keri:EJ9testStewardAID000000000000000";
const STREAM_ID: &str = "demo-duplex";
const CLOSED_AT_BASE: u64 = 1_780_000_000;
fn duplex_spec_with_text() -> StreamSpec {
let mut base = StreamSpec::duplex_voice_default();
// Add an outbound text channel so we can demo the per-tick
// adapter wrapping the no-`unsafe` property.
base.channels.push(ChannelSpec {
id: "text.out".into(),
direction: ChannelDirection::Outbound,
kind_tag: "text".into(),
mime: "text/plain;charset=utf-8".into(),
});
base
}
/// One scripted window of the simulated stream.
struct ScriptedWindow {
id: u64,
inbound_b: usize,
outbound_b: usize,
text_out: Option<&'static str>,
marker: Option<TickMarker>,
joules_uj: u64,
}
fn build_window(spec: &StreamSpec, w: &ScriptedWindow) -> StreamWindow {
let id = WindowId(w.id);
let mut window = StreamWindow::new(id);
if w.inbound_b > 0 {
window
.push(spec, Tick::audio(id, "audio.in", vec![0u8; w.inbound_b], 0))
.expect("inbound");
}
if w.outbound_b > 0 {
window
.push(
spec,
Tick::audio(id, "audio.out", vec![0u8; w.outbound_b], 100),
)
.expect("outbound");
}
if let Some(t) = w.text_out {
window
.push(spec, Tick::text(id, "text.out", t, 150))
.expect("text");
}
if let Some(m) = &w.marker {
window
.push(spec, Tick::marker(id, "markers", m.clone(), 200))
.expect("marker");
}
window
}
fn verdict_to_wire(v: &WindowVerdict) -> WindowVerdictWire {
match v {
WindowVerdict::Pass { chain } => WindowVerdictWire::Pass {
chain: chain.clone(),
},
WindowVerdict::Fail { reasons } => WindowVerdictWire::Fail {
reasons: reasons
.iter()
.map(|r| serde_json::to_string(r).unwrap_or_else(|_| "fail".into()))
.collect(),
},
WindowVerdict::Yield {
reason: jouleclaw_stream_oracle::WindowFailReason::BargeIn { seen_at },
} => WindowVerdictWire::Yield {
seen_at: seen_at.0,
},
WindowVerdict::Yield { reason } => WindowVerdictWire::Yield {
seen_at: match reason {
jouleclaw_stream_oracle::WindowFailReason::BargeIn { seen_at } => seen_at.0,
_ => 0,
},
},
WindowVerdict::NeedMore { detail } => WindowVerdictWire::NeedMore {
detail: detail.clone(),
},
}
}
fn conservation_to_wire(f: &WindowConservationFinding) -> WindowConservationWire {
match f {
WindowConservationFinding::Conforming => WindowConservationWire::Conforming,
WindowConservationFinding::Hallucination { unaccounted } => {
WindowConservationWire::Hallucination {
unaccounted: *unaccounted as u64,
}
}
WindowConservationFinding::BargeInIgnored { fired_at } => {
WindowConservationWire::BargeInIgnored {
fired_at: fired_at.0,
}
}
}
}
#[tokio::main(flavor = "current_thread")]
async fn main() {
println!(
"== wave-47 streaming pipeline demo (tick={} ms, barge-in deadline={} ms) ==\n",
DEFAULT_TICK_WINDOW_MS, BARGE_IN_DEADLINE_MS
);
let spec = duplex_spec_with_text();
// Stream Oracle: always-pass + barge-in detector + per-tick
// adapter wrapping a "no-`unsafe`" check on the text channel.
let oracle = CompositeStreamOracle::new("composite-stream")
.add(Box::new(AlwaysPassStream))
.add(Box::new(BargeInDetector))
.add(Box::new(PerTickAdapter::new(
"no-unsafe-on-text",
AlwaysPassOracle, // stub — a real Oracle would scan tick bytes
"text".to_string(),
"text.out".to_string(),
)));
let mut ledger = RollingConservationLedger::new(4, 256)
.with_barge_in_deadline_windows(1);
let script = vec![
// 0..5: clean conversation, agent emits short replies.
ScriptedWindow {
id: 0,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: None,
joules_uj: 80,
},
ScriptedWindow {
id: 1,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: None,
joules_uj: 90,
},
ScriptedWindow {
id: 2,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: Some(TickMarker::EndOfTurn),
joules_uj: 95,
},
ScriptedWindow {
id: 3,
inbound_b: 0,
outbound_b: 4800,
text_out: Some("hello, what can I help with?"),
marker: None,
joules_uj: 340,
},
ScriptedWindow {
id: 4,
inbound_b: 0,
outbound_b: 4800,
text_out: None,
marker: None,
joules_uj: 320,
},
ScriptedWindow {
id: 5,
inbound_b: 0,
outbound_b: 4800,
text_out: None,
marker: None,
joules_uj: 320,
},
// 6: barge-in fires.
ScriptedWindow {
id: 6,
inbound_b: 1600,
outbound_b: 4800,
text_out: None,
marker: Some(TickMarker::BargeIn),
joules_uj: 280,
},
// 7: well-behaved cascade yields — outbound drops to zero.
ScriptedWindow {
id: 7,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: None,
joules_uj: 60,
},
// 8..9: user continues, agent listens.
ScriptedWindow {
id: 8,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: None,
joules_uj: 70,
},
ScriptedWindow {
id: 9,
inbound_b: 6400,
outbound_b: 0,
text_out: None,
marker: Some(TickMarker::EndOfTurn),
joules_uj: 75,
},
// 10: agent reply.
ScriptedWindow {
id: 10,
inbound_b: 0,
outbound_b: 4800,
text_out: Some("got it, let me think."),
marker: None,
joules_uj: 340,
},
// 11: HALLUCINATION — outbound 32 KiB with no inbound, no
// substrate. The rolling ledger flags it.
ScriptedWindow {
id: 11,
inbound_b: 0,
outbound_b: 32 * 1024,
text_out: None,
marker: None,
joules_uj: 1200,
},
ScriptedWindow {
id: 12,
inbound_b: 0,
outbound_b: 0,
text_out: None,
marker: Some(TickMarker::End),
joules_uj: 20,
},
];
let mut pass = 0usize;
let mut yield_ = 0usize;
let mut fail = 0usize;
let mut hallucination = 0usize;
let mut total_uj: u64 = 0;
for sw in &script {
let window = build_window(&spec, sw);
let verdict = oracle.verify_window(&window).await.expect("verify");
let finding = ledger.tick(&window, &spec, /* substrate_bytes */ 0);
total_uj += sw.joules_uj;
match &verdict {
WindowVerdict::Pass { .. } => pass += 1,
WindowVerdict::Yield { .. } => yield_ += 1,
WindowVerdict::Fail { .. } => fail += 1,
WindowVerdict::NeedMore { .. } => {}
}
if matches!(finding, WindowConservationFinding::Hallucination { .. }) {
hallucination += 1;
}
// Mint the receipt.
let receipt = StreamWindowReceipt::build(
format!("{}:{:04}", STREAM_ID, sw.id),
format!(
"2026-06-05T00:00:{:02}Z",
CLOSED_AT_BASE.wrapping_add(sw.id) % 60
),
STEWARD,
STREAM_ID,
sw.id,
window.content_hash(),
window.ticks.len() as u64,
window.inbound_bytes(&spec) as u64,
window.outbound_bytes(&spec) as u64,
sw.joules_uj,
EnergyProvenanceWire::Estimator,
verdict_to_wire(&verdict),
conservation_to_wire(&finding),
);
// Sanity-check canonical encoding + content hash.
let canonical = receipt.canonical_bytes().expect("canonical");
let receipt_hash = receipt.content_hash().expect("hash");
let tag = match (&verdict, &finding) {
(WindowVerdict::Yield { .. }, _) => "YIELD",
(WindowVerdict::Fail { .. }, _) => "FAIL ",
(_, WindowConservationFinding::Hallucination { .. }) => "HALLU",
(_, WindowConservationFinding::BargeInIgnored { .. }) => "IGNRD",
_ => "PASS ",
};
println!(
" w{:02} {} ticks={} in={:>5}B out={:>6}B J={:>5}µJ [{}] hash={}",
sw.id,
tag,
window.ticks.len(),
window.inbound_bytes(&spec),
window.outbound_bytes(&spec),
sw.joules_uj,
short(&receipt_hash),
short(&hex_short(&canonical)),
);
}
println!();
println!("== summary ==");
println!(
" windows: {} | pass={} yield={} fail={} hallucination={}",
script.len(),
pass,
yield_,
fail,
hallucination
);
println!(" total joules: {} µJ", total_uj);
println!();
println!("✓ Streaming cascade substrate works end-to-end:");
println!(" StreamSpec → StreamWindow → CompositeStreamOracle (Pass | Yield | Fail | NeedMore)");
println!(" → RollingConservationLedger (Conforming | Hallucination | BargeInIgnored)");
println!(" → JCR-1 stream-window/1 receipt per window");
}
fn short(s: &str) -> String {
if s.len() <= 12 {
s.to_string()
} else {
format!("{}…", &s[..12])
}
}
fn hex_short(bytes: &[u8]) -> String {
let mut s = String::new();
for b in bytes.iter().take(6) {
use std::fmt::Write;
let _ = write!(s, "{b:02x}");
}
s
}