c4deb866d2
`sources::PHASE27_OPS` was referenced by eight architecture files and had
never been written, so 23 pipelines named kernels that did not exist. The
remaining 22 errors were `gpu_ops::*_on` helpers in the same state.
Nothing here was invented: every kernel's buffer slots, grid shape and
scalar order is fixed by an existing dispatch site, and every helper's
signature by its call site. The Mochi VAE stages dispatch directly, so
their layouts came from mochi.rs; the rest came from the `*_on` calls in
sam2, mochi, bark, sesame_csm, moonshine and chatterbox.
21 kernels in PHASE27_OPS:
patch/window im2col_patch_embed, window_partition, window_unpartition,
max_pool2d, patchify_3d, unpatchify_3d
attention split_qkv, kv_head_expand, rope_batched, flash_attention
activations elu, euler_step, modulation_split
codecs dequantize_rvq, dilated_conv1d
video VAE vae_conv1x1, vae_spatial_upsample_2x, vae_channel_reduce,
vae_temporal_upsample, vae_final_conv_sigmoid,
vae_extract_frame
flash_attention stages K/V tiles in the threadgroup memory the caller
already allocates and folds them in with an online softmax, so no S x S
score matrix is materialised. It is non-causal, which is correct for the
two diffusion transformers that dispatch it. The partition/patchify pairs
are exact inverses of each other.
13 helpers in gpu_ops.rs, each allocating its own shared output buffer as
the existing helpers in that module do.
The offline shader check added in the previous commit earned itself
immediately: flash_attention_f16 declared `uint2
threadgroup_position_in_grid` beside `uint
thread_position_in_threadgroup`, and MSL requires every position
attribute in a kernel to have the same component count. That is a
runtime pipeline-build failure, invisible to cargo, and no reviewer or
type checker would have caught it.
State: `--features metal` compiles with 0 errors and 0 clippy errors
across the ~19k lines this makes reachable; all 44 shader sources
compile; the 46 default tests still pass.
Still not correctness. `cargo test --features metal` runs the same 46
tests as the default build and both integration tests are ignored, so
not one of these kernels has ever been executed. The macOS runner has a
GPU, so that is now reachable and is the next thing worth doing.
Co-Authored-By: Claude Opus 5 <[email protected]>
110 lines
4.9 KiB
Python
Executable file
110 lines
4.9 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
"""Per-fabric compile guard for jouleclaw-omni: each count may fall, never rise.
|
|
|
|
check-fabric-errors.py <feature>
|
|
|
|
`metal` did not compile at all -- 3063 errors -- because no job ever built it.
|
|
40 architecture files carried `#[cfg(feature = "metal")]` blocks that had never
|
|
once been compiled; 2780 of those errors were imports the files never made, and
|
|
316 of those were `Result<T>` resolving to `core::result::Result<T, E>` for want
|
|
of the crate's one-parameter alias. That is the shape every unbuilt feature
|
|
drifts into, so every fabric now gets a job on hardware that has the device:
|
|
metal on macOS, cuda and vulkan on the linux host with an RTX 4050 and a
|
|
Vulkan loader.
|
|
|
|
A baseline above zero is a debt, not a pass. Lower it whenever the count falls;
|
|
the guard refuses to let it rise. Zero means the fabric compiles -- which is
|
|
still not the same as working, and wants a test that runs a kernel.
|
|
|
|
`rocm` is deliberately absent: `rocm = []` in Cargo.toml declares a feature
|
|
with no dependencies and no code, so it compiles perfectly while providing
|
|
nothing. An error count cannot see that, and a guard here would report health
|
|
it does not have.
|
|
"""
|
|
import json, subprocess, sys
|
|
|
|
BASELINES = {
|
|
"metal": 0, # 3063 -> 286 -> 111 -> 65 -> 45 -> 0; compiles 2026-10-20
|
|
"cuda": 0, # compiles on macOS; this is its first build on an NVIDIA host
|
|
"vulkan": 0, # likewise, first build against a real Vulkan loader
|
|
}
|
|
MANIFEST = "jouleclaw/jouleclaw-rs/Cargo.toml"
|
|
|
|
def main(argv: list[str]) -> int:
|
|
if len(argv) != 2 or argv[1] not in BASELINES:
|
|
print(f"usage: {argv[0]} <{'|'.join(BASELINES)}>")
|
|
return 2
|
|
feature = argv[1]
|
|
baseline = BASELINES[feature]
|
|
|
|
proc = subprocess.run(
|
|
["cargo", "check", "--manifest-path", MANIFEST, "-p", "jouleclaw-omni",
|
|
"--features", feature, "--keep-going", "--message-format=json"],
|
|
capture_output=True, text=True,
|
|
)
|
|
# cargo exits 0 when a target is skipped, so no output is not a pass.
|
|
if not proc.stdout.strip():
|
|
print(f"::error::cargo emitted no JSON for `{feature}` -- the check did not run")
|
|
print(proc.stderr[-2000:])
|
|
return 1
|
|
|
|
errors, codes, first_messages = 0, {}, []
|
|
for line in proc.stdout.splitlines():
|
|
try:
|
|
msg = json.loads(line)
|
|
except ValueError:
|
|
continue
|
|
if msg.get("reason") != "compiler-message":
|
|
continue
|
|
m = msg.get("message") or {}
|
|
if m.get("level") != "error":
|
|
continue
|
|
errors += 1
|
|
code = (m.get("code") or {}).get("code") or "(none)"
|
|
codes[code] = codes.get(code, 0) + 1
|
|
if len(first_messages) < 5:
|
|
sp = (m.get("spans") or [{}])[0]
|
|
where = f"{sp.get('file_name','?').split('/')[-1]}:{sp.get('line_start','?')}"
|
|
first_messages.append(f"{code} {where}: {(m.get('message') or '')[:90]}")
|
|
|
|
top = ", ".join(f"{c}={n}" for c, n in sorted(codes.items(), key=lambda x: -x[1])[:6])
|
|
print(f"jouleclaw-omni --features {feature}: {errors} errors (baseline {baseline})")
|
|
if codes:
|
|
print(f" by code: {top}")
|
|
|
|
# A clean error tally is not a build. cargo fails for reasons that never
|
|
# reach the compiler-message channel -- a build script that panics is the
|
|
# common one: `cudarc` aborts unless a cuda-version feature is chosen, and
|
|
# that produced 0 errors and a green check until this check existed.
|
|
#
|
|
# Only a *silent* failure is suspicious. With a baseline above zero cargo
|
|
# exits 101 because of those very errors, so gating on `errors <= baseline`
|
|
# fired on every healthy run of a fabric still in debt. Gate on a zero
|
|
# tally instead: that is the case where the exit status is the only witness.
|
|
if proc.returncode != 0 and errors == 0:
|
|
print(f"::error::`{feature}`: cargo exited {proc.returncode} while the error "
|
|
f"tally ({errors}) was within the baseline ({baseline}). Something failed "
|
|
f"outside the compiler: a build script, a missing system library, a "
|
|
f"feature the dependency requires. Counting errors cannot see it.")
|
|
print("--- cargo stderr, last 2000 chars ---")
|
|
print(proc.stderr[-2000:])
|
|
return 1
|
|
|
|
# Show what the errors are, not just how many: a count alone is not
|
|
# actionable from a CI log.
|
|
if errors:
|
|
for m in first_messages[:5]:
|
|
print(f" {m}")
|
|
|
|
if errors > baseline:
|
|
print(f"::error::`{feature}`: {errors - baseline} more errors than the baseline "
|
|
f"of {baseline}. Do not add to a fabric that does not compile.")
|
|
return 1
|
|
if errors < baseline:
|
|
print(f"::notice::`{feature}`: {baseline - errors} fewer than the baseline. "
|
|
f"Set BASELINES['{feature}'] = {errors} in scripts/check-fabric-errors.py "
|
|
f"to hold the gain.")
|
|
return 0
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main(sys.argv))
|