bench: fix layer_norm probe -- affine/seed axes + corrected docstring (#149) - #159
Merged
sbryngelson merged 2 commits intoAug 2, 2026
Merged
Conversation
…rected docstring (sbryngelson#149) The committed probe hardcoded seed=13 with both affine terms non-constant and its docstring asserted a 'scattered non-monotonic frontier' as fact. That frontier is an artifact of that one affine: the failure depends on the gamma/beta values, not the shape. Verified across M1 Max / M2 Pro / M5 Pro, the affine 2x2 separates two distinct family-specific bugs -- M1/M2 compile-fail when exactly one affine term is non-constant (fold asymmetry); M5 execute-fails with both terms live at large D. Changes: add --seed, --gamma/--beta, and an --affine-scan 2x2 mode; report the compile-vs-dispatch stage (C-FAIL/D-FAIL); rewrite the docstring to describe the verified two-bug finding and flag that any single scan is one affine's signature.
…alse C-FAIL)
The rms column passed beta='-', not a valid --beta choice, so the worker subprocess
errored in argparse and every rms cell read C-FAIL instead of OK. rms_norm ignores
beta; use a valid placeholder ('zeros').
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
layer_norm_compile_probe.pymerged in #158 hardcodedseed=13with both affine terms non-constant, and its docstring asserted a scattered non-monotonic frontier as established fact. That frontier is an artifact of that one affine -- the failure depends on the gamma/beta values, not the shape. Since the probe is onmain, that misleading docstring/behavior is worth correcting now.Verified across M1 Max / M2 Pro / M5 Pro, the affine 2x2 (
--affine-scan) separates two distinct, family-specific bugs:M1/M2 and M5 are exact opposites on the asymmetric configs. rms_norm/softmax never fail (single/zero affine slots).
Changes:
--seed,--gamma/--beta, and an--affine-scan2x2xD mode (the diagnostic that reveals the split);C-FAIL/D-FAIL);Smoke-tested on M5:
--one 512 10240(default rr) -> D-FAIL; same cell with--gamma rand --beta zeros-> OK (asymmetric passes on M5, as verified). ruff clean.