You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are characterizing the Apple Neural Engine's rooflines across the whole Apple Silicon line -- and we need runs from machines we do not have. If you have any Mac with an M-series chip, a short command adds your machine to the map, and you get credited by GitHub handle.
Why this matters
The ANE behaves differently across silicon generations and even across chassis of the same chip (a fanless MacBook Air throttles where a Mac mini does not). Two things we are mapping:
Numeric correctness cliffs -- the magnitudes at which fp16 math on the engine silently returns the wrong answer. These are silicon-dependent, and the collected data already shows it: the slice crop-DMA clamp at |value| > 4094 is present on M1 / M1 Max / M2 Pro but gone by M5 -- the pre-A16 -> A16 transition, now bracketed by real submissions. (Background: matmul saturates at fp16_max/2 (~32752); two transpose-fed cases return inf even below that #115, docs/cross-chip.md.)
How to submit
The tooling is live on main. From a clean checkout, pick your level:
# 1. cliffs only -- fastest, no sudo, works on battery (~2 min):
PYTHONPATH=. python3 bench/roofline_suite.py
# 2. RECOMMENDED -- cliffs + headline perf (peak GEMM, bandwidth, ridge, perf/W, decode).# A few minutes; prompts once for sudo to read watts (required for --perf). Battery + High Power is fine:
PYTHONPATH=. python3 bench/roofline_suite.py --perf
# then regenerate the table and open a PR with both files:
python3 bench/aggregate_rooflines.py
git add bench/results/rooflines/roofline-*.json bench/results/ROOFLINES.md
Open the PR and you are on the board. (--perf-full runs the complete paper-grade battery, ~30 min -- only needed for paper-quality reproduction, not for contributing a datapoint.)
Everything is recorded automatically: chip, model identifier, CPU/GPU cores, unified memory, macOS build, power source + Energy Mode (so a battery/High-Power run is labelled and comparable), and the exact code commit. See bench/results/rooflines/README.md. Chassis matters: the same M1 ANE sustains ~1.7x the GEMM on a 16-inch M1 Max (fans + High Power) as on a fanless 13-inch M1, and the table shows why.
You get credited -- automatically
Your GitHub handle is auto-detected from your git noreply email (GitHub's default) and shown in bench/results/ROOFLINES.md as a By @you column linking to your profile, plus a thanks line. Nothing to type. (If your git email is a generic one, pass --contributor YOUR-HANDLE.) This is a public, living cross-silicon dataset in the repo -- open to everyone regardless of the characterization paper's status.
Chips + GPU-core bins we are looking for
Each distinct chip + GPU-core count is its own datapoint: the fingerprint hashes gpu_cores, so a 14-core and a 16-core M1 Pro are separate rows even though their ANE is identical. Tick the exact config you ran (the GPU-core count prints in the run and lands in your JSON). If your GPU-core bin is not listed, or a count here is off, just comment it -- any binning is welcome. The Ultra/Max parts (Mac Studio / Mac Pro) are the biggest gaps and the highest ANE.
M1
M1 -- 7-core GPU
M1 -- 8-core GPU -- @diegobauavi (MacBook Pro 17,1)
M1 Pro -- 14-core GPU
M1 Pro -- 16-core GPU
M1 Max -- 24-core GPU
M1 Max -- 32-core GPU -- @sbryngelson (MacBook Pro 18,2, AC + High Power)
We are characterizing the Apple Neural Engine's rooflines across the whole Apple Silicon line -- and we need runs from machines we do not have. If you have any Mac with an M-series chip, a short command adds your machine to the map, and you get credited by GitHub handle.
Why this matters
The ANE behaves differently across silicon generations and even across chassis of the same chip (a fanless MacBook Air throttles where a Mac mini does not). Two things we are mapping:
|value| > 4094is present on M1 / M1 Max / M2 Pro but gone by M5 -- the pre-A16 -> A16 transition, now bracketed by real submissions. (Background: matmul saturates at fp16_max/2 (~32752); two transpose-fed cases return inf even below that #115,docs/cross-chip.md.)How to submit
The tooling is live on
main. From a clean checkout, pick your level:Open the PR and you are on the board. (
--perf-fullruns the complete paper-grade battery, ~30 min -- only needed for paper-quality reproduction, not for contributing a datapoint.)Everything is recorded automatically: chip, model identifier, CPU/GPU cores, unified memory, macOS build, power source + Energy Mode (so a battery/High-Power run is labelled and comparable), and the exact code commit. See
bench/results/rooflines/README.md. Chassis matters: the same M1 ANE sustains ~1.7x the GEMM on a 16-inch M1 Max (fans + High Power) as on a fanless 13-inch M1, and the table shows why.You get credited -- automatically
Your GitHub handle is auto-detected from your git noreply email (GitHub's default) and shown in
bench/results/ROOFLINES.mdas aBy @youcolumn linking to your profile, plus a thanks line. Nothing to type. (If your git email is a generic one, pass--contributor YOUR-HANDLE.) This is a public, living cross-silicon dataset in the repo -- open to everyone regardless of the characterization paper's status.Chips + GPU-core bins we are looking for
Each distinct chip + GPU-core count is its own datapoint: the fingerprint hashes
gpu_cores, so a 14-core and a 16-core M1 Pro are separate rows even though their ANE is identical. Tick the exact config you ran (the GPU-core count prints in the run and lands in your JSON). If your GPU-core bin is not listed, or a count here is off, just comment it -- any binning is welcome. The Ultra/Max parts (Mac Studio / Mac Pro) are the biggest gaps and the highest ANE.M1
M2
M3
M4
M5
Questions welcome in the comments. Thanks for helping build the map!