Skip to content

Speed up NNX graph flattening by avoiding unnecessary array-ref checks#5523

Open
KookiesNKareem wants to merge 1 commit into
google:mainfrom
KookiesNKareem:speed-up-nnx-array-ref-checks
Open

Speed up NNX graph flattening by avoiding unnecessary array-ref checks#5523
KookiesNKareem wants to merge 1 commit into
google:mainfrom
KookiesNKareem:speed-up-nnx-array-ref-checks

Conversation

@KookiesNKareem

@KookiesNKareem KookiesNKareem commented Jul 6, 2026

Copy link
Copy Markdown

This PR reduces Python overhead in NNX graph flattening by avoiding unnecessary is_array_ref checks in common non-ref paths.

Specifically:

  • _graph_flatten skips variablelib.is_array_ref(node) when the node is already known to be a graph node, pytree node, or Variable.
  • variablelib.is_array_ref fast-paths runtime Ref objects and concrete non-ref arrays, while preserving the existing jax.typeof fallback for AbstractRef and traced array cases.

This keeps mutable-ref behavior unchanged while reducing repeated calls into JAX abstract-value logic on the hot path.

Perf on a warmed no-op nnx.jit graph-overhead workload under jax==0.10.0:

  • Baseline median: 1089.38 us/call
  • Patched median: 936.28 us/call
  • Improvement: ~14.1%

Checklist

  • This PR fixes a minor issue (e.g.: typo or small bug) or improves the docs (you can dismiss the other checks if that's the case).
  • This change is discussed in a Github issue/discussion (please add a link).
  • The documentation and docstrings adhere to the documentation guidelines.
  • This change includes necessary high-coverage tests. (No quality testing = no merge!)

Validation:

  • tests/nnx under jax==0.10.0: 1686 passed, 10 skipped
  • tests/nnx under jax==0.10.2: 1686 passed, 10 skipped
  • Focused mutable-ref/variable/graph utility suite against the applied checkout under jax==0.10.0: 239 passed, 9 skipped
  • Verified is_array_ref(AbstractRef) preserves previous behavior.

@KookiesNKareem KookiesNKareem marked this pull request as ready for review July 6, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant