Skip to content

Read parallelism and config_summary from node_types - #146

Open
arav-agarwal2 wants to merge 1 commit into
mainfrom
feat/parallelism-per-node-type
Open

arav-agarwal2 wants to merge 1 commit into
mainfrom
feat/parallelism-per-node-type

Conversation

@arav-agarwal2

Copy link
Copy Markdown
Contributor

Companion to mlcommons/endpoints_policies#160, which moves disaggregated,
expert_parallel, tensor_parallel, pipeline_parallel, data_parallel and
config_summary out of the top level of system_desc.json and into each
node_types entry.

_point_parallelism read only the top level, with the structured
config_summary object as a fallback. It now reads the node-type placement
first — that being the current format — and falls back to the old one, so a
bundle in either shape works and one part-way through the move resolves field by
field. Within a node type, its own config_summary is the fallback, exactly as
the SUT-wide one was.

Two consequences that needed handling, not just the lookup

1. The cross-point identity check would have broken.
_system_desc_identity strips the fields that legitimately vary between points
of a frontier — and it strips them by top-level key:

return {k: v for k, v in data.items() if k not in _PER_POINT_SYSTEM_DESC_FIELDS}

With those fields nested, that no longer reaches them. Any frontier that tunes
its replica size per point — which §4.5.3's own C.3 worked example does — would
then differ on node_types and fail system-description-consistency with
differing: node_types. They are now stripped at both depths, so a bundle in
either placement compares on the same identity. Without this the format move
would have failed real submissions
, which is why it's in this PR rather than a
follow-up.

2. Node types that disagree yield None rather than a pick. §4.5.3 computes
one A_replica = TP·PP·EP for the whole SUT, so a Parallelism cannot carry two
values for a field. Normalising against a replica size no node type actually has
would be worse than skipping nodes-used — which is already what happens for a
bundle that declares no parallelism at all. Making §4.5.3's arithmetic per node
type is the real fix and is its own piece of work; point_power.py already
documents the gap it leaves ("for a heterogeneous system the split across sets is
not knowable").

Also

  • NodeType declares the six fields. ConfigSummary and _ConfigSummaryStr
    move above it so a node type's config_summary takes the same type as the
    SUT-wide one — a reorder, no change to either definition.
  • disaggregated is true if any node type is disaggregated.

Verification

1805 tests pass, 15 new. The new ones cover: the node-type placement read;
top-level still read; config_summary still the last fallback; node-types
winning over the top level; the two placements mixing per field; a node type's
own field beating its own config_summary; agreeing node types resolving;
disagreeing ones declining; any disaggregated node type disaggregating the SUT;
declaring nothing still None; and four on identity — nested parallelism and
nested config_summary excluded from identity, real hardware differences still
counted, and a malformed node_types passed through rather than raising.

🤖 Generated with Claude Code

§8.2.1 declares disaggregated, expert_parallel, tensor_parallel,
pipeline_parallel, data_parallel and config_summary inside each
node_types entry rather than at the top level of system_desc.json
(policies PR #160), because §4.5.3's node-count declaration is per node
type and has to be consistent with them.

The checker read only the top level, with the structured config_summary
object as a fallback. It now reads the node-type placement first, since
that is the current format, and falls back to the old one — so bundles
in either shape keep working, and one part-way through the move
resolves field by field.

Two things this had to get right beyond the lookup:

_system_desc_identity strips the fields that legitimately vary between
points of a frontier, by top-level key. With those fields nested, that
no longer reaches them, and any frontier that tunes its replica size per
point — which §4.5.3's C.3 example does — would differ on node_types and
fail system-description-consistency. They are stripped at both depths.

Node types that disagree on a field yield None rather than a pick. §4.5.3
computes one A_replica = TP·PP·EP for the whole SUT, so a Parallelism
cannot carry two values; normalising against a replica size no node type
has would be worse than skipping nodes-used, which is what already
happens for a bundle declaring no parallelism. Making that arithmetic
per node type is the real fix and is its own job.

NodeType declares the six fields, and ConfigSummary moves above it so a
node type's config_summary takes the same type as the SUT-wide one did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant