Repository navigation
Read parallelism and config_summary from node_types - #146
Open
arav-agarwal2 wants to merge 1 commit into
Open
arav-agarwal2 wants to merge 1 commit into
arav-agarwal2 wants to merge 1 commit into
Conversation
§8.2.1 declares disaggregated, expert_parallel, tensor_parallel, pipeline_parallel, data_parallel and config_summary inside each node_types entry rather than at the top level of system_desc.json (policies PR #160), because §4.5.3's node-count declaration is per node type and has to be consistent with them. The checker read only the top level, with the structured config_summary object as a fallback. It now reads the node-type placement first, since that is the current format, and falls back to the old one — so bundles in either shape keep working, and one part-way through the move resolves field by field. Two things this had to get right beyond the lookup: _system_desc_identity strips the fields that legitimately vary between points of a frontier, by top-level key. With those fields nested, that no longer reaches them, and any frontier that tunes its replica size per point — which §4.5.3's C.3 example does — would differ on node_types and fail system-description-consistency. They are stripped at both depths. Node types that disagree on a field yield None rather than a pick. §4.5.3 computes one A_replica = TP·PP·EP for the whole SUT, so a Parallelism cannot carry two values; normalising against a replica size no node type has would be worse than skipping nodes-used, which is what already happens for a bundle declaring no parallelism. Making that arithmetic per node type is the real fix and is its own job. NodeType declares the six fields, and ConfigSummary moves above it so a node type's config_summary takes the same type as the SUT-wide one did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to mlcommons/endpoints_policies#160, which moves
disaggregated,expert_parallel,tensor_parallel,pipeline_parallel,data_parallelandconfig_summaryout of the top level ofsystem_desc.jsonand into eachnode_typesentry._point_parallelismread only the top level, with the structuredconfig_summaryobject as a fallback. It now reads the node-type placementfirst — that being the current format — and falls back to the old one, so a
bundle in either shape works and one part-way through the move resolves field by
field. Within a node type, its own
config_summaryis the fallback, exactly asthe SUT-wide one was.
Two consequences that needed handling, not just the lookup
1. The cross-point identity check would have broken.
_system_desc_identitystrips the fields that legitimately vary between pointsof a frontier — and it strips them by top-level key:
With those fields nested, that no longer reaches them. Any frontier that tunes
its replica size per point — which §4.5.3's own C.3 worked example does — would
then differ on
node_typesand failsystem-description-consistencywithdiffering: node_types. They are now stripped at both depths, so a bundle ineither placement compares on the same identity. Without this the format move
would have failed real submissions, which is why it's in this PR rather than a
follow-up.
2. Node types that disagree yield
Nonerather than a pick. §4.5.3 computesone
A_replica = TP·PP·EPfor the whole SUT, so aParallelismcannot carry twovalues for a field. Normalising against a replica size no node type actually has
would be worse than skipping
nodes-used— which is already what happens for abundle that declares no parallelism at all. Making §4.5.3's arithmetic per node
type is the real fix and is its own piece of work;
point_power.pyalreadydocuments the gap it leaves ("for a heterogeneous system the split across sets is
not knowable").
Also
NodeTypedeclares the six fields.ConfigSummaryand_ConfigSummaryStrmove above it so a node type's
config_summarytakes the same type as theSUT-wide one — a reorder, no change to either definition.
disaggregatedis true if any node type is disaggregated.Verification
1805 tests pass, 15 new. The new ones cover: the node-type placement read;
top-level still read;
config_summarystill the last fallback; node-typeswinning over the top level; the two placements mixing per field; a node type's
own field beating its own
config_summary; agreeing node types resolving;disagreeing ones declining; any disaggregated node type disaggregating the SUT;
declaring nothing still
None; and four on identity — nested parallelism andnested
config_summaryexcluded from identity, real hardware differences stillcounted, and a malformed
node_typespassed through rather than raising.🤖 Generated with Claude Code