Skip to content

Declare parallelism and config_summary per node type - #160

Open
arav-agarwal2 wants to merge 1 commit into
mainfrom
feat/parallelism-per-node-type
Open

arav-agarwal2 wants to merge 1 commit into
mainfrom
feat/parallelism-per-node-type

Conversation

@arav-agarwal2

@arav-agarwal2 arav-agarwal2 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

§8.2.1's template puts disaggregated, expert_parallel, tensor_parallel,
pipeline_parallel, data_parallel and config_summary at the top level of
system_desc.json, so a SUT declares one partitioning for every node type it
has. This moves them inside each node_types entry.

§4.5.3 already assumes they are per node type

The provisioned node-count declaration is explicit about it:

declared at each point in the point's measurement YAML (§8.3), by node
type
: a single count for a homogeneous system, one count per node type for a
heterogeneous one.

and then requires:

The declared node count MUST be consistent with the parallelism configuration
declared in system_desc.json (§8.2) — tensor_parallel,
pipeline_parallel, expert_parallel, and data_parallel

A heterogeneous SUT that partitions differently on each node type cannot satisfy
both rules as written: one SUT-wide tensor_parallel has no way to express two
different values, so the consistency requirement is unsatisfiable rather than
merely unchecked. Moving the fields is what makes §4.5.3 expressible.

config_summary moves with them, being the concatenation of the five.
config_summary_notes stays SUT-wide and remains an input to each node type's
config_summary; its table row says so.

Compatibility

A homogeneous SUT has exactly one node_types entry, so it declares the six
fields once — same count, one nesting level deeper. That is every submission in
the corpus today, so nothing in flight becomes unexpressible; bundles do need
re-nesting, which the checker handles — see the companion PR
mlcommons/endpoints-submission-cli#146, which reads both placements.

Verification

  • Extracted the §8.2.1 template and parsed it: valid JSON, the six keys appear
    only inside node_types[0], and the remaining top-level keys after
    node_types are node_config, batch, config_summary_notes,
    tps_utilization — batch deliberately not moved.
  • pre-commit run --files endpoints_rules.md passes (markdownlint MD013).
  • Net diff is 8 lines changed either way: the six moved keys, the trailing-comma
    adjustments around them, and the config_summary row.

🤖 Generated with Claude Code

@arav-agarwal2
arav-agarwal2 requested a review from a team October 29, 2026 14:29
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

Comment thread endpoints_rules.md Outdated
| `config_summary_notes` | Free form field from the submitter to contain information not captured by other fields that concatenate into config_summary. |
| `tps_utilization` | reported_system_tps / (max of all reported_system_tps for all runs) |

> [!NOTE]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating this as we speak - I'll move config_summary inside.

Comment thread endpoints_rules.md Outdated

The template omits the optional `c_min_override`; its declaration rules are in §5.4.

`disaggregated`, `expert_parallel`, `tensor_parallel`, `pipeline_parallel` and

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to remove this section as it's just adding words to the rules without I think a good reason to.

§8.2.1's template put disaggregated, expert_parallel, tensor_parallel,
pipeline_parallel, data_parallel and config_summary at the top level of
system_desc.json, so a SUT declared one partitioning for every node type
it has.

§4.5.3 already contradicts that. The provisioned node-count declaration
there is explicitly per node type — "a single count for a homogeneous
system, one count per node type for a heterogeneous one" — and it "MUST
be consistent with the parallelism configuration declared in
system_desc.json (§8.2) — tensor_parallel, pipeline_parallel,
expert_parallel, and data_parallel". A heterogeneous SUT that partitions
differently on each node type cannot satisfy both: one SUT-wide TP has
no way to express two.

So the six fields move inside each node_types entry. config_summary goes
with them, being the concatenation of the five. A homogeneous SUT has
one entry and declares them once, exactly as before, which is every
submission in the corpus today.

config_summary_notes stays SUT-wide and is still an input to each node
type's config_summary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@arav-agarwal2
arav-agarwal2 force-pushed the feat/parallelism-per-node-type branch from 8479e4d to 14b0225 Compare October 29, 2026 14:50
@arav-agarwal2 arav-agarwal2 changed the title Declare parallelism per node type, not once per SUT Declare parallelism and config_summary per node type Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant