fix(validation): stop capping maxTokens below what models actually serve - #15
Merged
Conversation
Same defect as the Python SDK, same hardcoded 100000. `validateMaxTokens` rejected anything above it client-side, so the SDK — not the model, not the gateway — was the binding constraint on every caller. Verified against the live gateway 2026-07-21 by bypassing the guard: 19 models advertise a ceiling above 100000 and all 19 accepted it, including zai/glm-5.2 at 262144 and the entire 128000 class (claude-opus-4.8, claude-sonnet-5, claude-fable-5, gpt-5.6-sol/terra/luna, gpt-5.5, gpt-5.4, gpt-5.3-codex, glm-5/5.1/5-turbo). Zero rejections. The old message, "maxTokens too large (maximum: 100000)", reads like a provider response. It was taken for one during an investigation and recorded as an upstream model ceiling in a downstream token table — 19 identical "rejections" that never left the process. MAX_TOKENS_SANITY_LIMIT is now 1_000_000 and exported, documented as a typo guard, with a message stating the number is the SDK's and the gateway owns the real limit. The bound stays because a stray 1e9 or a byte count should fail locally rather than become a payment quote; it just must never be reachable by a real request. Adds test/unit/validation-max-tokens.test.ts — the guard had no coverage at all. Pins the real ceilings (128000, 262144), the typo case, that the bound sits above any plausible model, and that the message does not describe itself as a model limit. Suite 216 pass, typecheck clean.
Three gaps in the original commit, caught in review: - MAX_TOKENS_SANITY_LIMIT was exported from validation.ts but not re-exported from src/index.ts, so package consumers could not actually read it. - Nothing tested the range between the largest real model and the typo case. Lowering the constant back toward 262_144 would not have failed a single test, which is precisely the regression this file exists to catch. - The comment presented the probe results as if they were a maintained invariant. They are an as-of-date snapshot: this repo ships no model catalog, so CI cannot detect when those numbers go stale. Says so now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Same defect as the Python SDK, same hardcoded
100000. Companion to that PR.The defect
validateMaxTokensrejected anything above 100,000 client-side, so the SDK — not the model, not the gateway — was the binding constraint on every caller. Askingzai/glm-5.2for the 262,144 it advertises threw instead of sending.Verification
Probed the live gateway 2026-07-21 with the guard bypassed. All 19 models advertising a ceiling above 100,000 accepted it:
zai/glm-5.2at 262,144, and the entire 128,000 class (claude-opus-4.8,claude-sonnet-5,claude-fable-5,gpt-5.6-sol/terra/luna,gpt-5.5,gpt-5.4,gpt-5.3-codex,glm-5/5.1/5-turbo). Zero rejections.The message made it worse
"maxTokens too large (maximum: 100000)"reads like a provider response. It was taken for one during an investigation and recorded as an upstream model ceiling in a downstream token table — 19 identical "rejections" that never left the process.The fix
MAX_TOKENS_SANITY_LIMIT = 1_000_000, now exported, documented as a typo guard, with a message stating the number is the SDK's and the gateway owns the real limit. The bound stays so a stray1e9fails locally rather than becoming a payment quote; it just must never be reachable by a real request.Tests
The guard had no coverage at all. Adds
test/unit/validation-max-tokens.test.tspinning: the real ceilings (128,000 / 262,144), the typo case, that the bound sits above any plausible model, that non-integer and non-positive still reject, and — deliberately — that the message does not describe itself as a model limit, since that phrasing is what caused the downstream misreading.216 pass, typecheck clean.