Handle K2 reasoning boundaries and streamed tool calls

[hanseungwook]

Summary

• emit string-valued reasoning and reasoning_content at K2 reasoning boundaries, including "" for an empty reasoning segment
• normalize the empty boundary consistently in Chat Completions and the Responses API
• define safe non-streaming behavior when K2 omits the reasoning end token
• add fallback streaming tool-call parsing from Stream content and IFM tool calls in multi-format parser #10, preserving the original commits and authorship
• hand post-reasoning text to the tool parser even when the reasoning boundary and final tool call arrive in the same engine delta

K2 behavior

Non-streaming reasoning

• with a reasoning end token: everything before the first end token is reasoning; everything after it is content and may be parsed for tools
• without an end token, with <ifm|tool_calls>: the prefix is reasoning and the wrapped section is handed to the tool parser
• without an end token or <ifm|tool_calls>: the entire output is content, avoiding an empty visible response
• absent reasoning or content is represented as "", not null

Tool-like markup before an explicit reasoning end token remains reasoning and is never executed.

Streaming tool calls

• supports reasoning-end and tool-call data arriving in the same delta or separate deltas
• preserves tool-call IDs, names, arguments, multiple calls, and the final tool_calls finish reason
• K2 treats only the plural <ifm|tool_calls> wrapper as an IFM tool boundary; a standalone <ifm|tool_call> remains ordinary content
• generic multi_format retains Stream content and IFM tool calls in multi-format parser #10's broader compatibility behavior

The missing-reasoning-end fallback remains intentionally scoped to non-streaming parsing in this PR.

Testing

• tests/reasoning/test_k2_v3_reasoning_parser.py: 105 passed
• tests/entrypoints/openai/tool_parsers/test_multi_format_tool_parser.py: 42 passed
• tests/entrypoints/openai/test_serving_responses.py: 10 passed
• tests/entrypoints/openai/test_tool_parser_kwargs.py: 3 passed
• K2 Chat same-delta and split-delta integration cases: 2 passed
• Ruff and git diff --check: passed

The full Chat test file also ran all 9 CPU-only tests successfully; 6 GPT-OSS server tests could not initialize because the test environment has no detectable accelerator device.

[aaryamonvikram]

LGTM!

[aaryamonvikram]

Non-blocking note: K2V3ReasoningParser is also used by the Responses API. That path checks an empty reasoning value inconsistently: it first uses if delta_message.reasoning, which treats reasoning="" as absent, but later uses reasoning is not None, which treats it as present. This can cause the same streamed item to be classified first as a message and then as reasoning.

This does not affect Chat Completions and should not block this PR. It may be worth handling empty reasoning consistently in the Responses path in a future follow-up.

[hanseungwook]

Responses API follow-up note:

This PR keeps message.output_text.logprobs aligned only with tokens in each emitted response.output_text.delta. K2 reasoning and control tokens are partitioned away rather than being mislabeled as output-text logprobs. When an exact reasoning/content metadata partition is unavailable, the adapter omits those delta logprobs instead of returning a mismatched sequence.

What remains unsupported is a lossless public channel for every generated token:

• response.reasoning_text.delta has no standard logprobs field.
• Stripped reasoning delimiters and other control tokens have no semantic event.
• The simple Responses streaming path still does not emit tool-call items, so tool-source token metadata also has no compatible destination.

If vLLM wants to expose all sampled token IDs and logprobs, that should be a separate opt-in extension, such as a raw-token metadata event or explicit metadata fields on reasoning/tool events. Until then, usage counts all generated tokens, while Responses logprobs intentionally cover output text only.

[aaryamonvikram]

Two findings:

Blocking — forced tool calls can be lost when reasoning and the tool call end in the same engine delta.

In the named and tool_choice="required" paths, a final delta such as:

</ifm|think>[{"name":"get_weather","parameters":{"city":"Tokyo"}}]

is separated into reasoning="" and the remaining tool-call JSON, but that remaining content is not passed to the forced-tool parser during the same iteration. Because no subsequent delta arrives, the response contains no tool calls and finishes with "stop". Splitting the identical output into two deltas succeeds, so the result depends on engine chunking.

Non-blocking — Responses API truncation behavior.

We intentionally buffer output as reasoning until the closing-think token arrives. If the Responses API reaches finish_reason="length" before that token, the current no-close fallback returns:

reasoning=""
content=<buffered reasoning>

This means incomplete reasoning can be returned as public assistant content. This is specific to the Responses API path and should not block this PR.

[aaryamonvikram]

Everything looks good. For posterity, I’m leaving a couple of non-blocking comments about the Responses API behavior:

1. content_index is currently incremented for each streamed output-text delta. This index identifies the content part, rather than the delta number, so all delta and done events for a single text part should use the same index (normally 0). The current sequence can look like delta=0, delta=1, and done=2, which may confuse clients assembling the response.

2. When previous_response_id is used, the prior ResponseOutputMessage is reconstructed, but its ResponseReasoningItem is not. For K2, this can replay an assistant message without the required reasoning field—including the valid empty value reasoning_content=""—and cause the following turn to fail chat-template validation.

Both observations are specific to the Responses API and do not affect Chat Completions.

[hanseungwook]

Addressed in f178d9379.

1. Same-delta handoff for forced tools

Previously, the named and tool_choice="required" branches stopped after extracting reasoning. If the reasoning boundary and tool JSON arrived in the final engine delta, the code waited for another delta that never arrived and the tool call was lost.

Now, when reasoning ends, the remainder is passed to forced-tool handling immediately in the same iteration and the resulting tool-call delta is merged with the reasoning delta.

2. Correct delimiter accounting for complete required JSON

A complete required-tool array received in one chunk could produce {"city":"Tokyo"}] instead of {"city":"Tokyo"}.

The nesting calculation now uses the JSON prefix before parameters, so only valid function arguments are emitted.