benchmark and profiling suite

[majunze2001]

No description provided.

[gemini-code-assist]

Code Review

This pull request overhauls the Cornserve benchmarking infrastructure, introducing a unified framework for profiling Eric, MLLM, and Geri applications. Key features include a GPU-aware batch scheduler for synchronized experiment execution, a robust dataset sampling system with persistence, and integrated performance analysis using OpenTelemetry traces and Prometheus metrics. The changes also address trace leakage in streaming responses and improve resource management within the gateway. Feedback from the review highlighted a logic error in SSE stream handling for the talker service, a security risk regarding the use of eval() on external data, and a redundant parameter assignment in the image task generation logic.

[gemini-code-assist]

The invoke_talker_vocoder function appears to be missing logic to handle the data: prefix and the [DONE] sentinel common in SSE streams, which is handled correctly in invoke_mllm. If the talker service returns SSE, json.loads will fail on the raw line.

Suggested change

	buffer += chunk
	while b"\n" in buffer:
	line_bytes, buffer = buffer.split(b"\n", 1)
	line = line_bytes.decode("utf-8").strip()
	if not line:
	continue
	if line.startswith("data: "):
	line = line[6:]
	if not line:
	continue
	if line == "[DONE]":
	break
	try:
	data = json.loads(line)

[gemini-code-assist]

Using eval() on data retrieved from external sources (even Jaeger tags) is a security risk. Since the expected format is a simple numeric list, json.loads() is a safer alternative.

Suggested change

	return json.loads(shape_str)

[gemini-code-assist]

The encoder_tp_size parameter is redundantly assigned twice in the generated MixedQwenImageTask definition. One of these lines should be removed.

Suggested change

	f' encoder_tp_size={inner.encoder_tp_size},',
	f' encoder_tp_size={inner.encoder_tp_size},',
	f' encoder_tp_size={inner.encoder_tp_size},',