Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
d8141c1
Add stream to request options
thelovekesh Jun 18, 2026
f00f0c8
Add stream helpers to Response
thelovekesh Jun 18, 2026
64844fc
Add streaming support to http transport
thelovekesh Jun 18, 2026
3f75843
Update phpcs config to lint 20 files in parallel
thelovekesh Jun 18, 2026
998d695
Add interface for sse parser
thelovekesh Jun 19, 2026
afa0ba0
Add DTO for SSE
thelovekesh Jun 19, 2026
8751105
Add SSE spec compliant parser
thelovekesh Jun 19, 2026
31c2170
Update stream id type to string
thelovekesh Jun 19, 2026
8bb52fc
Remove dispatching event if file end is reached
thelovekesh Jun 19, 2026
4ff589d
Add mock for chunking stream
thelovekesh Jun 19, 2026
cf15f0b
Add test cases for SseEventStreamParser
thelovekesh Jun 19, 2026
cbb11c2
Fix aliases order
thelovekesh Jun 19, 2026
f67deb4
Remove dead code after adding spec driven stream discard if it doesn'…
thelovekesh Jun 19, 2026
127ddad
Add test cases for unexpected body endings
thelovekesh Jun 19, 2026
2fc0ab9
Add link to PHP_SOCK_CHUNK_SIZE default size
thelovekesh Jun 19, 2026
a42a4c1
Update `@see` usage
thelovekesh Jun 19, 2026
9781d4e
Add test case for streams having no terminating end line
thelovekesh Jun 19, 2026
ecd6235
Add interface for text generation models with streaming support
thelovekesh Jun 19, 2026
5e3c0d9
Add immutable class to hold generative ai result chunk
thelovekesh Jun 19, 2026
bf803a7
Add streamed result aggregator
thelovekesh Jun 19, 2026
90a9cb3
Add text streaming support in prompt builder
thelovekesh Jun 19, 2026
958d30f
Add text stream result API on client
thelovekesh Jun 19, 2026
d4eb377
Add streaming to openai compat model
thelovekesh Jun 19, 2026
76737ee
Add stream-text support to cli
thelovekesh Jun 19, 2026
d7502ed
Add completion callback support
thelovekesh Jun 19, 2026
f169b69
Add event dispatchers after streaming text results
thelovekesh Jun 19, 2026
bdda338
Add value object to store tool call delta
thelovekesh Jun 23, 2026
94edcd2
Update gen ai result chink value object to store tool call delta
thelovekesh Jun 23, 2026
2f31263
Add tool calls accumulation
thelovekesh Jun 23, 2026
befc706
Add support to parse function call for open ai compat provider
thelovekesh Jun 23, 2026
8c12cdc
Update gen ai result chunk to store additional data
thelovekesh Jun 23, 2026
f2aada5
Add additional data support
thelovekesh Jun 23, 2026
512ce47
Add exception for stream responses
thelovekesh Jun 23, 2026
96a895b
Add stream error exception
thelovekesh Jun 23, 2026
0e511db
Update gen ai result chunk responsibilities
thelovekesh Jun 24, 2026
412a813
Add value object to store candidate delta
thelovekesh Jun 24, 2026
0450a83
Add streamed chunks accumulator
thelovekesh Jun 24, 2026
5762881
Update gen ai result class to use chunk accumulator
thelovekesh Jun 24, 2026
4422749
Update candidate chinks handling
thelovekesh Jun 24, 2026
baab7a3
Update ServerSentEvent class path
thelovekesh Jun 24, 2026
a0937e5
Fix docblock
thelovekesh Jun 24, 2026
e5ddc70
Add exception if stream is being consumed again
thelovekesh Jun 24, 2026
d9e1809
Add mock to simulate stream failure
thelovekesh Jun 24, 2026
d197a79
Add stream helpers in openai compat model mock
thelovekesh Jun 24, 2026
a216e62
Add test cases for ChunkAccumulator
thelovekesh Jun 24, 2026
7822fd9
Add test cases for StreamedGenerativeAiResult
thelovekesh Jun 24, 2026
4a6182b
Add test case for streaming in open ai compat text gen modal
thelovekesh Jun 24, 2026
d32da75
Add streaming support in modal creation mock
thelovekesh Jun 24, 2026
760e95a
Add test cases for event dispatch in streamed results
thelovekesh Jun 24, 2026
4fd90a2
Add text stream in prompt builder
thelovekesh Jun 24, 2026
ef2de11
Add streaming support test cases for http transporter
thelovekesh Jun 24, 2026
f3f0dc9
Update code comments
thelovekesh Jun 24, 2026
b2f9fb7
Add value objects test cases
thelovekesh Jun 24, 2026
ba62029
Add test cases for exception handlers
thelovekesh Jun 24, 2026
35c8ba7
Add test for stream response
thelovekesh Jun 24, 2026
e55e429
Add test for streaming text support in ai client
thelovekesh Jun 24, 2026
91ae1c4
Add test case for stream option in request options
thelovekesh Jun 24, 2026
c96776b
Add event to generate result error event
thelovekesh Jun 24, 2026
9ee700e
Update chunk accumulator to force channel order
thelovekesh Jun 24, 2026
6587a5f
Fix docblock tag order
thelovekesh Jun 24, 2026
fec38bf
Add lifecycle events on gen ai result streams
thelovekesh Jun 24, 2026
43b15c2
Add tets cases for stream lifecycle events
thelovekesh Jun 24, 2026
b21bb8f
Add coercion for usage values
thelovekesh Jun 24, 2026
f05dce8
Update stream chunks data type
thelovekesh Jun 24, 2026
f866fcf
Update event dispatch in prompt builder
thelovekesh Jun 24, 2026
5f460ac
Update logic to determine the channel order
thelovekesh Jun 24, 2026
2fa7159
Update initial candidate index to 0
thelovekesh Jun 24, 2026
a3f098e
Remove finalized state flag
thelovekesh Jun 24, 2026
77c730d
Add coverage ignore for forward compat code
thelovekesh Jun 24, 2026
4a345d6
Add test case to assert default token usage values
thelovekesh Jun 24, 2026
1bd7ca9
Add test case for streamed body serialization
thelovekesh Jun 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions cli.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
* OPENAI_API_KEY=123456 php cli.php 'Your prompt here' --providerId=openai
* GOOGLE_API_KEY=123456 OPENAI_API_KEY=123456 php cli.php 'Your prompt here'
*
* To stream the response as it arrives, use --outputFormat=stream-text:
* OPENAI_API_KEY=123456 php cli.php 'Your prompt here' --providerId=openai --outputFormat=stream-text
*
* For large prompts (e.g., with images), use stdin or file input:
* cat prompt.json | php cli.php - --providerId=openai --modelId=gpt-4o
* php cli.php @prompt.json --providerId=openai --modelId=gpt-4o
Expand Down Expand Up @@ -190,7 +193,15 @@ static function ($item) {
}

try {
if ($outputFormat === 'image-json' || $outputFormat === 'image-base64') {
if ($outputFormat === 'stream-text') {
$stream = $promptBuilder->streamGenerateTextResult();
foreach ($stream as $chunk) {
echo $chunk->getDeltaText();
flush();
}
echo PHP_EOL;
$result = $stream->getFinalResult();
} elseif ($outputFormat === 'image-json' || $outputFormat === 'image-base64') {
$result = $promptBuilder->generateImageResult();
} else {
$result = $promptBuilder->generateTextResult();
Expand All @@ -204,7 +215,11 @@ static function ($item) {
logInfo("Using provider ID: \"{$result->getProviderMetadata()->getId()}\"");
logInfo("Using model ID: \"{$result->getModelMetadata()->getId()}\"");

$output = null;
switch ($outputFormat) {
case 'stream-text':
// The text was already streamed to stdout above.
break;
case 'result-json':
$output = json_encode($result, JSON_PRETTY_PRINT);
break;
Expand All @@ -222,4 +237,6 @@ static function ($item) {
$output = $result->toText();
}

printOutput($output);
if (is_string($output)) {
printOutput($output);
}
3 changes: 3 additions & 0 deletions phpcs.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@
<!-- Use PSR-12 standard -->
<rule ref="PSR12"/>

<!-- Check up to 20 files simultaneously. -->
<arg name="parallel" value="20"/>

<!-- Check PHP 7.4 compatibility -->
<rule ref="PHPCompatibility">
<!-- Exclude functions that are polyfilled in src/polyfills.php -->
Expand Down
28 changes: 28 additions & 0 deletions src/AiClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
use WordPress\AiClient\Providers\Models\DTO\ModelConfig;
use WordPress\AiClient\Providers\ProviderRegistry;
use WordPress\AiClient\Results\DTO\GenerativeAiResult;
use WordPress\AiClient\Results\StreamedGenerativeAiResult;

/**
* Main AI Client class providing both fluent and traditional APIs for AI operations.
Expand Down Expand Up @@ -290,6 +291,33 @@ public static function generateTextResult(
return self::getConfiguredPromptBuilder($prompt, $modelOrConfig, $registry)->generateTextResult();
}

/**
* Streams a text result using the traditional API approach.
*
* Iterate the returned object to consume chunks as they arrive, then call
* its getFinalResult() for the complete result.
*
* @since n.e.x.t
*
* @param Prompt $prompt The prompt content.
* @param ModelInterface|ModelConfig|null $modelOrConfig Optional specific model to use,
* or model configuration for auto-discovery,
* or null for defaults.
* @param ProviderRegistry|null $registry Optional custom registry. If null, uses default.
* @return StreamedGenerativeAiResult The streamed result.
*
* @throws \InvalidArgumentException If the prompt format is invalid.
* @throws \RuntimeException If no suitable model is found or it does not support streaming.
*/
public static function streamGenerateTextResult(
$prompt,
$modelOrConfig = null,
?ProviderRegistry $registry = null
): StreamedGenerativeAiResult {
self::validateModelOrConfigParameter($modelOrConfig);
return self::getConfiguredPromptBuilder($prompt, $modelOrConfig, $registry)->streamGenerateTextResult();
}

/**
* Generates an image using the traditional API approach.
*
Expand Down
62 changes: 62 additions & 0 deletions src/Builders/PromptBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
use WordPress\AiClient\Common\Exception\RuntimeException;
use WordPress\AiClient\Events\AfterGenerateResultEvent;
use WordPress\AiClient\Events\BeforeGenerateResultEvent;
use WordPress\AiClient\Events\GenerateResultErrorEvent;
use WordPress\AiClient\Files\DTO\File;
use WordPress\AiClient\Files\Enums\FileTypeEnum;
use WordPress\AiClient\Files\Enums\MediaOrientationEnum;
Expand All @@ -26,11 +27,13 @@
use WordPress\AiClient\Providers\Models\Enums\CapabilityEnum;
use WordPress\AiClient\Providers\Models\ImageGeneration\Contracts\ImageGenerationModelInterface;
use WordPress\AiClient\Providers\Models\SpeechGeneration\Contracts\SpeechGenerationModelInterface;
use WordPress\AiClient\Providers\Models\TextGeneration\Contracts\StreamingTextGenerationModelInterface;
use WordPress\AiClient\Providers\Models\TextGeneration\Contracts\TextGenerationModelInterface;
use WordPress\AiClient\Providers\Models\TextToSpeechConversion\Contracts\TextToSpeechConversionModelInterface;
use WordPress\AiClient\Providers\Models\VideoGeneration\Contracts\VideoGenerationModelInterface;
use WordPress\AiClient\Providers\ProviderRegistry;
use WordPress\AiClient\Results\DTO\GenerativeAiResult;
use WordPress\AiClient\Results\StreamedGenerativeAiResult;
use WordPress\AiClient\Tools\DTO\FunctionDeclaration;
use WordPress\AiClient\Tools\DTO\FunctionResponse;
use WordPress\AiClient\Tools\DTO\WebSearch;
Expand Down Expand Up @@ -1048,6 +1051,65 @@ public function generateTextResult(): GenerativeAiResult
return $this->generateResult(CapabilityEnum::textGeneration());
}

/**
* Streams a text result from the prompt.
*
* @since n.e.x.t
*
* @return StreamedGenerativeAiResult The streamed result.
* @throws InvalidArgumentException If the prompt or model validation fails.
* @throws RuntimeException If the model does not support streaming text generation.
Comment thread
justlevine marked this conversation as resolved.
*/
public function streamGenerateTextResult(): StreamedGenerativeAiResult
{
$this->includeOutputModalities(ModalityEnum::text());
$this->validateMessages();

$capability = CapabilityEnum::textGeneration();
$model = $this->getConfiguredModel($capability);

if (!$model instanceof StreamingTextGenerationModelInterface) {
throw new RuntimeException(
sprintf(
'Model "%s" does not support streaming text generation.',
$model->metadata()->getId()
)
);
}
Comment on lines +1071 to +1078

@thelovekesh thelovekesh Jun 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a model doesn't implement this interface, it will fail even if the model supports text generation. It fails because streaming is gated behind this opt-in interface that no provider actually implements. We can't discover our way around it either, since streaming isn't a CapabilityEnum, so discovery can't tell which models stream and may still pick one that doesn't.

The clean solution is to remove StreamingTextGenerationModelInterface and rather add streamGenerateTextResult(array $prompt): StreamedGenerativeAiResult to TextGenerationModelInterface. Streaming is a networking primitive, not a real model capability, so every text model can stream: natively where the provider's API supports it, or by emulating it (one generateTextResult() call yielded as a single chunk) where it doesn't. Adding a method to the interface sounds like a breaking change, but if we put the emulation default in the shared base (AbstractApiBasedModel) that every provider already extends, they all inherit it and nothing breaks.

Another way is to add a new capability and wire discovery to find a model with it, but that isn't standard given streaming isn't a model-level capability, it's a networking primitive, not something a model generates. We'd also still need a method on an interface plus every provider opting in, so it's more machinery for something that isn't really model-level.


$messages = $this->messages;

return $model->streamGenerateTextResult($messages)
->onStart(function () use ($messages, $model, $capability): void {
$this->dispatchEvent(new BeforeGenerateResultEvent($messages, $model, $capability));
})
->onComplete(function (GenerativeAiResult $result) use ($messages, $model, $capability): void {
$this->dispatchEvent(new AfterGenerateResultEvent($messages, $model, $capability, $result));
})
->onError(function (\Throwable $error) use ($messages, $model, $capability): void {
$this->dispatchEvent(new GenerateResultErrorEvent($messages, $model, $capability, $error));
});
}

/**
* Streams generated text from the prompt as it arrives.
*
* @since n.e.x.t
*
* @return iterable<string> The text deltas, in order.
* @throws InvalidArgumentException If the prompt or model validation fails.
* @throws RuntimeException If the model does not support streaming text generation.
*/
public function streamGenerateText(): iterable
{
foreach ($this->streamGenerateTextResult() as $chunk) {
$delta = $chunk->getDeltaText();
if ($delta !== '') {
yield $delta;
}
}
}

/**
* Generates an image result from the prompt.
*
Expand Down
122 changes: 122 additions & 0 deletions src/Events/GenerateResultErrorEvent.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
<?php

declare(strict_types=1);

namespace WordPress\AiClient\Events;

use Throwable;
use WordPress\AiClient\Messages\DTO\Message;
use WordPress\AiClient\Providers\Models\Contracts\ModelInterface;
use WordPress\AiClient\Providers\Models\Enums\CapabilityEnum;

/**
* Class GenerateResultErrorEvent.
*
* @since n.e.x.t
*/
class GenerateResultErrorEvent
{
/**
* @var list<Message> The messages that were sent to the model.
*/
private array $messages;

/**
* @var ModelInterface The model that processed the prompt.
*/
private ModelInterface $model;

/**
* @var CapabilityEnum|null The capability that was used for generation.
*/
private ?CapabilityEnum $capability;

/**
* @var Throwable The error that occurred during generation.
*/
private Throwable $error;

/**
* Constructor.
*
* @since n.e.x.t
*
* @param list<Message> $messages The messages that were sent to the model.
* @param ModelInterface $model The model that processed the prompt.
* @param CapabilityEnum|null $capability The capability that was used for generation.
* @param Throwable $error The error that occurred during generation.
*/
public function __construct(
array $messages,
ModelInterface $model,
?CapabilityEnum $capability,
Throwable $error
) {
$this->messages = $messages;
$this->model = $model;
$this->capability = $capability;
$this->error = $error;
}

/**
* Gets the messages that were sent to the model.
*
* @since n.e.x.t
*
* @return list<Message> The messages.
*/
public function getMessages(): array
{
return $this->messages;
}

/**
* Gets the model that processed the prompt.
*
* @since n.e.x.t
*
* @return ModelInterface The model.
*/
public function getModel(): ModelInterface
{
return $this->model;
}

/**
* Gets the capability that was used for generation.
*
* @since n.e.x.t
*
* @return CapabilityEnum|null The capability, or null if not specified.
*/
public function getCapability(): ?CapabilityEnum
{
return $this->capability;
}

/**
* Gets the error that occurred during generation.
*
* @since n.e.x.t
*
* @return Throwable The error.
*/
public function getError(): Throwable
{
return $this->error;
}

/**
* Performs a deep clone of the event.
*
* @since n.e.x.t
*/
public function __clone()
{
$clonedMessages = [];
foreach ($this->messages as $message) {
$clonedMessages[] = clone $message;
}
$this->messages = $clonedMessages;
}
}
Loading
Loading