We're recording token usage on our side in outputs, however I've discovered an inconsistency in our monitoring.
If a user is using an OpenAI model, the completionTokens key includes the thought tokens.
If a user is using a Gemini model, the completionTokens key excludes the thought tokens.
Relevant file paths:
https://github.com/prism-php/prism/blob/main/src/Providers/Gemini/Handlers/Text.php#L184
https://github.com/prism-php/prism/blob/main/src/Providers/OpenAI/Handlers/Text.php#L177
We're seeing records in our DB where the thought tokens is greater than the completion tokens - digging into it candidateTokens is only the output tokens while the separate thoughtTokensCount is just the thought tokens.
I'd expect some symmetry in the data - but any change here would be a breaking change for consumers.
Happy to make a PR if there is a preferred route (i.e completionTokens is either deducted thought tokens or adds them depending on the route?)
We're recording token usage on our side in outputs, however I've discovered an inconsistency in our monitoring.
If a user is using an OpenAI model, the completionTokens key includes the thought tokens.
If a user is using a Gemini model, the completionTokens key excludes the thought tokens.
Relevant file paths:
https://github.com/prism-php/prism/blob/main/src/Providers/Gemini/Handlers/Text.php#L184
https://github.com/prism-php/prism/blob/main/src/Providers/OpenAI/Handlers/Text.php#L177
We're seeing records in our DB where the thought tokens is greater than the completion tokens - digging into it candidateTokens is only the output tokens while the separate thoughtTokensCount is just the thought tokens.
I'd expect some symmetry in the data - but any change here would be a breaking change for consumers.
Happy to make a PR if there is a preferred route (i.e completionTokens is either deducted thought tokens or adds them depending on the route?)