fix(truncation strategy): add proper message history management through summarization#257
fix(truncation strategy): add proper message history management through summarization#257philipph-askui merged 18 commits intomainfrom
Conversation
…ges from the history thereby staying compatible with prompt caching
programminx-askui
left a comment
There was a problem hiding this comment.
This are my first comments.
I need to continue the review.
| # see https://docs.anthropic.com/en/api/messages#body-messages | ||
| MAX_MESSAGES = 100_000 | ||
|
|
||
| IMAGE_REMOVED_PLACEHOLDER = "[Screenshot removed to reduce message history length]" |
There was a problem hiding this comment.
I assume the description of the image is partly in the messages history or? So the LLM can assume what the App has done.
e.g. I see that an Error Message popped up. ...
There was a problem hiding this comment.
the rest of the content in the message history, e.g. the interpretation of the LLM remains unchanged. we just remove the image here. Generating a dedicated description for each image would create additional traffic and cost, hence I decided against that option
| not _is_tool_result_user_message(message) | ||
| and previous_message is not None | ||
| and previous_message.role == "assistant" | ||
| response = vlm_provider.create_message( |
There was a problem hiding this comment.
Is this Token Consumption tracked in our Report? Please say yes. :-D
There was a problem hiding this comment.
| MessageParam( | ||
| role="user", | ||
| content=( | ||
| "Please provide a concise summary of the " |
There was a problem hiding this comment.
How did you came up this the message?
Can me make the message configurable?
I would add something like. You are an CUA agent how should continue the conversation.. But
There was a problem hiding this comment.
for the exact content: lets keep it simple in the beginning. This simple prompt worked quite well in my test. If we see that the agent loses track after summarizing, we can improve it
|
@mlikasam-askui : please wait with reviewing until tomorrow. I made some refactorings after Dominiks comments and need time to test them before pushing. I will reach out to you as soon as this is ready. |
… simplify user-sided configuration
@mlikasam-askui : this is now ready to review :-) |
…gTruncationStrategy`
| return self._full_message_history | ||
|
|
||
|
|
||
| class SlidingImageWindowSummarizingTruncationStrategy(TruncationStrategy): |
There was a problem hiding this comment.
This "SlidingImageWindowSummarizingTruncationStrategy" is still experimental and will also create a warning log. I will test it in detail over the coming days.
Refactors truncation strategies to align with prompt caching
For reviewing: I think you don't need to review the experimental SlidingImageWindowSummarizingTruncationStrategy in detail at this point.