fix: make integrations tests pass - #241
Conversation
|
Thanks for the pull request, @Henrrypg! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #241 +/- ##
=======================================
Coverage 95.36% 95.36%
=======================================
Files 69 69
Lines 8088 8088
Branches 430 430
=======================================
Hits 7713 7713
Misses 281 281
Partials 94 94
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7b29097 to
c79c244
Compare
| '<div style="display:none"><!-- sponsored --></div>' | ||
| '<img src="https://example.com/cooking.png" alt="Cooking illustration" />' | ||
| "A complete recipe requires exactly five steps: " | ||
| "1. Gather ingredients. " |
There was a problem hiding this comment.
Instead of having the list here in the same block. I think the test would be stronger if blocks has many html type content pieces and the list is distributed in them.
|
|
||
| # Modified chat_with_context: "ask clarifying questions" and "guided questioning" rules | ||
| # removed so the model answers directly from content or states the limitation explicitly. | ||
| _GROUNDED_CHAT_INSTRUCTION = ( |
There was a problem hiding this comment.
Do you think it would be possible to change the test to actually use the prompt version that we have and user for chat_with_context. That would mean doing some minor updates there, but at the same time, we are making the base prompt better
|
/integration-test |
felipemontoya
left a comment
There was a problem hiding this comment.
Thanks for all the good work @Henrrypg. I think in its current form the tests are testing real capabilites of different models that we use in actual profiles.
Now, having them not as Xpass, but actual passed tests and having found one more bug during this PR of improvement in testing I would say we are a lot safer with this.
This PR add some fixes to prompts and instructions for integrations tests in order to make AI pass them