Skip to content

Commit 4b2bf65

Browse files
authored
Merge branch 'main' into fix/broken-changelog-links-20260923
2 parents 07565fd + 3fefe16 commit 4b2bf65

1 file changed

Lines changed: 5 additions & 5 deletions

File tree

‎src/content/docs/factories/benchmarks.mdx‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
2-
title: Benchmarking factory agent configurations
2+
title: Benchmarks
33
description: >-
4-
Benchmarks compare a factory agent's model and runner configurations on fixed
5-
tasks. Use the results to choose a production configuration.
4+
Warp Factories benchmarks compare agent model and runner configurations on
5+
fixed tasks so you can choose a production configuration.
66
sidebar:
77
label: "Benchmarks"
88
---
@@ -13,7 +13,7 @@ import VideoEmbed from '@components/VideoEmbed.astro';
1313
Warp Factories is in **Early Access** and available to a limited set of teams. [Request access](https://www.warp.dev/factories/request-access) to use it with your team.
1414
:::
1515

16-
Benchmarks compare model and runner configurations for one factory agent on the same fixed tasks. Use a benchmark to test a change on representative work before you apply it to your factory.
16+
Warp Factories benchmarks compare model and runner configurations for one factory agent on the same fixed tasks. Use a benchmark to test a change on representative work before you apply it to your factory.
1717
This video shows how to compare benchmark results for quality and cost before changing a coding agent's configuration.
1818
<VideoEmbed url="https://www.youtube.com/watch?v=LKx5MeUsptQ" title="Comparing factory benchmark results to reduce coding agent costs" />
1919

@@ -96,7 +96,7 @@ After the run completes, review the result as a comparison, not as a universal m
9696

9797
The run's "Total cost" includes model usage for trials and Scorer usage. It estimates those costs from credits at your team's current rate, so it is not a billed amount. A failed or cancelled benchmark shows only results that finished scoring before the run stopped.
9898

99-
## Apply a result
99+
## Apply a benchmark result
100100

101101
Change one configuration at a time. If the evidence supports a candidate, update the agent's model or runner in the factory dashboard, or submit the change through your [factory definition](/factories/factory-as-code/). Keep the relevant Scorers active, then compare later production runs with the baseline you recorded before the change.
102102

0 commit comments

Comments
 (0)