|
| 1 | +--- |
| 2 | +title: Self-hosted agent data storage |
| 3 | +description: >- |
| 4 | + Route agent transcripts, artifacts, and prompt attachments to your own cloud storage. |
| 5 | +--- |
| 6 | +import { VARS } from '@data/vars'; |
| 7 | +import { Tabs, TabItem } from '@astrojs/starlight/components'; |
| 8 | + |
| 9 | +Self-hosted agent data storage lets your team keep cloud agent run data — conversation transcripts, artifacts, and prompt attachments — in a storage bucket you own, instead of Warp-managed storage. The {VARS.WARP_AUTOMATION_PLATFORM} still orchestrates every run and routes model inference. |
| 10 | + |
| 11 | +:::note |
| 12 | +Self-hosted agent data storage is only available on Enterprise plans. [Contact sales](https://www.warp.dev/contact-sales) to learn more. |
| 13 | +::: |
| 14 | + |
| 15 | +## Supported storage providers |
| 16 | + |
| 17 | +Only **AWS S3** is available today. Google Cloud Storage and Azure Blob Storage support is planned. |
| 18 | + |
| 19 | +## What data can be self-hosted |
| 20 | + |
| 21 | +Three categories of agent run data can be routed to your own storage, each configured independently: |
| 22 | + |
| 23 | +* **Artifacts** - Files, Computer Use screenshots, and other binary artifacts an agent produces during a run. |
| 24 | +* **Conversation transcripts** - Saved transcripts of agent conversations, including system messages, tool calls, and tool results. |
| 25 | +* **Prompt attachments** - Files uploaded when starting or continuing an agent run. |
| 26 | + |
| 27 | +A category you don't map keeps using Warp-hosted storage. For example, you can route artifacts to your bucket while transcripts stay with Warp. |
| 28 | + |
| 29 | +Only team admins can connect, remap, or disconnect storage. See [Access, billing, and identity](/platform/team-access-billing-and-identity/) for more on team roles. |
| 30 | + |
| 31 | +:::caution |
| 32 | +Data is not migrated when changing storage mappings. It stays in its original location, but will no longer be accessible through Warp APIs. |
| 33 | +::: |
| 34 | + |
| 35 | +## How data flows |
| 36 | + |
| 37 | +Mapping a category to your bucket changes where that category's objects are written and read. Everything else about a run stays the same. |
| 38 | + |
| 39 | +Warp's control plane accesses self-hosted storage through a provider-specific IAM role that you provide, ensuring that all access is auditable and that Warp's |
| 40 | +permissions may be revoked at any time. The control plane writes to the bucket over the course of a run, such as by updating the conversation transcript after |
| 41 | +each turn. It also reads from the bucket to provide run data, including conversation history and the display of computer use screenshots. Warp periodically |
| 42 | +accesses persisted data for background maintenance such as to run [scorers](/factories/measure-and-improve/scorers/). |
| 43 | + |
| 44 | +Warp never caches bucket contents. It uses presigned URLs wherever possible so that data flows directly from your bucket to the client, without |
| 45 | +transiting Warp's backend. |
| 46 | + |
| 47 | +## Setting up AWS S3 |
| 48 | + |
| 49 | +### 1. Create an S3 bucket |
| 50 | + |
| 51 | +Create the bucket that will hold your agent data. Warp writes objects using your bucket's own encryption settings — both SSE-S3 (the default) and SSE-KMS are supported. |
| 52 | + |
| 53 | +<Tabs> |
| 54 | + <TabItem label="AWS CLI"> |
| 55 | + ```bash |
| 56 | + aws s3 mb s3://YOUR_BUCKET_NAME --region YOUR_AWS_REGION |
| 57 | + ``` |
| 58 | + </TabItem> |
| 59 | + <TabItem label="Terraform"> |
| 60 | + ```hcl title="main.tf" |
| 61 | + resource "aws_s3_bucket" "warp_agent_data" { |
| 62 | + bucket = "YOUR_BUCKET_NAME" |
| 63 | + } |
| 64 | + ``` |
| 65 | + </TabItem> |
| 66 | +</Tabs> |
| 67 | + |
| 68 | +See AWS's guide on [creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) for console and CLI equivalents. |
| 69 | + |
| 70 | +### 2. Open the connect dialog in the Admin Panel |
| 71 | + |
| 72 | +Self-hosted storage is configured in Warp's [Admin Panel](/enterprise/team-management/admin-panel/). The storage connection dialog generates the exact policies that your IAM role needs. |
| 73 | +Open it before creating the role, so you can copy those values directly. It can be reopened at any time. |
| 74 | + |
| 75 | +In the Admin Panel, go to the **Platform** settings tab. Under **Agent data storage**, click **Connect external storage** and choose **AWS S3**. Enter your bucket name and region in the "Bucket" and "Region" fields, and leave the dialog open — the next two steps use the policies it displays below the form. |
| 76 | + |
| 77 | +### 3. Create the IAM role using the trust policy Warp shows you |
| 78 | + |
| 79 | +The connect dialog already substitutes your team's external ID into the trust policy. |
| 80 | + |
| 81 | +<Tabs> |
| 82 | + <TabItem label="AWS CLI"> |
| 83 | + Save the trust policy from the dialog locally: |
| 84 | + |
| 85 | + ```json title="trust-policy.json" |
| 86 | + { |
| 87 | + "Version": "2012-10-17", |
| 88 | + "Statement": [ |
| 89 | + { |
| 90 | + "Effect": "Allow", |
| 91 | + "Principal": { "AWS": "arn:aws:iam::162908503950:role/warp-hosted-data-storage-prod" }, |
| 92 | + "Action": "sts:AssumeRole", |
| 93 | + "Condition": { |
| 94 | + "StringEquals": { "sts:ExternalId": "YOUR_TEAM_ID" } |
| 95 | + } |
| 96 | + } |
| 97 | + ] |
| 98 | + } |
| 99 | + ``` |
| 100 | + |
| 101 | + Then create the role: |
| 102 | + |
| 103 | + ```bash |
| 104 | + aws iam create-role \ |
| 105 | + --role-name warp-agent-data-storage \ |
| 106 | + --assume-role-policy-document file://trust-policy.json |
| 107 | + ``` |
| 108 | + </TabItem> |
| 109 | + <TabItem label="Terraform"> |
| 110 | + Set your Warp team ID as a variable, and add the following IAM role resource: |
| 111 | + |
| 112 | + ```hcl title="main.tf" |
| 113 | + resource "aws_iam_role" "warp_agent_data_storage" { |
| 114 | + name = "warp-agent-data-storage" |
| 115 | +
|
| 116 | + assume_role_policy = jsonencode({ |
| 117 | + Version = "2012-10-17" |
| 118 | + Statement = [{ |
| 119 | + Effect = "Allow" |
| 120 | + Principal = { AWS = "arn:aws:iam::162908503950:role/warp-hosted-data-storage-prod" } |
| 121 | + Action = "sts:AssumeRole" |
| 122 | + Condition = { |
| 123 | + StringEquals = { "sts:ExternalId" = var.warp_team_id } |
| 124 | + } |
| 125 | + }] |
| 126 | + }) |
| 127 | + } |
| 128 | + ``` |
| 129 | + </TabItem> |
| 130 | +</Tabs> |
| 131 | + |
| 132 | +See AWS's guides on [creating a role to delegate permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html) and on [using an external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) for background on this pattern. |
| 133 | + |
| 134 | +### 4. Attach the permissions policy |
| 135 | + |
| 136 | +This policy grants the role access to your bucket and nothing else. |
| 137 | + |
| 138 | +<Tabs> |
| 139 | + <TabItem label="AWS CLI"> |
| 140 | + Save the permissions policy from the dialog locally, with your bucket name substituted: |
| 141 | + |
| 142 | + ```json title="permissions-policy.json" |
| 143 | + { |
| 144 | + "Version": "2012-10-17", |
| 145 | + "Statement": [ |
| 146 | + { |
| 147 | + "Effect": "Allow", |
| 148 | + "Action": ["s3:ListBucket"], |
| 149 | + "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME" |
| 150 | + }, |
| 151 | + { |
| 152 | + "Effect": "Allow", |
| 153 | + "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], |
| 154 | + "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*" |
| 155 | + } |
| 156 | + ] |
| 157 | + } |
| 158 | + ``` |
| 159 | + |
| 160 | + Then attach it to the role: |
| 161 | + |
| 162 | + ```bash |
| 163 | + aws iam put-role-policy \ |
| 164 | + --role-name warp-agent-data-storage \ |
| 165 | + --policy-name warp-bucket-access \ |
| 166 | + --policy-document file://permissions-policy.json |
| 167 | + ``` |
| 168 | + </TabItem> |
| 169 | + <TabItem label="Terraform"> |
| 170 | + ```hcl title="main.tf" |
| 171 | + resource "aws_iam_role_policy" "warp_bucket_access" { |
| 172 | + name = "warp-bucket-access" |
| 173 | + role = aws_iam_role.warp_agent_data_storage.id |
| 174 | +
|
| 175 | + policy = jsonencode({ |
| 176 | + Version = "2012-10-17" |
| 177 | + Statement = [ |
| 178 | + { |
| 179 | + Effect = "Allow" |
| 180 | + Action = ["s3:ListBucket"] |
| 181 | + Resource = aws_s3_bucket.warp_agent_data.arn |
| 182 | + }, |
| 183 | + { |
| 184 | + Effect = "Allow" |
| 185 | + Action = ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"] |
| 186 | + Resource = "${aws_s3_bucket.warp_agent_data.arn}/*" |
| 187 | + } |
| 188 | + ] |
| 189 | + }) |
| 190 | + } |
| 191 | + ``` |
| 192 | + </TabItem> |
| 193 | +</Tabs> |
| 194 | + |
| 195 | +See AWS's [IAM policies for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-iam-policies.html) reference for the full set of available actions and conditions. |
| 196 | + |
| 197 | +#### If the bucket uses SSE-KMS |
| 198 | + |
| 199 | +Buckets encrypted with SSE-S3, the S3 default, need no further permissions. If your bucket uses SSE-KMS, add one more statement scoped to the ARN of the key that encrypts it. |
| 200 | + |
| 201 | +<Tabs> |
| 202 | + <TabItem label="AWS CLI"> |
| 203 | + Add this statement to the `Statement` array in `permissions-policy.json`, then reattach the policy: |
| 204 | + |
| 205 | + ```json title="permissions-policy.json" |
| 206 | + { |
| 207 | + "Effect": "Allow", |
| 208 | + "Action": ["kms:Decrypt", "kms:GenerateDataKey*", "kms:DescribeKey"], |
| 209 | + "Resource": "arn:aws:kms:YOUR_AWS_REGION:YOUR_ACCOUNT_ID:key/YOUR_KEY_ID", |
| 210 | + "Condition": { |
| 211 | + "StringLike": { "kms:ViaService": "s3.*.amazonaws.com" } |
| 212 | + } |
| 213 | + } |
| 214 | + ``` |
| 215 | + </TabItem> |
| 216 | + <TabItem label="Terraform"> |
| 217 | + Add this statement to the `Statement` list in `aws_iam_role_policy.warp_bucket_access`, referencing the key that encrypts the bucket: |
| 218 | + |
| 219 | + ```hcl title="main.tf" |
| 220 | + { |
| 221 | + Effect = "Allow" |
| 222 | + Action = ["kms:Decrypt", "kms:GenerateDataKey*", "kms:DescribeKey"] |
| 223 | + Resource = aws_kms_key.warp_agent_data.arn |
| 224 | + Condition = { |
| 225 | + StringLike = { "kms:ViaService" = "s3.*.amazonaws.com" } |
| 226 | + } |
| 227 | + } |
| 228 | + ``` |
| 229 | + </TabItem> |
| 230 | +</Tabs> |
| 231 | + |
| 232 | +### 5. Enter the role ARN and choose what to store |
| 233 | + |
| 234 | +Back in the connect dialog, paste your new role's ARN into the "Role ARN" field. In the "Data to store" dropdown, select **Artifacts**, **Conversation transcripts**, or **Prompt attachments** — whichever categories this bucket should store — then click **Connect**. |
| 235 | + |
| 236 | +Warp immediately assumes the role and checks that the bucket is reachable. If the trust policy or external ID don't match, or the permissions policy doesn't grant the required actions, the connection is rejected with an error describing which check failed. See [Troubleshooting](#troubleshooting). |
| 237 | + |
| 238 | +**Expected outcome:** The bucket appears as an option in each mapped category's storage dropdown, and new writes for those categories go to your bucket going forward. |
| 239 | + |
| 240 | +## Changing or removing storage |
| 241 | + |
| 242 | +Storage mappings can be changed at any time from **Manage connected storage**, under **Agent data storage** in the Admin Panel Platform section. |
| 243 | + |
| 244 | +* **Remap a category** - Choose a different bucket, or **Warp-hosted**, from that category's dropdown. Only new writes use the new target. |
| 245 | +* **Disconnect a bucket** - Remove the connection from the manage-storage drawer. A bucket must be unmapped from every category before it can be disconnected. |
| 246 | + |
| 247 | +:::caution |
| 248 | +Neither remapping nor disconnecting migrates or deletes existing data. If you disconnect a bucket, objects already written there remain in your AWS account — Warp simply stops referencing them. |
| 249 | +::: |
| 250 | + |
| 251 | +## Troubleshooting |
| 252 | + |
| 253 | +### "Warp could not assume the provided IAM role" |
| 254 | + |
| 255 | +The role's trust policy doesn't allow Warp's broker role as principal, or its `sts:ExternalId` condition doesn't match your team. Recopy the trust policy from the connect dialog and confirm it's attached to the role you entered. |
| 256 | + |
| 257 | +### "Warp was denied access to the S3 bucket" |
| 258 | + |
| 259 | +The trust policy is correct, but the role's permissions policy doesn't grant the required S3 actions on the bucket. Recheck the permissions policy against the one shown in the dialog, and confirm the bucket name matches exactly. |
| 260 | + |
| 261 | +### "Warp could not reach the S3 bucket" |
| 262 | + |
| 263 | +The bucket name or region is wrong, or the bucket doesn't exist. Confirm both match what you created in AWS. |
| 264 | + |
| 265 | +## Related pages |
| 266 | + |
| 267 | +* [Architecture](/platform/architecture/#data-security-and-boundaries) - How run data, control-plane data, and inference credentials flow through Warp's platform. |
| 268 | +* [Self-hosting overview](/platform/self-hosting/) - Run agent compute on your own infrastructure instead of Warp-managed servers. |
| 269 | +* [Cloud agent secrets](/platform/secrets/) - Store credentials agents use during a run. |
| 270 | +* [Admin Panel for teams](/enterprise/team-management/admin-panel/) - The full reference for team settings, including the Platform section. |
0 commit comments