From 83baa592c4b3a69c8262d50786a5256ffb1ee65a Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Wed, 15 Jul 2026 17:04:55 +0300 Subject: [PATCH 01/18] docs: add Forge product roadmap --- docs/index.md | 1 + docs/roadmap.md | 461 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 462 insertions(+) create mode 100644 docs/roadmap.md diff --git a/docs/index.md b/docs/index.md index 65c6d56ed..6a7134095 100644 --- a/docs/index.md +++ b/docs/index.md @@ -42,6 +42,7 @@ graph TD ## Quick Links +- [Product Roadmap](roadmap.md) — Where Forge is going and how major capabilities fit together - [Getting Started](getting-started.md) — Set up Forge in 10 minutes - [Feature Workflow](guide/feature-workflow.md) — How features flow through Forge - [Bug Workflow](guide/bug-workflow.md) — How bug diagnosis and implementation flow through Forge diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 000000000..255ee651f --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,461 @@ +# Forge Product Roadmap + +**Status:** Draft for discussion +**Planning horizon:** Outcome-based; dates and release assignments are intentionally TBD +**Last reviewed:** 2026-07-15 + +## Product direction + +Forge should grow from a Jira-to-GitHub automation into an extensible, governed SDLC +orchestrator: one control plane for turning product intent into validated changes and, +when policy permits, temporary environments. It should coordinate systems that already +own source control, execution, CI, and deployment rather than becoming a replacement +for them. + +The durable product boundary is: + +- Forge owns workflow state, policy gates, orchestration, audit history, and recovery. +- Provider adapters own interactions with Jira, GitHub, GitLab, and future systems. +- Execution drivers own where isolated agent work runs. +- Deployment plugins own provisioning and teardown in external platforms. +- Agents propose typed outcomes; the workflow validates them and performs side effects. + +This direction preserves Forge's differentiator—long-running, human-governed delivery +workflows—while making Jira, GitHub, and Podman defaults rather than permanent limits. + +## Product principles + +1. **Contracts before integrations.** Define stable domain contracts, then implement + GitHub, GitLab, Podman, Kubernetes, and external deployment systems behind them. +2. **Fail closed at trust boundaries.** Missing agent artifacts, failed execution, + ambiguous events, or unavailable policy checks must not silently advance a workflow. +3. **Humans govern consequential changes.** Approval requirements are explicit policy, + not prompt conventions. +4. **Traceability survives every backend.** Ticket, change request, commit, CI run, + environment, and deployment must share stable Forge correlation IDs. +5. **Credentials never enter agent context.** Drivers and adapters obtain short-lived, + least-privilege credentials outside the model boundary. +6. **Progressive adoption.** Local Podman and GitHub remain simple; enterprise features + are additive. +7. **No built-in cloud provisioner.** Forge coordinates deployment systems; it does not + manage raw cloud infrastructure state. + +## Roadmap themes + +### 1. Reliable orchestration foundation + +This is the prerequisite for every expansion. Provider and runtime abstractions will +multiply failure modes, so Forge must first make node outcomes and side effects explicit. + +**Deliverables** + +- Versioned, typed artifact envelope for agent handoffs with outcomes such as + `actionable`, `no_action`, `needs_input`, and `failed`; retain Markdown as a view. +- Fail-closed routing for missing, malformed, or incompatible artifacts. +- Explicit execution result contract; container or pod failure cannot lead to PR/MR + creation. +- Idempotent side-effect journal for comments, branches, change requests, deployments, + and teardown operations. +- Stable correlation index across ticket, repository, branch, PR/MR, CI run, and + environment; remove title parsing as an identity mechanism. +- Terminal-failure notifications, heartbeat/status updates, retry ownership, and + actionable transcript error summaries. +- Secret redaction and prompt-injection scanning at repository and output boundaries. +- Short-lived credential support, beginning with Vertex AI OIDC and extending the same + credential-broker pattern to source and deployment providers. +- Contract, replay, failure-injection, and workflow migration tests. + +**Current backlog incorporated:** +[artifact contracts #150](https://github.com/forge-sdlc/forge/issues/150), +[execution failure routing #147](https://github.com/forge-sdlc/forge/issues/147), +[terminal retry notification #140](https://github.com/forge-sdlc/forge/issues/140), +[container transcript errors #79](https://github.com/forge-sdlc/forge/issues/79), +[heartbeats #78](https://github.com/forge-sdlc/forge/issues/78), +[secret redaction #77](https://github.com/forge-sdlc/forge/issues/77), +[prompt-injection scanning #76](https://github.com/forge-sdlc/forge/issues/76), and +[short-lived credentials #82](https://github.com/forge-sdlc/forge/issues/82). + +**Exit criteria** + +- All workflow transitions consume validated typed outcomes. +- Replaying any supported webhook or worker message does not duplicate a side effect. +- Every terminal failure is visible to the user and carries a correlation ID. +- Failure-injection tests prove that unsuccessful execution cannot create a PR/MR. + +### 2. Source control provider platform + +“Multiple git sources” has two dimensions: a project may span repositories, and each +repository may live on a different provider or provider instance. Forge already supports +the first for GitHub; this theme adds the second. + +**Repository and workflow-state model** + +Use canonical repository identities rather than `owner/repo` strings. The provider is a +property of each repository reference, never of the workflow as a whole: + +```yaml +id: payments-api +provider: github # github | gitlab +connection: public-github # configured Forge connection +namespace: acme/payments +default_branch: main +change_request_mode: fork # fork | branch +``` + +A connection holds base/API URLs, webhook verification settings, credential reference, +TLS/CA configuration, and allowed namespaces. This is essential for multiple internal +GitLab deployments and prevents credentials from being embedded in Jira metadata. + +Do not extend the current `current_repo`, `current_pr_number`, and `pr_urls` fields with a +single workflow-level provider. That shape cannot safely represent PR #42 in GitHub and +MR !42 in GitLab, provider-specific CI state, or concurrent events from several +repositories. Replace it with a map of independently progressing repository work items: + +```yaml +repository_changes: + payments-api: # stable Forge repository ID + repository: + provider: github + connection: public-github + namespace: acme/payments + source_revision: main@abc123 + branch: forge/PROJ-123 + change_request: + native_id: "42" # opaque string; never globally unique + url: https://github.com/acme/payments/pull/42 + state: open + checks: + state: passed + runs: [] + review: + state: approved + execution: + state: completed + deployment-config: + repository: + provider: gitlab + connection: corp-gitlab + namespace: platform/deployment-config + source_revision: main@def456 + branch: forge/PROJ-123 + change_request: + native_id: "42" + url: https://gitlab.corp/platform/deployment-config/-/merge_requests/42 + state: open + checks: + state: running + runs: [] + review: + state: pending + execution: + state: completed +``` + +The durable external identity of a change request is the composite +`(connection, repository_id, native_id)`. URLs are presentation data, and native numbers +are opaque provider-local identifiers. A `current_work_item` may exist as a scheduling +cursor, but it must not be the source of truth for event routing or completion. + +Workflow-level status is a derived aggregate over `repository_changes`, using explicit +policy such as `all_required_changes_pass_ci`, `all_required_changes_approved`, and +`allow_partial_completion`. Each work item keeps its own execution, push, change-request, +CI, review, retry, and error state. This permits repositories to progress concurrently +and prevents a GitLab event from overwriting the active GitHub state. + +Cross-repository ordering must also be explicit. Work items may declare dependencies—for +example, deploy configuration waits for an application image digest—rather than relying +on list position. Outputs passed between repositories use typed artifacts and immutable +revisions, not mutable branch names. + +**Provider contract** + +- Repository discovery, clone/fetch URL, and default branch. +- Branch/fork creation and push authorization. +- Pull request / merge request create, update, comment, review, merge status, and close. +- CI status normalization, logs/artifacts lookup, and retry/cancel capabilities. +- Webhook verification, normalized events, actor identity, and delivery deduplication. +- Event routing by the composite change-request identity into exactly one repository work + item, followed by recomputation of aggregate workflow gates. +- Capability discovery so workflows can degrade deliberately when a provider lacks a + feature. + +**Delivery slices** + +1. Extract the existing GitHub implementation behind `SourceControlProvider` and prove + no behavioral regression. +2. Replace GitHub-specific workflow state and language with provider-neutral repository, + per-repository change request, review, and check-run models; migrate existing + checkpoints from the single-current-PR shape. +3. Add GitLab.com support for branch-based merge requests, pipelines/jobs, discussions, + approvals, and system hooks/project webhooks. +4. Add self-managed GitLab connections: arbitrary base URL, private CA bundle, proxy, + version/capability probing, group/project tokens, OAuth/service accounts, and multiple + simultaneous instances. +5. Support a single Forge workflow spanning GitHub and GitLab repositories, with + independent change requests but one aggregate approval and completion view. + +**Exit criteria** + +- The same provider contract suite passes for GitHub, GitLab.com, and a supported + self-managed GitLab version range. +- One test feature produces coordinated GitHub PR and GitLab MR changes. +- Interleaved and replayed GitHub/GitLab events update only their addressed repository + work item and produce the correct aggregate gate state. +- No workflow node imports a concrete source-control client. +- Connection credentials and private CA material never enter agent prompts or logs. + +### 3. Pluggable execution and Kubernetes support + +Kubernetes support must cover two separate user needs: deploying the Forge control plane +to Kubernetes and running isolated agent jobs on Kubernetes. They should be deliverable +independently. + +**Execution driver contract** + +- Submit an immutable execution specification: image digest, command, workspace, resource + limits, deadline, network policy profile, secrets references, and correlation labels. +- Observe status and heartbeats, stream bounded logs, cancel, collect typed results and + artifacts, and clean up idempotently. +- Drivers: existing local Podman, then Kubernetes Job; future drivers can include remote + container services without changing workflow nodes. + +**Kubernetes agent execution** + +- Kubernetes Jobs with per-run ServiceAccounts, security contexts, quotas, deadlines, + and TTL cleanup. +- Workspace transport via object storage or purpose-built PVCs; do not assume a shared + host filesystem. +- Default-deny network policies with explicit egress profiles. +- External Secrets / workload identity integration instead of environment-secret copies. +- Log and artifact size limits, cancellation, orphan reconciliation, and namespace-level + concurrency quotas. +- Compatibility with vanilla Kubernetes and OpenShift restricted security profiles. + +**Forge control-plane deployment** + +- Versioned OCI images and Helm chart for API, worker, Redis dependency/external Redis, + Service, Ingress/Route, probes, PodDisruptionBudget, autoscaling, and metrics. +- Database/checkpoint migrations and documented upgrade/rollback policy. +- HA worker semantics, graceful shutdown, queue draining, backups, and disaster recovery. +- Production security guide and reference values for OpenShift. + +This theme implements the intent of +[pluggable sandbox drivers #30](https://github.com/forge-sdlc/forge/issues/30). + +**Exit criteria** + +- Podman and Kubernetes drivers pass one conformance suite. +- A Forge deployment survives worker replacement without losing or duplicating work. +- An OpenShift restricted-profile installation completes an end-to-end workflow. +- Orphaned jobs and workspaces are reconciled after control-plane restart. + +### 4. Workflow prototyping and evolution + +Prototyping should let maintainers design and evaluate workflows safely without forking +the orchestrator or exposing real projects to experimental routing. Skills customize node +behavior; prototypes must also be able to change graph structure. + +**Deliverables** + +- Versioned workflow definition and registry with typed inputs, outputs, gates, retry + policies, permissions, and capability requirements. +- Workflow scaffold CLI and validation/lint command. +- Visual graph rendering plus a step-by-step simulator using fixture events and recorded + adapter responses. +- `dry-run` mode: agents may generate artifacts, but external writes are captured as an + inspectable side-effect plan. +- `shadow` mode: run a candidate workflow against copied/sanitized events without writes + and compare decisions, cost, latency, and artifacts to the active version. +- Project-level pinning, canary rollout, immutable workflow version per in-flight run, + checkpoint migration rules, and one-click rollback for new runs. +- Evaluation datasets and scorecards for artifact quality, approval revisions, CI + first-pass rate, completion rate, cost, and time. +- A stable extension API only after two internal workflow prototypes prove the contract. + +**Exit criteria** + +- A new experimental workflow can be scaffolded and simulated without editing worker + routing code. +- Dry-run mode performs zero external mutations, verified by adapter contract tests. +- In-flight workflows remain on their original version during a rollout. +- A candidate version can be promoted or rolled back using measured evaluation results. + +### 5. External deployment and ephemeral environments + +[Issue #28](https://github.com/forge-sdlc/forge/issues/28) should be refined into a +generic lifecycle-hook and deployment-plugin capability. Deployment controllers, +GitOps systems, and infrastructure-automation services integrate through the same +contract; none are dependencies of Forge core. + +**Refined scope** + +Forge owns the decision and lifecycle record; the plugin owns infrastructure operations. +The first use case is a preview/demo environment built from unmerged change requests. +Conversational ticket intake is a separate upstream integration and is not required for +the deployment MVP. + +**Environment record** + +- Stable environment ID, owner, ticket and PR/MR references. +- Requested template, immutable source revisions/image digests, parameters, and policy. +- Provider operation ID, lifecycle state, timestamps, TTL, cost/size classification. +- Non-secret outputs such as URLs; credentials are delivered through a secret broker or + one-time access mechanism, never Jira/PR comments. +- Teardown reason, status, retries, and audit history. + +**Plugin contract** + +- `validate(request)`, `provision(request, idempotency_key)`, `status(operation_id)`, + `outputs(operation_id)`, and `destroy(operation_id, idempotency_key)`. +- Signed/authenticated callbacks plus polling fallback. +- Capability declaration, health check, timeouts, retry classification, and redacted + errors. +- Hooks initially available after change-request creation, after required CI passes, and + on close/merge/ticket completion/TTL expiration. Policy selects which hooks are active. + +**Safety policy** + +- Allowlisted project templates, parameter schemas, quotas, maximum TTL, concurrency and + budget limits, approved target connections, and optional human deployment approval. +- Unique Forge environment IDs are passed to plugins; plugins remain responsible for + provider-specific naming and collision handling. +- A durable reconciler performs teardown. Webhook-only teardown is insufficient because + events can be missed and Forge can be offline at expiry. +- Provisioning failure must not mutate source history or weaken CI gates. +- Deployment success does not imply production release approval. + +**Delivery slices** + +1. Proposal and threat model; environment state machine and lifecycle hook contract. +2. No-op/reference plugin and conformance suite; dry-run and manual trigger. +3. Generic job-orchestration plugin for invoking allowlisted automation templates. +4. Kubernetes/GitOps plugin for applying approved, parameterized environment templates. +5. Automatic TTL and PR/MR/ticket teardown reconciliation; access-output delivery. +6. Multiple change requests, refresh/redeploy, quotas, cost reporting, and plugin SDK. + +**Exit criteria** + +- Killing Forge during provision or teardown converges to the correct state after restart. +- Duplicate hooks cannot create duplicate environments. +- Expired environments are detected and destroyed within the defined SLO. +- No infrastructure or access credentials are exposed to agents or ticket/change-request + comments. + +### 6. Human experience, quality, and economics + +Platform breadth is only valuable if users can understand and govern it. + +**Deliverables** + +- Concise Jira progress updates and a provider-neutral aggregate view across all + repositories and environments. +- Review split artifacts on their parent before creating child tickets; revisions update + stable items rather than deleting and recreating them. +- Concurrent CI observation and human review where policy allows, while merge readiness + still requires both. +- Better Langfuse span names and end-to-end workflow statistics: duration, revisions, + model/token cost, first-pass CI, failure class, and environment lifetime. +- Layered prompt efficiency, context budgets, caching, and per-node model policy. +- Pre-change-request validation defined by project policy/skills. + +**Current backlog incorporated:** +[revision identity #91](https://github.com/forge-sdlc/forge/issues/91), +[parent-first review #84](https://github.com/forge-sdlc/forge/issues/84), +[concurrent CI/review #137](https://github.com/forge-sdlc/forge/issues/137), +[Langfuse labels #138](https://github.com/forge-sdlc/forge/issues/138), and +[prompt efficiency #39](https://github.com/forge-sdlc/forge/issues/39), plus the existing +workflow-status and statistics proposals. + +**Exit criteria** + +- Users can identify current stage, owner, next action, and every related PR/MR/environment + from the parent ticket. +- Every major stage reports latency, cost, revision count, and outcome. +- Baseline and post-change evaluation show cost improvements without lower completion or + quality scores. + +## Recommended sequence + +The themes overlap, but their enabling order should be explicit. + +| Horizon | Primary outcome | Included work | +| --- | --- | --- | +| **Now: Trust the core** | Forge never advances ambiguously and users can diagnose failures | Typed artifacts, execution failure semantics, correlation/indexing, idempotency, terminal notifications, redaction/injection defenses, status/telemetry | +| **Next: Create extension seams** | Current behavior runs through stable abstractions | Source-control provider contract with GitHub adapter; execution driver contract with Podman adapter; versioned workflow definitions; lifecycle hook proposal | +| **Then: Add enterprise backends** | Forge works in heterogeneous enterprise environments | GitLab.com and self-managed GitLab; Kubernetes Job driver; Helm/OpenShift deployment; short-lived credentials/private CA support | +| **Then: Safely experiment and deploy** | Teams can evaluate new workflows and create governed preview environments | Simulator/dry-run/shadow/canary; deployment plugin runtime; generic automation-template plugin; TTL reconciler | +| **Later: Broaden the ecosystem** | External contributors can extend Forge without core changes | GitOps and deployment-controller plugins, supported SDKs, additional ticket/source/execution/deployment adapters, workflow template catalog, organization policy and portfolio analytics | + +Do not start all integrations simultaneously. A useful vertical-slice order is GitHub +through the new source contract, Podman through the new execution contract, then one +GitLab instance and one Kubernetes Job. Each abstraction should be proven by at least two +implementations before being declared stable. + +## Cross-cutting architecture decisions + +These decisions should be captured as proposals/ADRs before implementation: + +1. **Configuration ownership:** move from GitHub-shaped Jira properties to project + configuration referencing centrally managed provider connections. +2. **Plugin boundary:** begin with in-process Python interfaces where trusted; use a + versioned HTTP/event contract for deployment plugins that handle higher privilege or + independent scaling. Do not load arbitrary plugin code into the worker. +3. **State durability:** define which state belongs in LangGraph checkpoints versus a + queryable operational store for correlation, idempotency, and environment lifecycle. +4. **Workflow compatibility:** define immutable workflow versions and checkpoint + migrations before user-authored graph definitions. +5. **Identity and authorization:** map Jira, GitHub, GitLab, and Forge service identities + into an auditable actor model with project policy enforcement. +6. **Support matrix:** publish tested GitLab, Kubernetes, OpenShift, Redis, and plugin API + versions with deprecation policy. + +## Measures of success + +Track these by project, provider, workflow version, and execution driver: + +- Workflow completion and human-escalation rates. +- Incorrect-advance rate after failed/malformed agent or execution output (target: zero). +- Duplicate external side effects under event replay (target: zero). +- Median and p95 lead time by workflow stage; time waiting for humans is separate. +- Artifact revision count and CI first-pass rate. +- Cost per completed change and model tokens by stage. +- PR/MR review turnaround and change failure/reopen rate. +- Execution queue time, success rate, orphan rate, and cleanup latency. +- Preview-environment provision time, success rate, TTL compliance, and leaked resources. +- Provider/driver conformance pass rate and upgrade compatibility. + +## Questions for product discussion + +The roadmap can proceed with the direction above, but these answers change prioritization: + +1. Is Forge's primary next user a local/open-source team or an enterprise platform team? + This decides whether workflow prototyping or Kubernetes/GitLab lands first after the + contracts. +2. Does “Kubernetes support” primarily mean running Forge, running agent sandboxes, or + both? This roadmap recommends both as separately shippable tracks. +3. Must one Jira project mix GitHub and multiple GitLab instances in the same workflow, + or is one source provider per project an acceptable first milestone? +4. Who authors workflow prototypes: Forge maintainers, platform engineers, or ordinary + project users? The last group implies a much larger security and UX scope. +5. Should preview environments be created automatically after CI, or require an explicit + approval/command by default? The safer default is explicit approval with project-level + opt-in to automation. +6. Which external deployment model should prove the contract first: invoking an + allowlisted automation job or reconciling a declarative environment resource? The + automation-job model is recommended first because its contract is narrower. +7. What is the desired extension model: built-in adapters maintained by Forge, separately + deployed plugins, or both with different trust levels? +8. Is Jira intended to remain the required system of record, or should issue-tracker + abstraction enter the longer-term roadmap? + +## Near-term proposal backlog + +Before implementation, create and review focused proposals in this order: + +1. Typed agent artifact and node-result contract. +2. Stable correlation identity and idempotent side-effect journal. +3. Source-control provider contract and repository connection model. +4. Execution driver contract and Kubernetes threat model. +5. Versioned workflow prototype, simulation, and rollout model. +6. Lifecycle hooks, environment state machine, and deployment plugin contract. +7. Enterprise identity, credentials, and policy model spanning all adapters. From c56d8b16d10c367b6df3748e31e27f9a04566477 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:29:22 +0300 Subject: [PATCH 02/18] docs: expand roadmap product decisions --- docs/roadmap.md | 181 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 140 insertions(+), 41 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 255ee651f..1ae7c626a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -2,7 +2,7 @@ **Status:** Draft for discussion **Planning horizon:** Outcome-based; dates and release assignments are intentionally TBD -**Last reviewed:** 2026-07-15 +**Last reviewed:** 2026-08-10 ## Product direction @@ -187,11 +187,13 @@ revisions, not mutable branch names. per-repository change request, review, and check-run models; migrate existing checkpoints from the single-current-PR shape. 3. Add GitLab.com support for branch-based merge requests, pipelines/jobs, discussions, - approvals, and system hooks/project webhooks. + approvals, and system hooks/project webhooks. The first supported GitLab release must + also support a workflow mixing GitHub and GitLab repositories; single-provider-only + project support is not an acceptable milestone. 4. Add self-managed GitLab connections: arbitrary base URL, private CA bundle, proxy, version/capability probing, group/project tokens, OAuth/service accounts, and multiple simultaneous instances. -5. Support a single Forge workflow spanning GitHub and GitLab repositories, with +5. Harden mixed-provider workflows across multiple simultaneous GitLab instances, with independent change requests but one aggregate approval and completion view. **Exit criteria** @@ -249,14 +251,33 @@ This theme implements the intent of - An OpenShift restricted-profile installation completes an end-to-end workflow. - Orphaned jobs and workspaces are reconciled after control-plane restart. -### 4. Workflow prototyping and evolution +### 4. Product prototyping and workflow evolution -Prototyping should let maintainers design and evaluate workflows safely without forking -the orchestrator or exposing real projects to experimental routing. Skills customize node -behavior; prototypes must also be able to change graph structure. +Product prototyping is a discovery workflow for Forge users. Starting from an initial +PRD or feature idea, a user can ask Forge to build one or more competing prototypes, +interact with and revise them, compare their behavior, and feed what was learned back +into the PRD and product plan before committing to a production implementation. This is +distinct from prototyping Forge's own workflow graph, which remains a platform-engineering +capability. **Deliverables** +- A PRD-to-prototype discovery workflow that turns explicit uncertainties and hypotheses + into one or more time-boxed prototype options with comparable goals and evaluation + criteria. +- Isolated, disposable prototype workspaces and optional preview environments where users + can exercise behavior, provide feedback, and request revisions without creating a + production PR/MR or representing the prototype as production-ready code. +- Side-by-side comparison of alternative prototypes using user feedback, behavior, + feasibility, architecture implications, risks, cost, and measured results—not only an + agent preference. +- A governed learning step that proposes concrete PRD and plan updates, records which + prototype evidence supports each change, and requires user approval before modifying + the canonical artifacts. +- An explicit transition from discovery to delivery: discard all options, continue + prototyping, or select an option and generate a coherent implementation plan. Reuse + validated decisions and evidence, but regenerate production-quality implementation + rather than silently promoting disposable prototype code. - Versioned workflow definition and registry with typed inputs, outputs, gates, retry policies, permissions, and capability requirements. - Workflow scaffold CLI and validation/lint command. @@ -274,6 +295,12 @@ behavior; prototypes must also be able to change graph structure. **Exit criteria** +- Given one PRD with an unresolved product or implementation choice, a user can create, + revise, and compare at least two working prototype options before selecting either. +- Approved learnings update the PRD and implementation plan with traceable prototype and + user-feedback evidence; rejected learnings leave the canonical artifacts unchanged. +- Prototype code and environments are clearly marked disposable, isolated from production + delivery, and cleaned up according to policy. - A new experimental workflow can be scaffolded and simulated without editing worker routing code. - Dry-run mode performs zero external mutations, verified by adapter contract tests. @@ -292,7 +319,8 @@ contract; none are dependencies of Forge core. Forge owns the decision and lifecycle record; the plugin owns infrastructure operations. The first use case is a preview/demo environment built from unmerged change requests. Conversational ticket intake is a separate upstream integration and is not required for -the deployment MVP. +the deployment MVP. Creation requires explicit approval or command by default; projects +may opt into automatic creation after CI through policy. **Environment record** @@ -341,7 +369,64 @@ the deployment MVP. - No infrastructure or access credentials are exposed to agents or ticket/change-request comments. -### 6. Human experience, quality, and economics +### 6. Security remediation and verification debt + +Security and testing are workflow outcomes, not only implementation-stage tools. Forge +should support both urgent vulnerability remediation and systematic improvement of +existing code whose behavior is insufficiently verified. + +**Security deliverables** + +- CVE remediation workflow: ingest a vulnerability advisory or scanner finding, resolve + affected repositories and dependency paths, assess exploitability and priority, + propose the smallest safe upgrade or mitigation, generate regression tests, and retain + advisory-to-commit evidence. +- Policy-controlled security stages in implementation workflows: dependency, secret, + static-analysis, and generated-code weakness scans before publication and again through + repository-owned CI. Normalize findings into typed artifacts with severity, + confidence, location, remediation, and suppress/accept-risk decisions. +- Fail closed for findings above project policy thresholds; require an auditable human + exception for accepted risk. Scanner outage or malformed output must not be treated as + a pass. +- Harden Forge itself: sandbox capability contracts, default-deny execution profiles, + pre-push validation, structured execution security evidence, credential isolation, + secret scanning, and prompt-injection defenses. + +**Verification-debt deliverables** + +- A dedicated test workflow for existing features and code: discover unverified behavior + from requirements, incidents, change history, coverage/mutation reports, and code risk; + prioritize verification debt; generate focused tests without requiring a feature + implementation; and open reviewable PRs/MRs with traceability to the behavior covered. +- Improve test planning in the normal planning and implementation workflows with explicit + behavior inventories, risk-based test matrices, negative/boundary/concurrency cases, + and a clear split between deterministic repository validation and model-reviewed + evidence. +- Measure meaningful verification gains with behavior/risk coverage, mutation score, + escaped-defect history, and flaky-test impact rather than line coverage alone. + +**Current security backlog incorporated:** +[sandbox capability requirements #265](https://github.com/forge-sdlc/forge/issues/265), +[sandbox hardening #266](https://github.com/forge-sdlc/forge/issues/266), +[structured security evidence #264](https://github.com/forge-sdlc/forge/issues/264), +[fail-closed pre-push validation #263](https://github.com/forge-sdlc/forge/issues/263), +[OpenShell boundary spike #262](https://github.com/forge-sdlc/forge/issues/262), +[credential isolation #82](https://github.com/forge-sdlc/forge/issues/82), +[secret scanning #77](https://github.com/forge-sdlc/forge/issues/77), and +[prompt-injection auditing #76](https://github.com/forge-sdlc/forge/issues/76). + +**Exit criteria** + +- A CVE advisory can produce a policy-compliant remediation PR/MR with linked scan and + regression-test evidence. +- Generated changes cannot be published when a required security stage fails, is + unavailable, or returns invalid output. +- A verification-debt run can add tests to existing code and report the behaviors and + risks newly covered without changing production behavior. +- Forge execution security controls and accepted exceptions are queryable by correlation + ID. + +### 7. Human experience, quality, and economics Platform breadth is only valuable if users can understand and govern it. @@ -355,7 +440,15 @@ Platform breadth is only valuable if users can understand and govern it. still requires both. - Better Langfuse span names and end-to-end workflow statistics: duration, revisions, model/token cost, first-pass CI, failure class, and environment lifetime. -- Layered prompt efficiency, context budgets, caching, and per-node model policy. +- Layered prompt efficiency, context budgets, caching, and per-stage model policy. +- An `auto` model option per stage that dynamically routes by task complexity, context, + required capabilities, latency/cost budget, and observed quality. Policies must support + allowlists, deterministic fallback, retry/escalation to a stronger model, and a pinned + model override for reproducibility. +- MLflow integration as an optional experiment/evaluation backend: record workflow and + stage parameters, model/provider identity, prompt/artifact versions, datasets, metrics, + costs, latency, and correlation IDs; compare routing and workflow variants without + making MLflow a runtime dependency or storing secrets/raw sensitive context by default. - Pre-change-request validation defined by project policy/skills. **Current backlog incorporated:** @@ -373,6 +466,10 @@ workflow-status and statistics proposals. - Every major stage reports latency, cost, revision count, and outcome. - Baseline and post-change evaluation show cost improvements without lower completion or quality scores. +- Automatic routing meets configured quality and latency floors while reducing cost + against pinned-model baselines, with every routing decision auditable. +- The same evaluation run can be inspected in Forge telemetry and, when configured, + MLflow using a shared correlation ID. ## Recommended sequence @@ -380,10 +477,10 @@ The themes overlap, but their enabling order should be explicit. | Horizon | Primary outcome | Included work | | --- | --- | --- | -| **Now: Trust the core** | Forge never advances ambiguously and users can diagnose failures | Typed artifacts, execution failure semantics, correlation/indexing, idempotency, terminal notifications, redaction/injection defenses, status/telemetry | -| **Next: Create extension seams** | Current behavior runs through stable abstractions | Source-control provider contract with GitHub adapter; execution driver contract with Podman adapter; versioned workflow definitions; lifecycle hook proposal | -| **Then: Add enterprise backends** | Forge works in heterogeneous enterprise environments | GitLab.com and self-managed GitLab; Kubernetes Job driver; Helm/OpenShift deployment; short-lived credentials/private CA support | -| **Then: Safely experiment and deploy** | Teams can evaluate new workflows and create governed preview environments | Simulator/dry-run/shadow/canary; deployment plugin runtime; generic automation-template plugin; TTL reconciler | +| **Now: Trust the core** | Forge never advances ambiguously and users can diagnose failures | Typed artifacts, execution failure semantics, correlation/indexing, idempotency, terminal notifications, redaction/injection defenses, sandbox hardening, security evidence, status/telemetry | +| **Next: Create extension seams** | Current behavior runs through stable abstractions | Source-control and issue-tracker provider contracts with built-in adapters; execution driver contract with Podman adapter; versioned workflow definitions; lifecycle hook proposal | +| **Then: Add enterprise backends** | Enterprise platform teams can operate Forge in heterogeneous environments while open-source users retain a simple path | Mixed GitHub/GitLab workflows from the first GitLab milestone; GitLab.com and self-managed GitLab; Kubernetes agent Jobs; Helm/OpenShift deployment; short-lived credentials/private CA support | +| **Then: Discover, experiment, and deploy** | Teams can learn through working product options, evaluate workflows, and create governed preview environments | PRD-driven competing prototypes and feedback into planning; simulator/dry-run/shadow/canary; security and verification-debt workflows; dynamic model routing and MLflow evaluation; deployment plugin runtime; allowlisted automation-job plugin; TTL reconciler | | **Later: Broaden the ecosystem** | External contributors can extend Forge without core changes | GitOps and deployment-controller plugins, supported SDKs, additional ticket/source/execution/deployment adapters, workflow template catalog, organization policy and portfolio analytics | Do not start all integrations simultaneously. A useful vertical-slice order is GitHub @@ -397,9 +494,11 @@ These decisions should be captured as proposals/ADRs before implementation: 1. **Configuration ownership:** move from GitHub-shaped Jira properties to project configuration referencing centrally managed provider connections. -2. **Plugin boundary:** begin with in-process Python interfaces where trusted; use a - versioned HTTP/event contract for deployment plugins that handle higher privilege or - independent scaling. Do not load arbitrary plugin code into the worker. +2. **Plugin boundary:** support both Forge-maintained built-in adapters and separately + deployed external plugins behind the same versioned interfaces and conformance suites. + Begin with in-process Python interfaces for trusted built-ins; use a versioned + HTTP/event contract for external or higher-privilege plugins. Do not load arbitrary + plugin code into the worker. 3. **State durability:** define which state belongs in LangGraph checkpoints versus a queryable operational store for correlation, idempotency, and environment lifecycle. 4. **Workflow compatibility:** define immutable workflow versions and checkpoint @@ -424,29 +523,26 @@ Track these by project, provider, workflow version, and execution driver: - Preview-environment provision time, success rate, TTL compliance, and leaked resources. - Provider/driver conformance pass rate and upgrade compatibility. -## Questions for product discussion - -The roadmap can proceed with the direction above, but these answers change prioritization: - -1. Is Forge's primary next user a local/open-source team or an enterprise platform team? - This decides whether workflow prototyping or Kubernetes/GitLab lands first after the - contracts. -2. Does “Kubernetes support” primarily mean running Forge, running agent sandboxes, or - both? This roadmap recommends both as separately shippable tracks. -3. Must one Jira project mix GitHub and multiple GitLab instances in the same workflow, - or is one source provider per project an acceptable first milestone? -4. Who authors workflow prototypes: Forge maintainers, platform engineers, or ordinary - project users? The last group implies a much larger security and UX scope. -5. Should preview environments be created automatically after CI, or require an explicit - approval/command by default? The safer default is explicit approval with project-level - opt-in to automation. -6. Which external deployment model should prove the contract first: invoking an - allowlisted automation job or reconciling a declarative environment resource? The - automation-job model is recommended first because its contract is narrower. -7. What is the desired extension model: built-in adapters maintained by Forge, separately - deployed plugins, or both with different trust levels? -8. Is Jira intended to remain the required system of record, or should issue-tracker - abstraction enter the longer-term roadmap? +## Product decisions + +1. **Primary audience:** prioritize enterprise platform teams while preserving a simple, + supported adoption path for local and open-source users. +2. **Kubernetes scope:** support both running the Forge control plane and running isolated + agent sandboxes, as separately shippable tracks. +3. **Mixed source providers:** a Jira project must be able to mix GitHub repositories and + repositories from multiple GitLab instances from the first supported GitLab milestone. +4. **Product prototyping:** Forge users start from an initial PRD, explore and revise one + or more working options, compare the results, and feed approved learning into a + coherent PRD and plan before production implementation. +5. **Preview environments:** require explicit approval or command by default, with + project-policy opt-in for automatic creation. +6. **First deployment integration:** prove the external deployment contract by invoking + an allowlisted automation job before adding declarative environment reconciliation. +7. **Extension model:** provide Forge-maintained built-ins and interfaces/conformance + suites for separately deployed external plugins, with trust-appropriate isolation. +8. **Issue tracking:** abstract Jira behind an issue-tracker interface. Ship Jira as the + built-in default while allowing external issue-tracker plugins; Jira is not a permanent + architectural requirement. ## Near-term proposal backlog @@ -456,6 +552,9 @@ Before implementation, create and review focused proposals in this order: 2. Stable correlation identity and idempotent side-effect journal. 3. Source-control provider contract and repository connection model. 4. Execution driver contract and Kubernetes threat model. -5. Versioned workflow prototype, simulation, and rollout model. +5. PRD-driven product-prototyping workflow and versioned workflow simulation/rollout model. 6. Lifecycle hooks, environment state machine, and deployment plugin contract. 7. Enterprise identity, credentials, and policy model spanning all adapters. +8. Security remediation, generated-code scanning, and accepted-risk policy. +9. Verification-debt workflow and risk-based test-planning contract. +10. Dynamic model-routing policy and MLflow evaluation integration. From c51ccc46e722523c7501d7b6001715aba2630e5a Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:33:35 +0300 Subject: [PATCH 03/18] docs: sharpen Forge product positioning --- docs/roadmap.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 1ae7c626a..c10e34dff 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -6,11 +6,11 @@ ## Product direction -Forge should grow from a Jira-to-GitHub automation into an extensible, governed SDLC -orchestrator: one control plane for turning product intent into validated changes and, -when policy permits, temporary environments. It should coordinate systems that already -own source control, execution, CI, and deployment rather than becoming a replacement -for them. +Forge is an extensible, governed SDLC orchestrator: a durable control plane for turning +product intent into traceable artifacts, validated changes, and, when policy permits, +temporary environments. It coordinates long-running agent workflows across issue +tracking, source control, execution, CI, review, and deployment while preserving those +systems as the authoritative owners of their respective operations. The durable product boundary is: From 9da4a953c5e8e4178a367721851cc88c0f46d80d Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:35:09 +0300 Subject: [PATCH 04/18] docs: distinguish current and planned capabilities --- docs/roadmap.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index c10e34dff..44a299ca7 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -6,11 +6,11 @@ ## Product direction -Forge is an extensible, governed SDLC orchestrator: a durable control plane for turning -product intent into traceable artifacts, validated changes, and, when policy permits, -temporary environments. It coordinates long-running agent workflows across issue -tracking, source control, execution, CI, review, and deployment while preserving those -systems as the authoritative owners of their respective operations. +Forge is a governed SDLC orchestrator for turning product intent into traceable artifacts +and validated changes through long-running agent workflows. This roadmap evolves Forge +into an extensible control plane spanning issue tracking, source control, execution, CI, +review, and deployment, including policy-governed temporary environments. These systems +remain the authoritative owners of their respective operations. The durable product boundary is: From b45bbed97cb8199f8c9e5b9a52cc72692de24720 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:39:27 +0300 Subject: [PATCH 05/18] docs: clarify adoption and infrastructure principles --- docs/roadmap.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 44a299ca7..bacb9212e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -35,10 +35,17 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent environment, and deployment must share stable Forge correlation IDs. 5. **Credentials never enter agent context.** Drivers and adapters obtain short-lived, least-privilege credentials outside the model boundary. -6. **Progressive adoption.** Local Podman and GitHub remain simple; enterprise features - are additive. -7. **No built-in cloud provisioner.** Forge coordinates deployment systems; it does not - manage raw cloud infrastructure state. +6. **Enterprise-grade without enterprise lock-in.** Prioritize the governance, + isolation, identity, and heterogeneous-provider needs of platform teams while keeping + a complete local and open-source path with sensible defaults. Advanced backends and + policies extend the core workflow; they must not make GitHub and local Podman users + operate enterprise infrastructure to adopt Forge. +7. **Orchestrate infrastructure; do not become its control plane.** Forge owns deployment + intent, policy gates, lifecycle records, correlation, and reconciliation. External + deployment plugins own provider-specific provisioning and teardown, and the target + platform remains authoritative for infrastructure state. This boundary lets Forge + govern environments without embedding cloud credentials, resource schemas, or a + general-purpose cloud provisioner in the core. ## Roadmap themes From 47f35d1a7de04a9e29802cc80f2bd7dd5cffa4c8 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:42:11 +0300 Subject: [PATCH 06/18] docs: clarify Forge central deployment model --- docs/roadmap.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index bacb9212e..673becb22 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -35,11 +35,11 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent environment, and deployment must share stable Forge correlation IDs. 5. **Credentials never enter agent context.** Drivers and adapters obtain short-lived, least-privilege credentials outside the model boundary. -6. **Enterprise-grade without enterprise lock-in.** Prioritize the governance, - isolation, identity, and heterogeneous-provider needs of platform teams while keeping - a complete local and open-source path with sensible defaults. Advanced backends and - policies extend the core workflow; they must not make GitHub and local Podman users - operate enterprise infrastructure to adopt Forge. +6. **A central platform with an open ecosystem.** Forge is designed to run as a centrally + operated service that gives platform teams consistent governance, identity, + isolation, and orchestration across projects and providers. Its open-source model + should enable organizations to self-host, inspect, extend, and contribute to that + platform without redefining Forge as a local developer tool. 7. **Orchestrate infrastructure; do not become its control plane.** Forge owns deployment intent, policy gates, lifecycle records, correlation, and reconciliation. External deployment plugins own provider-specific provisioning and teardown, and the target @@ -486,7 +486,7 @@ The themes overlap, but their enabling order should be explicit. | --- | --- | --- | | **Now: Trust the core** | Forge never advances ambiguously and users can diagnose failures | Typed artifacts, execution failure semantics, correlation/indexing, idempotency, terminal notifications, redaction/injection defenses, sandbox hardening, security evidence, status/telemetry | | **Next: Create extension seams** | Current behavior runs through stable abstractions | Source-control and issue-tracker provider contracts with built-in adapters; execution driver contract with Podman adapter; versioned workflow definitions; lifecycle hook proposal | -| **Then: Add enterprise backends** | Enterprise platform teams can operate Forge in heterogeneous environments while open-source users retain a simple path | Mixed GitHub/GitLab workflows from the first GitLab milestone; GitLab.com and self-managed GitLab; Kubernetes agent Jobs; Helm/OpenShift deployment; short-lived credentials/private CA support | +| **Then: Add enterprise backends** | Platform teams can centrally operate Forge in heterogeneous environments using an open, self-hostable platform | Mixed GitHub/GitLab workflows from the first GitLab milestone; GitLab.com and self-managed GitLab; Kubernetes agent Jobs; Helm/OpenShift deployment; short-lived credentials/private CA support | | **Then: Discover, experiment, and deploy** | Teams can learn through working product options, evaluate workflows, and create governed preview environments | PRD-driven competing prototypes and feedback into planning; simulator/dry-run/shadow/canary; security and verification-debt workflows; dynamic model routing and MLflow evaluation; deployment plugin runtime; allowlisted automation-job plugin; TTL reconciler | | **Later: Broaden the ecosystem** | External contributors can extend Forge without core changes | GitOps and deployment-controller plugins, supported SDKs, additional ticket/source/execution/deployment adapters, workflow template catalog, organization policy and portfolio analytics | @@ -532,8 +532,9 @@ Track these by project, provider, workflow version, and execution driver: ## Product decisions -1. **Primary audience:** prioritize enterprise platform teams while preserving a simple, - supported adoption path for local and open-source users. +1. **Primary audience:** prioritize enterprise platform teams operating Forge as a + central service. Preserve open-source self-hosting, extensibility, and contribution; + local execution of Forge is not a primary product mode. 2. **Kubernetes scope:** support both running the Forge control plane and running isolated agent sandboxes, as separately shippable tracks. 3. **Mixed source providers:** a Jira project must be able to mix GitHub repositories and From 0986439a53accec7e2ffacdd6a265a13215a4c71 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:44:10 +0300 Subject: [PATCH 07/18] docs: strengthen product principles --- docs/roadmap.md | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 673becb22..cfe6d22d8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -25,16 +25,32 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent ## Product principles -1. **Contracts before integrations.** Define stable domain contracts, then implement - GitHub, GitLab, Podman, Kubernetes, and external deployment systems behind them. -2. **Fail closed at trust boundaries.** Missing agent artifacts, failed execution, - ambiguous events, or unavailable policy checks must not silently advance a workflow. -3. **Humans govern consequential changes.** Approval requirements are explicit policy, - not prompt conventions. -4. **Traceability survives every backend.** Ticket, change request, commit, CI run, - environment, and deployment must share stable Forge correlation IDs. -5. **Credentials never enter agent context.** Drivers and adapters obtain short-lived, - least-privilege credentials outside the model boundary. +1. **Contracts before integrations.** Forge models source control, issue tracking, + execution, artifacts, and deployment through stable domain contracts before adding + provider-specific behavior. GitHub, GitLab, Podman, Kubernetes, and external plugins + must prove those contracts through shared conformance suites, with capabilities and + limitations declared explicitly rather than leaking provider assumptions into the + workflow. +2. **Fail closed at trust boundaries.** Missing or malformed agent artifacts, failed + execution, ambiguous events, unavailable scanners, and unmet policy checks must stop + progression instead of being interpreted as success. Every stopped transition should + preserve diagnostic evidence and expose a deliberate retry, recovery, exception, or + human-escalation path. +3. **Humans govern consequential changes.** Forge may analyze, propose, prototype, and + automate within policy, but approvals for publication, accepted risk, deployment, and + other material effects are explicit, auditable workflow gates—not prompt conventions + or assumptions inferred by an agent. Projects may automate a gate only through a + visible policy decision with a defined scope. +4. **Traceability survives every backend.** Tickets, artifacts, prototype decisions, + repositories, commits, PRs/MRs, CI runs, environments, and deployments share stable + Forge correlation identities independent of provider-local names or URLs. A user or + operator must be able to reconstruct why a transition occurred and which inputs, + policies, actors, and outputs contributed to it. +5. **Credentials never enter agent context.** Adapters, execution drivers, and credential + brokers obtain short-lived, least-privilege access outside the model boundary and + expose only the narrowly scoped operation an agent needs. Secrets must not appear in + prompts, generated artifacts, workspace history, logs, or external comments, and + credential use remains attributable to a Forge workflow and policy decision. 6. **A central platform with an open ecosystem.** Forge is designed to run as a centrally operated service that gives platform teams consistent governance, identity, isolation, and orchestration across projects and providers. Its open-source model From 5f59b7f0638d823bd625c74282341afca6f161c3 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:50:30 +0300 Subject: [PATCH 08/18] docs: condense product principles --- docs/roadmap.md | 56 ++++++++++++++++++------------------------------- 1 file changed, 20 insertions(+), 36 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index cfe6d22d8..a5964de4d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -25,43 +25,27 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent ## Product principles -1. **Contracts before integrations.** Forge models source control, issue tracking, - execution, artifacts, and deployment through stable domain contracts before adding - provider-specific behavior. GitHub, GitLab, Podman, Kubernetes, and external plugins - must prove those contracts through shared conformance suites, with capabilities and - limitations declared explicitly rather than leaking provider assumptions into the - workflow. -2. **Fail closed at trust boundaries.** Missing or malformed agent artifacts, failed - execution, ambiguous events, unavailable scanners, and unmet policy checks must stop - progression instead of being interpreted as success. Every stopped transition should - preserve diagnostic evidence and expose a deliberate retry, recovery, exception, or - human-escalation path. -3. **Humans govern consequential changes.** Forge may analyze, propose, prototype, and - automate within policy, but approvals for publication, accepted risk, deployment, and - other material effects are explicit, auditable workflow gates—not prompt conventions - or assumptions inferred by an agent. Projects may automate a gate only through a - visible policy decision with a defined scope. -4. **Traceability survives every backend.** Tickets, artifacts, prototype decisions, - repositories, commits, PRs/MRs, CI runs, environments, and deployments share stable - Forge correlation identities independent of provider-local names or URLs. A user or - operator must be able to reconstruct why a transition occurred and which inputs, - policies, actors, and outputs contributed to it. -5. **Credentials never enter agent context.** Adapters, execution drivers, and credential - brokers obtain short-lived, least-privilege access outside the model boundary and - expose only the narrowly scoped operation an agent needs. Secrets must not appear in - prompts, generated artifacts, workspace history, logs, or external comments, and - credential use remains attributable to a Forge workflow and policy decision. -6. **A central platform with an open ecosystem.** Forge is designed to run as a centrally - operated service that gives platform teams consistent governance, identity, - isolation, and orchestration across projects and providers. Its open-source model - should enable organizations to self-host, inspect, extend, and contribute to that - platform without redefining Forge as a local developer tool. +1. **Contracts before integrations.** Stable domain contracts and conformance suites + isolate workflows from provider-specific behavior and declare backend capabilities + explicitly. +2. **Fail closed at trust boundaries.** Invalid artifacts, failed execution, ambiguous + events, or unavailable policy checks stop progression with evidence and a deliberate + recovery path. +3. **Humans govern consequential changes.** Publication, accepted risk, and deployment + require explicit, auditable policy gates; automation is enabled by scoped policy, not + inferred by an agent. +4. **Traceability survives every backend.** Stable Forge identities correlate artifacts, + actors, policies, repositories, changes, CI runs, environments, and deployments across + provider boundaries. +5. **Credentials never enter agent context.** Brokers provide short-lived, + least-privilege operations outside the model boundary, keeping secrets out of prompts, + artifacts, workspaces, logs, and comments. +6. **A central platform with an open ecosystem.** Forge is centrally operated for + consistent cross-project governance while remaining open source, self-hostable, + extensible, and open to contribution—not repositioned as a local developer tool. 7. **Orchestrate infrastructure; do not become its control plane.** Forge owns deployment - intent, policy gates, lifecycle records, correlation, and reconciliation. External - deployment plugins own provider-specific provisioning and teardown, and the target - platform remains authoritative for infrastructure state. This boundary lets Forge - govern environments without embedding cloud credentials, resource schemas, or a - general-purpose cloud provisioner in the core. + intent, policy, and lifecycle reconciliation; plugins and target platforms own + provisioning and infrastructure state. ## Roadmap themes From de6d1e53f44f371fa48330a3862afe596bb336eb Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 10:54:42 +0300 Subject: [PATCH 09/18] docs: clarify provider-neutral core principle --- docs/roadmap.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index a5964de4d..b0069592f 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -25,9 +25,9 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent ## Product principles -1. **Contracts before integrations.** Stable domain contracts and conformance suites - isolate workflows from provider-specific behavior and declare backend capabilities - explicitly. +1. **Provider-neutral core.** Workflow state and logic depend on versioned domain + interfaces, never a specific external system; every backend declares its capabilities + and proves the same contract through conformance tests. 2. **Fail closed at trust boundaries.** Invalid artifacts, failed execution, ambiguous events, or unavailable policy checks stop progression with evidence and a deliberate recovery path. From f1bb7768b4a2cd2cef47922513adeed570bbd7f9 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 11:10:49 +0300 Subject: [PATCH 10/18] docs: clarify orchestration foundation theme --- docs/roadmap.md | 47 +++++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index b0069592f..2a88dda65 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -49,28 +49,29 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent ## Roadmap themes -### 1. Reliable orchestration foundation +### 1. Deterministic and recoverable orchestration -This is the prerequisite for every expansion. Provider and runtime abstractions will -multiply failure modes, so Forge must first make node outcomes and side effects explicit. +Before adding providers, runtimes, or workflows, Forge must make every transition +validated, replay-safe, traceable, and recoverable. This foundation prevents ambiguous +agent or execution results from advancing a workflow and gives operators enough evidence +to understand and resume failures. **Deliverables** -- Versioned, typed artifact envelope for agent handoffs with outcomes such as - `actionable`, `no_action`, `needs_input`, and `failed`; retain Markdown as a view. -- Fail-closed routing for missing, malformed, or incompatible artifacts. -- Explicit execution result contract; container or pod failure cannot lead to PR/MR - creation. -- Idempotent side-effect journal for comments, branches, change requests, deployments, - and teardown operations. -- Stable correlation index across ticket, repository, branch, PR/MR, CI run, and - environment; remove title parsing as an identity mechanism. -- Terminal-failure notifications, heartbeat/status updates, retry ownership, and - actionable transcript error summaries. -- Secret redaction and prompt-injection scanning at repository and output boundaries. -- Short-lived credential support, beginning with Vertex AI OIDC and extending the same - credential-broker pattern to source and deployment providers. -- Contract, replay, failure-injection, and workflow migration tests. +- **Explicit outcomes:** versioned, typed node and execution results distinguish + `actionable`, `no_action`, `needs_input`, and `failed`; missing, malformed, or + incompatible results fail closed. Markdown remains a view, not the workflow contract. +- **Replay-safe effects:** an idempotent journal governs comments, branches, PRs/MRs, + deployments, and teardown so retries and duplicate events cannot repeat mutations. +- **Durable identity:** stable correlation IDs link tickets, repositories, revisions, + PRs/MRs, CI runs, and environments without relying on titles or URLs for routing. +- **Visible recovery:** heartbeats, status updates, transcript summaries, retry ownership, + and terminal notifications show what failed, who acts next, and how work can resume. +- **Baseline trust controls:** redact secrets, scan untrusted instructions and outputs, + and broker short-lived credentials outside agent context, beginning with Vertex AI + OIDC. +- **Proof under failure:** contract, replay, failure-injection, and migration tests verify + these guarantees across restarts and workflow upgrades. **Current backlog incorporated:** [artifact contracts #150](https://github.com/forge-sdlc/forge/issues/150), @@ -84,10 +85,12 @@ multiply failure modes, so Forge must first make node outcomes and side effects **Exit criteria** -- All workflow transitions consume validated typed outcomes. -- Replaying any supported webhook or worker message does not duplicate a side effect. -- Every terminal failure is visible to the user and carries a correlation ID. -- Failure-injection tests prove that unsuccessful execution cannot create a PR/MR. +- Every transition consumes a validated outcome; unsuccessful or unknown execution cannot + create a PR/MR or cross another consequential gate. +- Replaying supported events or recovering after restart produces no duplicate effect and + preserves correlation history. +- Every stalled or terminal workflow exposes its failure, owner, evidence, and supported + recovery action. ### 2. Source control provider platform From bbce7df2735e06cbc37bd8227fb59a34ac12a1c0 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 11:16:06 +0300 Subject: [PATCH 11/18] docs: frame orchestration work as enabling direction --- docs/roadmap.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 2a88dda65..91de89795 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -51,10 +51,10 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent ### 1. Deterministic and recoverable orchestration -Before adding providers, runtimes, or workflows, Forge must make every transition -validated, replay-safe, traceable, and recoverable. This foundation prevents ambiguous -agent or execution results from advancing a workflow and gives operators enough evidence -to understand and resume failures. +Forge is moving toward transitions that are validated, replay-safe, traceable, and +recoverable. These capabilities should be delivered incrementally and adopted by each +new provider, runtime, and workflow, enabling expansion without multiplying ambiguous +state or unrecoverable failures. **Deliverables** From a4cd844099cc45883c1ae88f3c8260c13255082a Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 11:34:38 +0300 Subject: [PATCH 12/18] docs: map roadmap themes to repository tracking --- docs/roadmap.md | 116 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 91 insertions(+), 25 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 91de89795..e2e167e5c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -73,15 +73,22 @@ state or unrecoverable failures. - **Proof under failure:** contract, replay, failure-injection, and migration tests verify these guarantees across restarts and workflow upgrades. -**Current backlog incorporated:** -[artifact contracts #150](https://github.com/forge-sdlc/forge/issues/150), -[execution failure routing #147](https://github.com/forge-sdlc/forge/issues/147), -[terminal retry notification #140](https://github.com/forge-sdlc/forge/issues/140), -[container transcript errors #79](https://github.com/forge-sdlc/forge/issues/79), -[heartbeats #78](https://github.com/forge-sdlc/forge/issues/78), -[secret redaction #77](https://github.com/forge-sdlc/forge/issues/77), -[prompt-injection scanning #76](https://github.com/forge-sdlc/forge/issues/76), and -[short-lived credentials #82](https://github.com/forge-sdlc/forge/issues/82). +**Related tracking (reviewed 2026-08-10)** + +- Open issues: [typed artifact contracts #150](https://github.com/forge-sdlc/forge/issues/150), + [structured model outputs #252](https://github.com/forge-sdlc/forge/issues/252), + [heartbeats #78](https://github.com/forge-sdlc/forge/issues/78), + [container transcript errors #79](https://github.com/forge-sdlc/forge/issues/79), and + [idempotent terminal retrospectives #261](https://github.com/forge-sdlc/forge/issues/261). +- Active PRs: [heartbeat logging #158](https://github.com/forge-sdlc/forge/pull/158), + [transcript error surfacing #239](https://github.com/forge-sdlc/forge/pull/239), and + [terminal retrospectives #271](https://github.com/forge-sdlc/forge/pull/271). +- Merged foundations: [execution failure routing #147](https://github.com/forge-sdlc/forge/issues/147), + [terminal retry notification PR #155](https://github.com/forge-sdlc/forge/pull/155), + [per-run trace IDs #80](https://github.com/forge-sdlc/forge/issues/80), and + [cross-worker ticket serialization PR #212](https://github.com/forge-sdlc/forge/pull/212). +- Tracking gap: the side-effect journal and cross-provider correlation index do not yet + have dedicated issues. **Exit criteria** @@ -206,6 +213,15 @@ revisions, not mutable branch names. 5. Harden mixed-provider workflows across multiple simultaneous GitLab instances, with independent change requests but one aggregate approval and completion view. +**Related tracking (reviewed 2026-08-10)** + +- Open issue: [configurable source providers and mixed-provider workflows #162](https://github.com/forge-sdlc/forge/issues/162). +- Merged foundation: [multi-repository PR lifecycle tracking PR #238](https://github.com/forge-sdlc/forge/pull/238) + resolved [issue #135](https://github.com/forge-sdlc/forge/issues/135). +- Tracking gaps: GitLab.com, self-managed GitLab, multiple simultaneous GitLab instances, + provider conformance tests, and checkpoint migration need dedicated issues beneath + #162. + **Exit criteria** - The same provider contract suite passes for GitHub, GitLab.com, and a supported @@ -254,6 +270,17 @@ independently. This theme implements the intent of [pluggable sandbox drivers #30](https://github.com/forge-sdlc/forge/issues/30). +**Related tracking (reviewed 2026-08-10)** + +- Open issue and implementation PR: [pluggable sandbox drivers #30](https://github.com/forge-sdlc/forge/issues/30) + and [Kubernetes driver PR #243](https://github.com/forge-sdlc/forge/pull/243). +- Security and runtime design: [OpenShell boundary spike #262](https://github.com/forge-sdlc/forge/issues/262), + [driver capabilities #265](https://github.com/forge-sdlc/forge/issues/265), + [sandbox hardening #266](https://github.com/forge-sdlc/forge/issues/266), and + [structured execution security evidence #264](https://github.com/forge-sdlc/forge/issues/264). +- Tracking gap: production OCI images, Helm/OpenShift deployment, HA operations, + upgrades, backup, and disaster recovery do not yet have dedicated issues. + **Exit criteria** - Podman and Kubernetes drivers pass one conformance suite. @@ -303,6 +330,18 @@ capability. first-pass rate, completion rate, cost, and time. - A stable extension API only after two internal workflow prototypes prove the contract. +**Related tracking (reviewed 2026-08-10)** + +- Merged foundation: [PRD approval workflow issue #33](https://github.com/forge-sdlc/forge/issues/33) + and [implementation PR #83](https://github.com/forge-sdlc/forge/pull/83). +- Open feedback work: [artifact Q&A #163](https://github.com/forge-sdlc/forge/issues/163), + [decompose draft review #218](https://github.com/forge-sdlc/forge/issues/218), and + [draft review PR #242](https://github.com/forge-sdlc/forge/pull/242). +- Related developer tooling: [local skill testing #296](https://github.com/forge-sdlc/forge/issues/296). +- Tracking gaps: competing PRD-driven product prototypes, prototype comparison and + learning capture, disposable preview workspaces, workflow simulation, dry-run/shadow + execution, version pinning, canary rollout, and rollback need dedicated issues. + **Exit criteria** - Given one PRD with an unresolved product or implementation choice, a user can create, @@ -332,6 +371,13 @@ Conversational ticket intake is a separate upstream integration and is not requi the deployment MVP. Creation requires explicit approval or command by default; projects may opt into automatic creation after CI through policy. +**Related tracking (reviewed 2026-08-10)** + +- Open proposal: [staging/demo environments #28](https://github.com/forge-sdlc/forge/issues/28). +- Tracking gaps: lifecycle hooks, the environment state machine, plugin conformance, + allowlisted automation jobs, approval policy, and TTL reconciliation need dedicated + issues after #28 is refined. + **Environment record** - Stable environment ID, owner, ticket and PR/MR references. @@ -415,15 +461,29 @@ existing code whose behavior is insufficiently verified. - Measure meaningful verification gains with behavior/risk coverage, mutation score, escaped-defect history, and flaky-test impact rather than line coverage alone. -**Current security backlog incorporated:** -[sandbox capability requirements #265](https://github.com/forge-sdlc/forge/issues/265), -[sandbox hardening #266](https://github.com/forge-sdlc/forge/issues/266), -[structured security evidence #264](https://github.com/forge-sdlc/forge/issues/264), -[fail-closed pre-push validation #263](https://github.com/forge-sdlc/forge/issues/263), -[OpenShell boundary spike #262](https://github.com/forge-sdlc/forge/issues/262), -[credential isolation #82](https://github.com/forge-sdlc/forge/issues/82), -[secret scanning #77](https://github.com/forge-sdlc/forge/issues/77), and -[prompt-injection auditing #76](https://github.com/forge-sdlc/forge/issues/76). +**Related tracking (reviewed 2026-08-10)** + +- Open security issues: [sandbox capability requirements #265](https://github.com/forge-sdlc/forge/issues/265), + [sandbox hardening #266](https://github.com/forge-sdlc/forge/issues/266), + [structured security evidence #264](https://github.com/forge-sdlc/forge/issues/264), + [fail-closed pre-push validation #263](https://github.com/forge-sdlc/forge/issues/263), + [OpenShell boundary spike #262](https://github.com/forge-sdlc/forge/issues/262), + [credential isolation #82](https://github.com/forge-sdlc/forge/issues/82), + [secret scanning #77](https://github.com/forge-sdlc/forge/issues/77), and + [prompt-injection auditing #76](https://github.com/forge-sdlc/forge/issues/76). +- Active security PRs: [pre-push validation #272](https://github.com/forge-sdlc/forge/pull/272) + and [prompt-injection auditing #287](https://github.com/forge-sdlc/forge/pull/287). +- Merged Forge hardening: [security batch PR #231](https://github.com/forge-sdlc/forge/pull/231) + resolved issues [#219–#226](https://github.com/forge-sdlc/forge/pull/231), + covering CORS, authorization, command/path injection, endpoint exposure, error leakage, + and skill-source validation. +- Verification-debt proposals: [test generation node #256](https://github.com/forge-sdlc/forge/issues/256) + and [test coverage workflow #257](https://github.com/forge-sdlc/forge/issues/257) + were closed without implementation; [deterministic pre-PR validation #174](https://github.com/forge-sdlc/forge/issues/174) + remains open. +- Tracking gaps: CVE intake/remediation, normalized security stages and findings, + accepted-risk policy, risk-based test planning, mutation/behavior coverage, and flaky + test accounting need dedicated issues. **Exit criteria** @@ -461,13 +521,19 @@ Platform breadth is only valuable if users can understand and govern it. making MLflow a runtime dependency or storing secrets/raw sensitive context by default. - Pre-change-request validation defined by project policy/skills. -**Current backlog incorporated:** -[revision identity #91](https://github.com/forge-sdlc/forge/issues/91), -[parent-first review #84](https://github.com/forge-sdlc/forge/issues/84), -[concurrent CI/review #137](https://github.com/forge-sdlc/forge/issues/137), -[Langfuse labels #138](https://github.com/forge-sdlc/forge/issues/138), and -[prompt efficiency #39](https://github.com/forge-sdlc/forge/issues/39), plus the existing -workflow-status and statistics proposals. +**Related tracking (reviewed 2026-08-10)** + +- Open issues and PRs: [revision identity #91](https://github.com/forge-sdlc/forge/issues/91), + [concurrent CI/review #137](https://github.com/forge-sdlc/forge/issues/137) with + [PR #143](https://github.com/forge-sdlc/forge/pull/143), and + [prompt efficiency #39](https://github.com/forge-sdlc/forge/issues/39). +- Merged foundations: [parent-first review #84](https://github.com/forge-sdlc/forge/issues/84), + [Langfuse labels #138](https://github.com/forge-sdlc/forge/issues/138), and + [provider-neutral per-stage model policy issue #175](https://github.com/forge-sdlc/forge/issues/175) + with [PR #251](https://github.com/forge-sdlc/forge/pull/251). +- Tracking gaps: automatic model routing and escalation, routing evaluation baselines, + MLflow integration, aggregate provider-neutral status, and end-to-end workflow economics + need dedicated issues. **Exit criteria** From 86f23822933a9f3dd95ac1b58070564c9eaad260 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 12:33:41 +0300 Subject: [PATCH 13/18] docs: generalize explicit outcome direction --- docs/roadmap.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index e2e167e5c..8f4e43d34 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -58,9 +58,8 @@ state or unrecoverable failures. **Deliverables** -- **Explicit outcomes:** versioned, typed node and execution results distinguish - `actionable`, `no_action`, `needs_input`, and `failed`; missing, malformed, or - incompatible results fail closed. Markdown remains a view, not the workflow contract. +- **Explicit outcomes:** nodes and execution drivers return validated, versioned results + with unambiguous completion and failure semantics; invalid results fail closed. - **Replay-safe effects:** an idempotent journal governs comments, branches, PRs/MRs, deployments, and teardown so retries and duplicate events cannot repeat mutations. - **Durable identity:** stable correlation IDs link tickets, repositories, revisions, From de3a5148b72b171fea5f295fb5bd8294ecd8cac0 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 12:37:39 +0300 Subject: [PATCH 14/18] docs: focus source provider roadmap on outcomes --- docs/roadmap.md | 147 ++++++++++-------------------------------------- 1 file changed, 30 insertions(+), 117 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 8f4e43d34..5fe39bdfd 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -102,134 +102,47 @@ state or unrecoverable failures. “Multiple git sources” has two dimensions: a project may span repositories, and each repository may live on a different provider or provider instance. Forge already supports -the first for GitHub; this theme adds the second. - -**Repository and workflow-state model** - -Use canonical repository identities rather than `owner/repo` strings. The provider is a -property of each repository reference, never of the workflow as a whole: - -```yaml -id: payments-api -provider: github # github | gitlab -connection: public-github # configured Forge connection -namespace: acme/payments -default_branch: main -change_request_mode: fork # fork | branch -``` - -A connection holds base/API URLs, webhook verification settings, credential reference, -TLS/CA configuration, and allowed namespaces. This is essential for multiple internal -GitLab deployments and prevents credentials from being embedded in Jira metadata. - -Do not extend the current `current_repo`, `current_pr_number`, and `pr_urls` fields with a -single workflow-level provider. That shape cannot safely represent PR #42 in GitHub and -MR !42 in GitLab, provider-specific CI state, or concurrent events from several -repositories. Replace it with a map of independently progressing repository work items: - -```yaml -repository_changes: - payments-api: # stable Forge repository ID - repository: - provider: github - connection: public-github - namespace: acme/payments - source_revision: main@abc123 - branch: forge/PROJ-123 - change_request: - native_id: "42" # opaque string; never globally unique - url: https://github.com/acme/payments/pull/42 - state: open - checks: - state: passed - runs: [] - review: - state: approved - execution: - state: completed - deployment-config: - repository: - provider: gitlab - connection: corp-gitlab - namespace: platform/deployment-config - source_revision: main@def456 - branch: forge/PROJ-123 - change_request: - native_id: "42" - url: https://gitlab.corp/platform/deployment-config/-/merge_requests/42 - state: open - checks: - state: running - runs: [] - review: - state: pending - execution: - state: completed -``` - -The durable external identity of a change request is the composite -`(connection, repository_id, native_id)`. URLs are presentation data, and native numbers -are opaque provider-local identifiers. A `current_work_item` may exist as a scheduling -cursor, but it must not be the source of truth for event routing or completion. - -Workflow-level status is a derived aggregate over `repository_changes`, using explicit -policy such as `all_required_changes_pass_ci`, `all_required_changes_approved`, and -`allow_partial_completion`. Each work item keeps its own execution, push, change-request, -CI, review, retry, and error state. This permits repositories to progress concurrently -and prevents a GitLab event from overwriting the active GitHub state. - -Cross-repository ordering must also be explicit. Work items may declare dependencies—for -example, deploy configuration waits for an application image digest—rather than relying -on list position. Outputs passed between repositories use typed artifacts and immutable -revisions, not mutable branch names. - -**Provider contract** - -- Repository discovery, clone/fetch URL, and default branch. -- Branch/fork creation and push authorization. -- Pull request / merge request create, update, comment, review, merge status, and close. -- CI status normalization, logs/artifacts lookup, and retry/cancel capabilities. -- Webhook verification, normalized events, actor identity, and delivery deduplication. -- Event routing by the composite change-request identity into exactly one repository work - item, followed by recomputation of aggregate workflow gates. -- Capability discovery so workflows can degrade deliberately when a provider lacks a - feature. +the first for GitHub; this theme adds provider choice without weakening coordinated +multi-repository delivery. -**Delivery slices** +**Outcomes** + +- GitHub, GitLab.com, and self-managed GitLab operate behind one provider-neutral source + control capability, with provider differences exposed deliberately. +- A single workflow can mix GitHub repositories and repositories from multiple GitLab + instances from the first supported GitLab milestone. +- Repository changes progress independently while Forge presents unified approval, CI, + and completion status across the workflow. +- Connections, credentials, webhook identity, and provider-specific configuration remain + centrally governed and outside agent context. + +**Sequence** + +1. Move current GitHub behavior behind the provider boundary without regression. +2. Introduce GitLab.com together with mixed GitHub/GitLab workflow support. +3. Add self-managed and multiple GitLab instances, then harden compatibility across the + supported provider matrix. -1. Extract the existing GitHub implementation behind `SourceControlProvider` and prove - no behavioral regression. -2. Replace GitHub-specific workflow state and language with provider-neutral repository, - per-repository change request, review, and check-run models; migrate existing - checkpoints from the single-current-PR shape. -3. Add GitLab.com support for branch-based merge requests, pipelines/jobs, discussions, - approvals, and system hooks/project webhooks. The first supported GitLab release must - also support a workflow mixing GitHub and GitLab repositories; single-provider-only - project support is not an acceptable milestone. -4. Add self-managed GitLab connections: arbitrary base URL, private CA bundle, proxy, - version/capability probing, group/project tokens, OAuth/service accounts, and multiple - simultaneous instances. -5. Harden mixed-provider workflows across multiple simultaneous GitLab instances, with - independent change requests but one aggregate approval and completion view. +Issue [#162](https://github.com/forge-sdlc/forge/issues/162) is the canonical technical +plan for repository identity, workflow state, provider contracts, migrations, and event +routing. **Related tracking (reviewed 2026-08-10)** - Open issue: [configurable source providers and mixed-provider workflows #162](https://github.com/forge-sdlc/forge/issues/162). - Merged foundation: [multi-repository PR lifecycle tracking PR #238](https://github.com/forge-sdlc/forge/pull/238) resolved [issue #135](https://github.com/forge-sdlc/forge/issues/135). -- Tracking gaps: GitLab.com, self-managed GitLab, multiple simultaneous GitLab instances, - provider conformance tests, and checkpoint migration need dedicated issues beneath - #162. +- Tracking gaps: the delivery slices in #162 should be split into implementation issues + as work is scheduled. **Exit criteria** -- The same provider contract suite passes for GitHub, GitLab.com, and a supported - self-managed GitLab version range. -- One test feature produces coordinated GitHub PR and GitLab MR changes. -- Interleaved and replayed GitHub/GitLab events update only their addressed repository - work item and produce the correct aggregate gate state. -- No workflow node imports a concrete source-control client. -- Connection credentials and private CA material never enter agent prompts or logs. +- One workflow produces coordinated changes across GitHub and GitLab.com, and across the + supported self-managed GitLab matrix. +- Mixed-provider events and failures remain isolated to the correct repository change + while aggregate workflow gates stay correct. +- Provider conformance tests show equivalent core behavior, and credentials or private + connection material never enter agent prompts or logs. ### 3. Pluggable execution and Kubernetes support From cdbdd6b68b8786e7c0877316e10d3e8fe0a384ff Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 12:46:50 +0300 Subject: [PATCH 15/18] docs: streamline roadmap structure and voice --- docs/roadmap.md | 336 +++++++++++++++++------------------------------- 1 file changed, 121 insertions(+), 215 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 5fe39bdfd..cd8c2fe55 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -47,8 +47,32 @@ workflows—while making Jira, GitHub, and Podman defaults rather than permanent intent, policy, and lifecycle reconciliation; plugins and target platforms own provisioning and infrastructure state. +## Product decisions + +1. **Primary audience:** enterprise platform teams operating Forge as a central service; + open source supports self-hosting, extension, and contribution rather than making + local execution the primary product mode. +2. **Kubernetes scope:** both the Forge control plane and isolated agent execution, + delivered as separate tracks. +3. **Mixed source providers:** GitHub and multiple GitLab instances must coexist within + one project from the first supported GitLab milestone. +4. **Product prototyping:** users compare working options from an initial PRD and feed + approved learning into the PRD and plan before production implementation. +5. **Preview environments:** explicit approval by default, with project-policy opt-in to + automatic creation. +6. **First deployment integration:** invoke an allowlisted automation job before adding + declarative environment reconciliation. +7. **Extension model:** Forge-maintained built-ins and separately deployed external + plugins share interfaces and conformance suites with trust-appropriate isolation. +8. **Issue tracking:** Jira is the built-in default behind an interface, not a permanent + architectural requirement. + ## Roadmap themes +Each theme uses the same structure: intended outcomes, related tracking, and observable +exit criteria. Technical schemas and implementation contracts belong in the linked issues +and proposals. Tracking status was reviewed on 2026-08-10. + ### 1. Deterministic and recoverable orchestration Forge is moving toward transitions that are validated, replay-safe, traceable, and @@ -56,7 +80,7 @@ recoverable. These capabilities should be delivered incrementally and adopted by new provider, runtime, and workflow, enabling expansion without multiplying ambiguous state or unrecoverable failures. -**Deliverables** +**Outcomes** - **Explicit outcomes:** nodes and execution drivers return validated, versioned results with unambiguous completion and failure semantics; invalid results fail closed. @@ -72,7 +96,7 @@ state or unrecoverable failures. - **Proof under failure:** contract, replay, failure-injection, and migration tests verify these guarantees across restarts and workflow upgrades. -**Related tracking (reviewed 2026-08-10)** +**Tracking** - Open issues: [typed artifact contracts #150](https://github.com/forge-sdlc/forge/issues/150), [structured model outputs #252](https://github.com/forge-sdlc/forge/issues/252), @@ -127,7 +151,7 @@ Issue [#162](https://github.com/forge-sdlc/forge/issues/162) is the canonical te plan for repository identity, workflow state, provider contracts, migrations, and event routing. -**Related tracking (reviewed 2026-08-10)** +**Tracking** - Open issue: [configurable source providers and mixed-provider workflows #162](https://github.com/forge-sdlc/forge/issues/162). - Merged foundation: [multi-repository PR lifecycle tracking PR #238](https://github.com/forge-sdlc/forge/pull/238) @@ -146,43 +170,28 @@ routing. ### 3. Pluggable execution and Kubernetes support -Kubernetes support must cover two separate user needs: deploying the Forge control plane -to Kubernetes and running isolated agent jobs on Kubernetes. They should be deliverable -independently. - -**Execution driver contract** +Kubernetes support has two independent tracks: running isolated agent work and operating +the Forge control plane. Both build on provider-neutral execution semantics so workflow +logic does not depend on Podman, Kubernetes, or a future runtime. -- Submit an immutable execution specification: image digest, command, workspace, resource - limits, deadline, network policy profile, secrets references, and correlation labels. -- Observe status and heartbeats, stream bounded logs, cancel, collect typed results and - artifacts, and clean up idempotently. -- Drivers: existing local Podman, then Kubernetes Job; future drivers can include remote - container services without changing workflow nodes. - -**Kubernetes agent execution** - -- Kubernetes Jobs with per-run ServiceAccounts, security contexts, quotas, deadlines, - and TTL cleanup. -- Workspace transport via object storage or purpose-built PVCs; do not assume a shared - host filesystem. -- Default-deny network policies with explicit egress profiles. -- External Secrets / workload identity integration instead of environment-secret copies. -- Log and artifact size limits, cancellation, orphan reconciliation, and namespace-level - concurrency quotas. -- Compatibility with vanilla Kubernetes and OpenShift restricted security profiles. +**Outcomes** -**Forge control-plane deployment** +- Podman and Kubernetes execute the same bounded work specification and expose consistent + lifecycle, logs, artifacts, cancellation, and cleanup behavior. +- Kubernetes agent jobs use workload identity, least privilege, resource limits, + controlled egress, and durable workspace/artifact transport. +- Forge ships as an operable Kubernetes/OpenShift service with supported installation, + upgrades, rollback, scaling, recovery, and production security guidance. +- Runtime failures, restarts, and orphaned resources converge without losing or + duplicating workflow work. -- Versioned OCI images and Helm chart for API, worker, Redis dependency/external Redis, - Service, Ingress/Route, probes, PodDisruptionBudget, autoscaling, and metrics. -- Database/checkpoint migrations and documented upgrade/rollback policy. -- HA worker semantics, graceful shutdown, queue draining, backups, and disaster recovery. -- Production security guide and reference values for OpenShift. +**Sequence** -This theme implements the intent of -[pluggable sandbox drivers #30](https://github.com/forge-sdlc/forge/issues/30). +1. Establish the execution-driver boundary and retain Podman behavior. +2. Add conformant Kubernetes agent execution and sandbox hardening. +3. Deliver and harden the Kubernetes/OpenShift control-plane distribution. -**Related tracking (reviewed 2026-08-10)** +**Tracking** - Open issue and implementation PR: [pluggable sandbox drivers #30](https://github.com/forge-sdlc/forge/issues/30) and [Kubernetes driver PR #243](https://github.com/forge-sdlc/forge/pull/243). @@ -202,47 +211,24 @@ This theme implements the intent of ### 4. Product prototyping and workflow evolution -Product prototyping is a discovery workflow for Forge users. Starting from an initial -PRD or feature idea, a user can ask Forge to build one or more competing prototypes, -interact with and revise them, compare their behavior, and feed what was learned back -into the PRD and product plan before committing to a production implementation. This is -distinct from prototyping Forge's own workflow graph, which remains a platform-engineering -capability. - -**Deliverables** - -- A PRD-to-prototype discovery workflow that turns explicit uncertainties and hypotheses - into one or more time-boxed prototype options with comparable goals and evaluation - criteria. -- Isolated, disposable prototype workspaces and optional preview environments where users - can exercise behavior, provide feedback, and request revisions without creating a - production PR/MR or representing the prototype as production-ready code. -- Side-by-side comparison of alternative prototypes using user feedback, behavior, - feasibility, architecture implications, risks, cost, and measured results—not only an - agent preference. -- A governed learning step that proposes concrete PRD and plan updates, records which - prototype evidence supports each change, and requires user approval before modifying - the canonical artifacts. -- An explicit transition from discovery to delivery: discard all options, continue - prototyping, or select an option and generate a coherent implementation plan. Reuse - validated decisions and evidence, but regenerate production-quality implementation - rather than silently promoting disposable prototype code. -- Versioned workflow definition and registry with typed inputs, outputs, gates, retry - policies, permissions, and capability requirements. -- Workflow scaffold CLI and validation/lint command. -- Visual graph rendering plus a step-by-step simulator using fixture events and recorded - adapter responses. -- `dry-run` mode: agents may generate artifacts, but external writes are captured as an - inspectable side-effect plan. -- `shadow` mode: run a candidate workflow against copied/sanitized events without writes - and compare decisions, cost, latency, and artifacts to the active version. -- Project-level pinning, canary rollout, immutable workflow version per in-flight run, - checkpoint migration rules, and one-click rollback for new runs. -- Evaluation datasets and scorecards for artifact quality, approval revisions, CI - first-pass rate, completion rate, cost, and time. -- A stable extension API only after two internal workflow prototypes prove the contract. - -**Related tracking (reviewed 2026-08-10)** +This theme contains two related capabilities: product discovery for Forge users and safe +evolution of Forge workflows by platform engineers. Both rely on isolated experiments, +measured comparison, and an explicit decision before promotion. + +**Outcomes** + +- Users can turn an uncertain PRD or feature idea into multiple time-boxed prototypes, + revise and compare them, and approve evidence-backed updates to the PRD and plan. +- Prototype code and environments remain disposable and clearly separate from production + delivery; validated decisions may carry forward, but production code is regenerated. +- Platform engineers can define, validate, simulate, and evaluate versioned workflows + without editing orchestrator routing code or mutating live projects. +- Dry-run, shadow, pinning, canary, migration, and rollback controls support measured + workflow evolution without changing in-flight runs unexpectedly. +- Stable extension APIs follow proven internal implementations rather than speculative + abstraction. + +**Tracking** - Merged foundation: [PRD approval workflow issue #33](https://github.com/forge-sdlc/forge/issues/33) and [implementation PR #83](https://github.com/forge-sdlc/forge/pull/83). @@ -256,78 +242,43 @@ capability. **Exit criteria** -- Given one PRD with an unresolved product or implementation choice, a user can create, - revise, and compare at least two working prototype options before selecting either. -- Approved learnings update the PRD and implementation plan with traceable prototype and - user-feedback evidence; rejected learnings leave the canonical artifacts unchanged. -- Prototype code and environments are clearly marked disposable, isolated from production - delivery, and cleaned up according to policy. -- A new experimental workflow can be scaffolded and simulated without editing worker - routing code. -- Dry-run mode performs zero external mutations, verified by adapter contract tests. -- In-flight workflows remain on their original version during a rollout. -- A candidate version can be promoted or rolled back using measured evaluation results. +- A user can compare at least two working options for one unresolved PRD decision and + apply only approved learnings to the canonical artifacts. +- Prototype resources are isolated, visibly disposable, and cleaned up by policy. +- A candidate workflow can be simulated, evaluated, promoted, and rolled back without + external dry-run mutations or unintended changes to in-flight runs. ### 5. External deployment and ephemeral environments -[Issue #28](https://github.com/forge-sdlc/forge/issues/28) should be refined into a -generic lifecycle-hook and deployment-plugin capability. Deployment controllers, -GitOps systems, and infrastructure-automation services integrate through the same -contract; none are dependencies of Forge core. +Forge should govern deployment intent and environment lifecycle while external systems +remain responsible for infrastructure operations and state. The first use case is a +preview environment for unmerged changes, created only after explicit approval by +default. + +**Outcomes** -**Refined scope** +- A provider-neutral lifecycle and plugin boundary supports deployment controllers, + GitOps systems, and automation services without making them Forge dependencies. +- Environment records connect immutable inputs, policy, ownership, status, outputs, TTL, + teardown, and audit history without exposing credentials. +- Allowlisted templates, quotas, budgets, approvals, and target connections constrain + what can be provisioned and where. +- Idempotent reconciliation converges provisioning and teardown across duplicate events, + missed callbacks, expiry, and Forge restarts. -Forge owns the decision and lifecycle record; the plugin owns infrastructure operations. -The first use case is a preview/demo environment built from unmerged change requests. -Conversational ticket intake is a separate upstream integration and is not required for -the deployment MVP. Creation requires explicit approval or command by default; projects -may opt into automatic creation after CI through policy. +**Sequence** -**Related tracking (reviewed 2026-08-10)** +1. Define the lifecycle, threat model, and reference plugin. +2. Prove the contract by invoking an allowlisted automation job. +3. Add declarative/GitOps integrations, automatic teardown, quotas, cost reporting, and + an external plugin SDK. + +**Tracking** - Open proposal: [staging/demo environments #28](https://github.com/forge-sdlc/forge/issues/28). -- Tracking gaps: lifecycle hooks, the environment state machine, plugin conformance, - allowlisted automation jobs, approval policy, and TTL reconciliation need dedicated - issues after #28 is refined. - -**Environment record** - -- Stable environment ID, owner, ticket and PR/MR references. -- Requested template, immutable source revisions/image digests, parameters, and policy. -- Provider operation ID, lifecycle state, timestamps, TTL, cost/size classification. -- Non-secret outputs such as URLs; credentials are delivered through a secret broker or - one-time access mechanism, never Jira/PR comments. -- Teardown reason, status, retries, and audit history. - -**Plugin contract** - -- `validate(request)`, `provision(request, idempotency_key)`, `status(operation_id)`, - `outputs(operation_id)`, and `destroy(operation_id, idempotency_key)`. -- Signed/authenticated callbacks plus polling fallback. -- Capability declaration, health check, timeouts, retry classification, and redacted - errors. -- Hooks initially available after change-request creation, after required CI passes, and - on close/merge/ticket completion/TTL expiration. Policy selects which hooks are active. - -**Safety policy** - -- Allowlisted project templates, parameter schemas, quotas, maximum TTL, concurrency and - budget limits, approved target connections, and optional human deployment approval. -- Unique Forge environment IDs are passed to plugins; plugins remain responsible for - provider-specific naming and collision handling. -- A durable reconciler performs teardown. Webhook-only teardown is insufficient because - events can be missed and Forge can be offline at expiry. -- Provisioning failure must not mutate source history or weaken CI gates. -- Deployment success does not imply production release approval. - -**Delivery slices** - -1. Proposal and threat model; environment state machine and lifecycle hook contract. -2. No-op/reference plugin and conformance suite; dry-run and manual trigger. -3. Generic job-orchestration plugin for invoking allowlisted automation templates. -4. Kubernetes/GitOps plugin for applying approved, parameterized environment templates. -5. Automatic TTL and PR/MR/ticket teardown reconciliation; access-output delivery. -6. Multiple change requests, refresh/redeploy, quotas, cost reporting, and plugin SDK. +- Tracking gaps: lifecycle hooks, environment state, plugin conformance, allowlisted + automation jobs, approval policy, and TTL reconciliation need dedicated issues after + #28 is refined. **Exit criteria** @@ -343,37 +294,20 @@ Security and testing are workflow outcomes, not only implementation-stage tools. should support both urgent vulnerability remediation and systematic improvement of existing code whose behavior is insufficiently verified. -**Security deliverables** - -- CVE remediation workflow: ingest a vulnerability advisory or scanner finding, resolve - affected repositories and dependency paths, assess exploitability and priority, - propose the smallest safe upgrade or mitigation, generate regression tests, and retain - advisory-to-commit evidence. -- Policy-controlled security stages in implementation workflows: dependency, secret, - static-analysis, and generated-code weakness scans before publication and again through - repository-owned CI. Normalize findings into typed artifacts with severity, - confidence, location, remediation, and suppress/accept-risk decisions. -- Fail closed for findings above project policy thresholds; require an auditable human - exception for accepted risk. Scanner outage or malformed output must not be treated as - a pass. -- Harden Forge itself: sandbox capability contracts, default-deny execution profiles, - pre-push validation, structured execution security evidence, credential isolation, - secret scanning, and prompt-injection defenses. - -**Verification-debt deliverables** - -- A dedicated test workflow for existing features and code: discover unverified behavior - from requirements, incidents, change history, coverage/mutation reports, and code risk; - prioritize verification debt; generate focused tests without requiring a feature - implementation; and open reviewable PRs/MRs with traceability to the behavior covered. -- Improve test planning in the normal planning and implementation workflows with explicit - behavior inventories, risk-based test matrices, negative/boundary/concurrency cases, - and a clear split between deterministic repository validation and model-reviewed - evidence. -- Measure meaningful verification gains with behavior/risk coverage, mutation score, - escaped-defect history, and flaky-test impact rather than line coverage alone. - -**Related tracking (reviewed 2026-08-10)** +**Outcomes** + +- A CVE workflow traces advisories or findings to affected code, assesses risk, proposes + minimal remediation, adds regression tests, and preserves advisory-to-change evidence. +- Policy-controlled security stages scan dependencies, secrets, source, and generated + changes; required checks fail closed and accepted risk requires an auditable exception. +- Forge hardening covers sandbox boundaries, pre-push validation, credential isolation, + security evidence, secret scanning, and prompt-injection defenses. +- A verification-debt workflow discovers risky unverified behavior in existing code, + prioritizes it, and opens focused test-only changes with behavior-level traceability. +- Planning uses risk-based behavior matrices and measures verification gains beyond line + coverage, including mutation effectiveness and flaky-test impact. + +**Tracking** - Open security issues: [sandbox capability requirements #265](https://github.com/forge-sdlc/forge/issues/265), [sandbox hardening #266](https://github.com/forge-sdlc/forge/issues/266), @@ -412,28 +346,22 @@ existing code whose behavior is insufficiently verified. Platform breadth is only valuable if users can understand and govern it. -**Deliverables** - -- Concise Jira progress updates and a provider-neutral aggregate view across all - repositories and environments. -- Review split artifacts on their parent before creating child tickets; revisions update - stable items rather than deleting and recreating them. -- Concurrent CI observation and human review where policy allows, while merge readiness - still requires both. -- Better Langfuse span names and end-to-end workflow statistics: duration, revisions, - model/token cost, first-pass CI, failure class, and environment lifetime. -- Layered prompt efficiency, context budgets, caching, and per-stage model policy. -- An `auto` model option per stage that dynamically routes by task complexity, context, - required capabilities, latency/cost budget, and observed quality. Policies must support - allowlists, deterministic fallback, retry/escalation to a stronger model, and a pinned - model override for reproducibility. -- MLflow integration as an optional experiment/evaluation backend: record workflow and - stage parameters, model/provider identity, prompt/artifact versions, datasets, metrics, - costs, latency, and correlation IDs; compare routing and workflow variants without - making MLflow a runtime dependency or storing secrets/raw sensitive context by default. -- Pre-change-request validation defined by project policy/skills. - -**Related tracking (reviewed 2026-08-10)** +**Outcomes** + +- Users see concise progress, ownership, next actions, and aggregate status across every + repository and environment associated with a workflow. +- Artifact review preserves stable identity and happens before irreversible decomposition + or publication; CI and human review may proceed concurrently without weakening gates. +- End-to-end telemetry reports outcome, latency, revisions, failures, quality, and cost by + workflow stage and configuration. +- Per-stage model policy supports a pinned model or auditable `auto` routing with budgets, + capability constraints, deterministic fallback, and escalation. +- Optional MLflow export compares workflow and routing variants using shared correlation + IDs without becoming a runtime dependency or exporting sensitive context by default. +- Prompt efficiency, context budgets, caching, and repository-owned pre-PR validation + reduce cost and avoidable feedback cycles without lowering quality. + +**Tracking** - Open issues and PRs: [revision identity #91](https://github.com/forge-sdlc/forge/issues/91), [concurrent CI/review #137](https://github.com/forge-sdlc/forge/issues/137) with @@ -511,28 +439,6 @@ Track these by project, provider, workflow version, and execution driver: - Preview-environment provision time, success rate, TTL compliance, and leaked resources. - Provider/driver conformance pass rate and upgrade compatibility. -## Product decisions - -1. **Primary audience:** prioritize enterprise platform teams operating Forge as a - central service. Preserve open-source self-hosting, extensibility, and contribution; - local execution of Forge is not a primary product mode. -2. **Kubernetes scope:** support both running the Forge control plane and running isolated - agent sandboxes, as separately shippable tracks. -3. **Mixed source providers:** a Jira project must be able to mix GitHub repositories and - repositories from multiple GitLab instances from the first supported GitLab milestone. -4. **Product prototyping:** Forge users start from an initial PRD, explore and revise one - or more working options, compare the results, and feed approved learning into a - coherent PRD and plan before production implementation. -5. **Preview environments:** require explicit approval or command by default, with - project-policy opt-in for automatic creation. -6. **First deployment integration:** prove the external deployment contract by invoking - an allowlisted automation job before adding declarative environment reconciliation. -7. **Extension model:** provide Forge-maintained built-ins and interfaces/conformance - suites for separately deployed external plugins, with trust-appropriate isolation. -8. **Issue tracking:** abstract Jira behind an issue-tracker interface. Ship Jira as the - built-in default while allowing external issue-tracker plugins; Jira is not a permanent - architectural requirement. - ## Near-term proposal backlog Before implementation, create and review focused proposals in this order: From f6920d0d35f14bd6d3ccb839672a383f1027fc44 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 12:53:46 +0300 Subject: [PATCH 16/18] docs: drop recommended roadmap sequence --- docs/roadmap.md | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index cd8c2fe55..e374fc266 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -387,23 +387,6 @@ Platform breadth is only valuable if users can understand and govern it. - The same evaluation run can be inspected in Forge telemetry and, when configured, MLflow using a shared correlation ID. -## Recommended sequence - -The themes overlap, but their enabling order should be explicit. - -| Horizon | Primary outcome | Included work | -| --- | --- | --- | -| **Now: Trust the core** | Forge never advances ambiguously and users can diagnose failures | Typed artifacts, execution failure semantics, correlation/indexing, idempotency, terminal notifications, redaction/injection defenses, sandbox hardening, security evidence, status/telemetry | -| **Next: Create extension seams** | Current behavior runs through stable abstractions | Source-control and issue-tracker provider contracts with built-in adapters; execution driver contract with Podman adapter; versioned workflow definitions; lifecycle hook proposal | -| **Then: Add enterprise backends** | Platform teams can centrally operate Forge in heterogeneous environments using an open, self-hostable platform | Mixed GitHub/GitLab workflows from the first GitLab milestone; GitLab.com and self-managed GitLab; Kubernetes agent Jobs; Helm/OpenShift deployment; short-lived credentials/private CA support | -| **Then: Discover, experiment, and deploy** | Teams can learn through working product options, evaluate workflows, and create governed preview environments | PRD-driven competing prototypes and feedback into planning; simulator/dry-run/shadow/canary; security and verification-debt workflows; dynamic model routing and MLflow evaluation; deployment plugin runtime; allowlisted automation-job plugin; TTL reconciler | -| **Later: Broaden the ecosystem** | External contributors can extend Forge without core changes | GitOps and deployment-controller plugins, supported SDKs, additional ticket/source/execution/deployment adapters, workflow template catalog, organization policy and portfolio analytics | - -Do not start all integrations simultaneously. A useful vertical-slice order is GitHub -through the new source contract, Podman through the new execution contract, then one -GitLab instance and one Kubernetes Job. Each abstraction should be proven by at least two -implementations before being declared stable. - ## Cross-cutting architecture decisions These decisions should be captured as proposals/ADRs before implementation: From 4b234d49fd21ffcc8b1321d626c6f6f96dde896d Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 13:09:08 +0300 Subject: [PATCH 17/18] docs: remove cross-cutting architecture section --- docs/roadmap.md | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index e374fc266..c3dc4963c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -387,26 +387,6 @@ Platform breadth is only valuable if users can understand and govern it. - The same evaluation run can be inspected in Forge telemetry and, when configured, MLflow using a shared correlation ID. -## Cross-cutting architecture decisions - -These decisions should be captured as proposals/ADRs before implementation: - -1. **Configuration ownership:** move from GitHub-shaped Jira properties to project - configuration referencing centrally managed provider connections. -2. **Plugin boundary:** support both Forge-maintained built-in adapters and separately - deployed external plugins behind the same versioned interfaces and conformance suites. - Begin with in-process Python interfaces for trusted built-ins; use a versioned - HTTP/event contract for external or higher-privilege plugins. Do not load arbitrary - plugin code into the worker. -3. **State durability:** define which state belongs in LangGraph checkpoints versus a - queryable operational store for correlation, idempotency, and environment lifecycle. -4. **Workflow compatibility:** define immutable workflow versions and checkpoint - migrations before user-authored graph definitions. -5. **Identity and authorization:** map Jira, GitHub, GitLab, and Forge service identities - into an auditable actor model with project policy enforcement. -6. **Support matrix:** publish tested GitLab, Kubernetes, OpenShift, Redis, and plugin API - versions with deprecation policy. - ## Measures of success Track these by project, provider, workflow version, and execution driver: From 18995942c0ae8d3502610857fafbe16609f9e264 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Mon, 10 Aug 2026 13:12:20 +0300 Subject: [PATCH 18/18] docs: streamline roadmap success measures --- docs/roadmap.md | 33 ++++++++------------------------- 1 file changed, 8 insertions(+), 25 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index c3dc4963c..2ff7e9a41 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -391,28 +391,11 @@ Platform breadth is only valuable if users can understand and govern it. Track these by project, provider, workflow version, and execution driver: -- Workflow completion and human-escalation rates. -- Incorrect-advance rate after failed/malformed agent or execution output (target: zero). -- Duplicate external side effects under event replay (target: zero). -- Median and p95 lead time by workflow stage; time waiting for humans is separate. -- Artifact revision count and CI first-pass rate. -- Cost per completed change and model tokens by stage. -- PR/MR review turnaround and change failure/reopen rate. -- Execution queue time, success rate, orphan rate, and cleanup latency. -- Preview-environment provision time, success rate, TTL compliance, and leaked resources. -- Provider/driver conformance pass rate and upgrade compatibility. - -## Near-term proposal backlog - -Before implementation, create and review focused proposals in this order: - -1. Typed agent artifact and node-result contract. -2. Stable correlation identity and idempotent side-effect journal. -3. Source-control provider contract and repository connection model. -4. Execution driver contract and Kubernetes threat model. -5. PRD-driven product-prototyping workflow and versioned workflow simulation/rollout model. -6. Lifecycle hooks, environment state machine, and deployment plugin contract. -7. Enterprise identity, credentials, and policy model spanning all adapters. -8. Security remediation, generated-code scanning, and accepted-risk policy. -9. Verification-debt workflow and risk-based test-planning contract. -10. Dynamic model-routing policy and MLflow evaluation integration. +- **Reliability:** incorrect advances and duplicate external effects (target: zero), plus + visible and recoverable terminal failures. +- **Delivery:** workflow completion, human escalation, stage lead time excluding human + wait, CI first-pass rate, review turnaround, and change failure or reopen rate. +- **Quality and economics:** artifact revisions, quality outcomes, model tokens and cost + per completed change, and routing performance against pinned-model baselines. +- **Platform operations:** provider/driver conformance, execution queue and success rates, + orphan cleanup, environment provisioning, TTL compliance, and leaked resources.