[RFC 006] add external environment import rfc#794
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Darktex
left a comment
There was a problem hiding this comment.
Note: This is an automated review by Claude Code, not a human review.
Alignment Review Report
Automated Checks
- Lint: PASS (no new source code introduced; markdown only)
- Debug code: CLEAN (no new code files in this PR)
Tier 1: Fixes Required
-
rfcs/006-external-environment-imports.md:40— Broken repository link. The RFC references[PR #726](https://github.com/huggingface/OpenEnv/pull/726)but the repository ismeta-pytorch/OpenEnv. The correct URL should behttps://github.com/meta-pytorch/OpenEnv/pull/726. -
rfcs/006-external-environment-imports.md:1— RFC header format deviation. Therfcs/README.mdtemplate specifies the title line as# RFC: [Title]but the new file uses# RFC 006: External Environment Imports. All existing RFCs (001–005) follow# RFC: [Title]with the ID in the metadata block. The title should be# RFC: External Environment Imports.
Tier 2: Alignment Discussion
ALIGNMENT FLAG: New HTTP compatibility routes may conflict with WebSocket-first communication invariant
- Principle at stake: INVARIANTS.md §4 — "WebSocket for all environment communication. No custom protocols." (with a noted ongoing transition from HTTP)
- The concern: The RFC proposes six new HTTP routes (
/list_environments,/{env_name}/splits,/{env_name}/tasks, etc.) as "compatibility bridges." While the RFC correctly labels these as infrastructure-facing metadata routes, explicitly limits them to non-agent, non-reset, non-reward surfaces, and commits to a WebSocket-native equivalent before 1.0, they still expand the HTTP surface during a period when the project is trying to deprecate HTTP (PR #252). The RFC's Open Question #1 directly acknowledges this tension but leaves resolution open. The concern is whether accepting these routes now sets a precedent that makes the HTTP deprecation path harder. - Suggested reviewer: @Darktex
ALIGNMENT FLAG: RFC 006 number re-assignment from roadmap
- Principle at stake: PRINCIPLES.md — "Key Decisions Made ... should not be changed without a new RFC"
- The concern:
rfcs/000-project-phases.mdexplicitly named RFC 006 as "Enabling production performance simulation to minimize training-production delta." This PR re-assigns 006 to external environment imports and changes that roadmap entry to "upcoming RFC." The amendment history in RFC 000 already references "RFCs 005-007" as part of a November 2025 amendment. Silently recycling an already-named RFC slot without a formal RFC 000 amendment entry could cause confusion about what was decided and when. The PR description does mention this is intentional, but RFC 000 has no updated amendment entry. - Suggested reviewer: @Darktex
Note: PR #795 (also from @burtenshaw) independently assigns RFC 006 to a different topic (HF RL environment datasets). These two PRs collide on the same RFC number and must be deconflicted before either lands.
Summary
- 2 mechanical issues to fix (broken link, header format)
- 2 alignment points for human review (HTTP surface expansion, RFC number reassignment without amendment log)
The RFC itself is well-structured and thorough. The boundary rules, TaskProvider-as-protocol decision, and explicit prohibition on agents accessing task discovery or simulation controls are all correctly aligned with OpenEnv's dual-API invariant. The two Tier 2 flags are process/precedent concerns rather than correctness blockers.
Automated review by Claude Code | Learn more
Darktex
left a comment
There was a problem hiding this comment.
Note: This is an automated review by Claude Code, not a human review.
Alignment Review Report
Automated Checks
- Lint: PASS — no source code changes; the diff is docs/RFC only, no Python lint surface
- Debug code: CLEAN — no debug artifacts introduced by this PR
Tier 1: Fixes Required
-
rfcs/006-external-environment-imports.md — RFC numbering collision. RFC number 006 is also claimed by the simultaneously-open PR #795 (
rfcs/006-hf-rl-environment-datasets.md, same authorburtenshaw), and an additional collision was observed withrfcs/006-multi-component-rewards.md(PR #624, author adithya-s-k). Using the same number for multiple distinct RFCs makes the index and cross-references permanently ambiguous. This PR must use a unique RFC number (007 or higher), with the filename and the**RFC ID**header updated consistently, and a coordinated merge ordering established. This is a blocking issue. -
rfcs/000-project-phases.md — The PR removes the
(RFC 006)reference from the "production performance simulation" roadmap item, replacing it with(upcoming RFC). After the patch lands, the roadmap item is effectively orphaned with no RFC assignment. Add a forward reference or tracking issue link so the intent is not lost. -
rfcs/006-external-environment-imports.md — RFC header style — Confirm the
**RFC ID**/**Status**header style matches whatever the template inrfcs/README.mdprescribes; existing RFCs are inconsistent (colon inside vs. outside the bold span). Match the template. -
rfcs/README.md — new category header — The PR appends the entry under a new
### External Environment Importssection. Confirm with the core team whether this deserves its own category or belongs under an existing one, and that it is reflected consistently in the README's structure. -
rfcs/006-external-environment-imports.md — verify package/identifier names — Strings such as
openrewardstandardappear as single words in the detection list and dependency mapping. If these are real PyPI package names they should be verified and linked; if they should be hyphenated/underscored, fix them, since the RFC text will be copied into generated code. -
Implementation files referenced in the RFC do not exist on
main— The RFC's Implementation Notes list files (src/openenv/cli/commands/import_env.py,src/openenv/cli/importers/*.py, additions tointerfaces.py,types.py,http_server.py) that do not exist in the current repo, implying PR #726 has not merged. The RFC body says it "documents the design implemented in PR #726." Either tie the RFC review explicitly to the PR #726 merge gate, or mark the RFCDraftuntil #726 is in review on the same base.
Tier 2: Alignment Discussion
ALIGNMENT FLAG: RFC-after-implementation bypasses the design-review gate
- Principle at stake: Design/Alignment is human-owned and meant to precede implementation (CLAUDE.md RFC process); the RFC explicitly states it retroactively documents code already written in PR #726.
- The concern: The architectural decisions under review (TaskProvider protocol, ORS-compatible HTTP routes, generated wrapper contract, boundary rules) are already implemented, so a reviewer requesting design changes faces much higher friction. Landing post-hoc RFCs normalizes documenting after the fact, degrading the review gate over time.
- Suggested reviewer: @Darktex
ALIGNMENT FLAG: ORS-compatible HTTP task routes widen the infrastructure-agent boundary against the WebSocket-only trajectory
- Principle at stake: INVARIANTS.md — "WebSocket for all environment communication... No custom protocols"; HTTP is being deprecated (PR #252).
- The concern: The RFC adds new HTTP routes (
GET /list_environments,GET /{env}/splits,POST /{env}/tasks, etc.) as a "compatibility bridge," moving opposite to the stated WebSocket-only direction. The RFC frames this as acceptable ("before 1.0") but provides no deprecation deadline or tracking issue for the WebSocket-equivalent. - Suggested reviewer: @Darktex
ALIGNMENT FLAG: reset() default task selection in generated wrappers
- Principle at stake: INVARIANTS.md — "Agents cannot access reset/simulation controls"; PRINCIPLES.md — "Agents cannot reset."
- The concern: The ORS wrapper's
reset()chooses "the first task from the first available split" when no selector is given. The RFC should be explicit that this default-selection logic lives exclusively in the server-side wrapper invoked by orchestration, that no agent-accessible code path can reachreset()directly or indirectly, and address the auto-reset-on-error scenario. - Suggested reviewer: @Darktex
ALIGNMENT FLAG: SUPPORTS_CONCURRENT_SESSIONS = False hardcoded in generated wrappers
- Principle at stake: PRINCIPLES.md — "One env = one trajectory."
- The concern: The generated wrapper contract introduces
SUPPORTS_CONCURRENT_SESSIONS, implying a concurrent-sessions concept exists or is planned. Introducing this attribute without an RFC explaining its semantics makes it an undocumented invariant. The RFC should explain what it is, where it is defined in the baseEnvironmentclass, and what behavior it controls — or remove it from the contract if it is not yet a real feature. - Suggested reviewer: @Darktex
Summary
- 6 mechanical issues to fix (RFC number collision is blocking; others must be addressed before merge)
- 4 alignment points for human review
The RFC number collision (shared with PR #795 and PR #624) is a hard blocker. The RFC-after-implementation pattern is a process-level concern that warrants explicit acknowledgment from @Darktex before this RFC is accepted.
Automated review by Claude Code | Learn more
This PR adds RFC 006 for deterministic external environment imports.
It documents the design behind PR #726, including the
openenv importCLI, importer registry, AST-based ORS/OpenReward and Verifiers detection, generated wrappers, task/split discovery, MCP-style tool calls, and the boundary rules needed to keep task discovery infrastructure-facing.It also updates the RFC index and changes the old roadmap reference to avoid reusing RFC 006 for a different future proposal.
Validation:
git diff --cached --checkbefore commit.