Skip to content

add the Tokio transport adapter (#4766) - #4766

Closed
mariusae wants to merge 8 commits into
meta-pytorch:mainfrom
mariusae:export-D117242767
Closed

mariusae wants to merge 8 commits into
meta-pytorch:mainfrom
mariusae:export-D117242767

Conversation

@mariusae

@mariusae mariusae commented Aug 31, 2026

Copy link
Copy Markdown
Member

Summary:

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor spawn_client, spawn_server, spawn_duplex, or spawn_duplex_routed so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: Transport::spawn_* constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 31, 2026
@meta-codesync

meta-codesync Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

@mariusae has exported this pull request. If you are a Meta employee, you can view the originating Diff in D117242767.

mariusae added a commit to mariusae/monarch that referenced this pull request Sep 2, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
@meta-codesync meta-codesync Bot changed the title add the Tokio transport adapter add the Tokio transport adapter (#4766) Sep 2, 2026
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 2, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 2, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 2, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 4, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 10, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 10, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 10, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 11, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 12, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 12, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 12, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 12, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 14, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 14, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 15, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 15, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
mariusae added a commit to mariusae/monarch that referenced this pull request Sep 15, 2026
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
Summary:
Introduce the dependency-free wire types at the base of Chrysalis and document the architecture and review sequence. PIDs name process incarnations, while fixed-width QUIC connection IDs embed the destination PID and a process-local demultiplexing suffix.

Walkthrough: Start in `chrysalis-core`, where `Pid` defines process identity and `ConnectionId` defines the routable QUIC wire layout. Then read `chrysalis/DESIGN.md` for the complete model and `chrysalis/WALKTHROUGH.md` for the order in which the stack assembles it.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/58f98269-78c5-42b3-a1a4-6e9a7a1adaf2

Differential Revision: D117242759
Summary:
Add the carrier layer that moves atomic datagrams without interpreting QUIC. A common socket contract supports UDP, Unix datagrams, and in-process channels, while `DatagramSocketSet` selects a carrier from an opaque destination address.

Walkthrough: Read `datagram.rs` for the transport-independent address, receive, and transmit contracts. Then follow the concrete implementations in `udp.rs`, `unix.rs`, and `inproc.rs`; `socket_set.rs` composes those carriers behind one socket and `shutdown.rs` provides structured teardown.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/aebf6303-aaad-470c-ae40-e0a0f60550bf

Differential Revision: D117242760
Summary:
Route datagrams by the destination PID embedded in each QUIC connection ID. Routes reference lifecycle gates, so expiring one gate fences every derived destination without synchronously deleting a subtree of forwarding entries.

Walkthrough: Begin with `route.rs`: `RouteGate` supplies the shared fence, `Route` binds a destination to a next hop and gate, and `Router` performs forwarding. `switch.rs` then demultiplexes local PIDs and delegates every non-local datagram to that router without inspecting QUIC streams.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/15fc8742-10cf-4dbd-af8d-70a218e2d8c7

Differential Revision: D117242761
Summary:
Add the ownership and completion model shared by transport drivers and runtime adapters. Bounded ingress, completion credits, ordered submissions, operation IDs, and explicit wakeups make resource retention and backpressure part of the API without depending on Tokio or a QUIC implementation.

Walkthrough: `command.rs` and `submission.rs` define accepted work and ownership transfer. `completion.rs` defines every resource-return and lifecycle event. `credit.rs`, `queue.rs`, and `ingress.rs` enforce bounded admission, while `id.rs` and `notify.rs` provide driver-scoped identity and runtime-neutral wakeups.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/a39624e5-45e3-4696-8d96-77c9e2f95eb6

Differential Revision: D117242762
Summary:
Add a runtime-neutral UDP engine that owns stable packet slots on a dedicated driver thread. The engine submits batched receive and transmit operations through io_uring and exposes the packet-I/O boundary needed by the QUIC state machine.

Walkthrough: `config.rs` defines bounded ring and buffer sizing. `socket.rs` transfers a bound UDP socket into the engine. `driver.rs` owns the ring, receive and transmit slot lifecycles, GSO submissions, GRO splitting, completion processing, and structured shutdown.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/96ad08dc-3b3a-4fb4-9c4f-59d5a9415577

Differential Revision: D117242763
Summary:
Define the final identity, admission, buffer-ownership, command, and packet-I/O boundaries used by the runtime-neutral quiche endpoint. The crate binds quiche to Chrysalis CIDs and completion transport without introducing an async runtime.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/6ae92406-f8be-46ae-8b73-6d4b869276c2

Differential Revision: D117242765
Summary:
Implement the final runtime-neutral quiche endpoint over bounded completion transport and packet slots. The driver authenticates certificate-derived PIDs, routes fixed CIDs directly, bounds server admission and retained completions, schedules only runnable streams, preserves directional operation order, returns ownership on cancellation, and reclaims terminal state.

When a client server name is an IP literal, create the quiche connection without DNS SNI and call `Connection::set_host_ip_addr` before the first packet. This makes verified connections match the certificate `iPAddress` SAN. DNS names retain the existing SNI and DNS SAN path. Exercise the IP-literal path with a verified end-to-end IPv6 handshake.

Walkthrough: `driver.rs` owns endpoint admission, completion credits, shutdown, packet-loop orchestration, and the lifecycle overview. `driver/network.rs` owns CID routing, Retry, connection admission, packet generation, and lifecycle statistics. `driver/connection.rs` owns authenticated connection state plus ordered send and receive queues, runnable-stream scheduling, cancellation, half-state transitions, and reclamation. `driver/tests.rs` exercises handshake, multiplexing, backpressure, shutdown, authentication, cancellation, and stream control.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/c932f625-1719-4ce0-9502-6074ffcb7f68

Reviewed By: shayne-fletcher

Differential Revision: D117242766
Summary:
Pull Request resolved: meta-pytorch#4766

Translate the completion-driven Chrysalis transport into ergonomic Tokio connections and streams without moving protocol ownership into the runtime. The adapter preserves bounded admission, caller buffer ownership, operation cancellation, and structured driver shutdown. Name every transport constructor `spawn_client`, `spawn_server`, `spawn_duplex`, or `spawn_duplex_routed` so the API makes creation of the driver thread and completion pump explicit.

Walkthrough: `Transport::spawn_*` constructs the low-level endpoint, starts its dedicated driver thread, and starts the Tokio completion pump. Connection and stream handles translate command receipts and operation completions into Tokio futures while cancellation remains an explicit message to the runtime-neutral driver.

Design document: https://mdoc.internalmeta.com/doc/e6918c35-b3d1-4118-ac39-6308b0cb55ba

Walkthrough document: https://mdoc.internalmeta.com/doc/1c2c7a60-06fb-493d-9ce2-96f18e099b75

Reviewed By: shayne-fletcher

Differential Revision: D117242767
@meta-codesync

meta-codesync Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

This pull request has been merged in 1c51813.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant