Skip to content

docs: explain how to parallelize work within a single stream - #810

Merged
dbrattli merged 2 commits into
masterfrom
docs/parallelism-within-stream
Jul 27, 2026
Merged

docs: explain how to parallelize work within a single stream#810
dbrattli merged 2 commits into
masterfrom
docs/parallelism-within-stream

Conversation

@dbrattli

Copy link
Copy Markdown
Collaborator

Follow-up to #713.

Problem

The "CPU Concurrency" section of the getting-started guide demonstrates concurrency across three independent pipelines, but never states that the items of a single pipeline are delivered one at a time. Readers reasonably conclude that adding observe_on(ThreadPoolScheduler(n)) will process items in parallel, then report the resulting sequential execution as a scheduler bug.

This is by design — ScheduledObserver drains one notification at a time to honour the Rx contract that on_next never overlaps — but nothing in the docs says so. It has come up repeatedly: #713, #67, #100, #528.

Changes

  • docs/get_started.rst — new "Parallelizing Work Within a Single Stream" section showing the sequential-vs-parallel comparison and the flat_map fan-out pattern (both the just(...) + subscribe_on form and the more direct from_callable(..., scheduler=...) form), plus a note on the concat_map ordering trade-off and merge(max_concurrent=...).
  • docs/get_started.rst — an .. important:: admonition in the existing subscribe_on/observe_on discussion, linking to the new section.
  • reactivex/operators/_observeon.py — matching note in the observe_on docstring, so the clarification is reachable from the API reference and from IDE tooltips.

No library behaviour changes.

Verification

Every snippet was executed against this branch. With a 2-second intense_calculation over three items:

wall clock order
observe_on(pool) + map 6.0s preserved
flat_map + subscribe_on(pool) 2.0s completion order
flat_map + from_callable(..., scheduler=pool) 2.0s completion order
concat_map + from_callable(..., scheduler=pool) 6.0s preserved

which matches what the new section claims, including the ordering caveat.

sphinx-build -W --keep-going produces 67 warnings on this branch and 67 on master — no new ones. ruff check, ruff format --check and pyright pass clean on the changed module, and tests/test_observable/test_observeon.py passes.

🤖 Generated with Claude Code

The "CPU Concurrency" guide only demonstrated concurrency across
independent pipelines, and never stated that the items of a single
pipeline are delivered one at a time. Readers reasonably concluded that
adding observe_on(ThreadPoolScheduler(n)) would process items in
parallel, and reported the resulting sequential execution as a bug.

Add a "Parallelizing Work Within a Single Stream" section showing the
flat_map fan-out pattern, note the ordering and max_concurrent
trade-offs, and cross-reference it from the observe_on discussion and
docstring.

Closes #713. Same underlying question as #67, #100 and #528.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coveralls

coveralls commented Jul 27, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 93.495%. remained the same — docs/parallelism-within-stream into master

@dbrattli
dbrattli merged commit 93a4f04 into master Jul 27, 2026
43 checks passed
@dbrattli
dbrattli deleted the docs/parallelism-within-stream branch July 27, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants