Skip to content

session: don't park request loop on window-change, bound env churn - #15

Merged
raggi merged 1 commit into
masterfrom
raggi/gliderssh-window
Oct 25, 2026
Merged

raggi merged 1 commit into
masterfrom
raggi/gliderssh-window

Conversation

@raggi

@raggi raggi commented Oct 25, 2026

Copy link
Copy Markdown
Member

An authenticated client could exhaust memory in the server process with per-session state that persisted after disconnect, in two ways:

  1. A window-change request did a blocking send on the session's window channel. The channel has capacity for one window size and the initial pty-req already fills it, so if the session handler never drains the channel returned by Pty(), the very first window-change parks the handleRequests goroutine forever. The parked goroutine, the session, and everything it references (such as accumulated environment variables) leak even after the client disconnects, and the request loop stops servicing any further requests for the session.

  2. Before a shell or exec request was handled, every env request appended an arbitrarily sized "key=value" string to the session's environment with no cap on count or size, so an env churn loop grew the heap without bound.

Window-change sends are now non-blocking and keep-latest: when the consumer is not keeping up, the stale buffered value is replaced with the newest window size, so a slow consumer eventually observes the most recent size and the request loop never blocks. Break requests had the same blocking send on the registered break channel and are now dropped (replied false) when the consumer is not keeping up. Env requests are rejected once a session has maxEnvs (128) variables.

While here, fix a data race between handleRequests writing the pty window size and the handler goroutine reading it via Pty(): both now take the session mutex, which previously went unused for pty state.

Thanks to the security audit that identified the memory exhaustion condition (approximately 536 MB allocated in 7 seconds of churn, persisting long after disconnect when a parked goroutine retained the session).

Updates tailscale/corp#50380

An authenticated client could exhaust memory in the server process
with per-session state that persisted after disconnect, in two ways:

1. A window-change request did a blocking send on the session's
   window channel. The channel has capacity for one window size and
   the initial pty-req already fills it, so if the session handler
   never drains the channel returned by Pty(), the very first
   window-change parks the handleRequests goroutine forever. The
   parked goroutine, the session, and everything it references (such
   as accumulated environment variables) leak even after the client
   disconnects, and the request loop stops servicing any further
   requests for the session.

2. Before a shell or exec request was handled, every env request
   appended an arbitrarily sized "key=value" string to the session's
   environment with no cap on count or size, so an env churn loop
   grew the heap without bound.

Window-change sends are now non-blocking and keep-latest: when the
consumer is not keeping up, the stale buffered value is replaced with
the newest window size, so a slow consumer eventually observes the
most recent size and the request loop never blocks. Break requests
had the same blocking send on the registered break channel and are
now dropped (replied false) when the consumer is not keeping up.
Env requests are rejected once a session has maxEnvs (128) variables.

While here, fix a data race between handleRequests writing the pty
window size and the handler goroutine reading it via Pty(): both now
take the session mutex, which previously went unused for pty state.

Thanks to the security audit that identified the memory exhaustion
condition (approximately 536 MB allocated in 7 seconds of churn,
persisting long after disconnect when a parked goroutine retained the
session).

Updates tailscale/corp#50380
@raggi
raggi merged commit 8d94c04 into master Oct 25, 2026
4 checks passed
raggi added a commit that referenced this pull request Oct 26, 2026
The nonblocking window-change path only retried its send when it drained
the stale buffered value itself. If the PTY consumer drained that value
first, the latest size was silently lost.

Retry the send after either drain outcome while keeping channel operations
nonblocking. Add a per-session test gate and a deterministic regression
test for the consumer-drains-before-discard interleaving.

Fixes: #15 / 6662764 (session: don't park request loop on window-change, bound env churn)
raggi added a commit that referenced this pull request Oct 26, 2026
The nonblocking window-change path only retried its send when it drained
the stale buffered value itself. If the PTY consumer drained that value
first, the latest size was silently lost.

Retry the send after either drain outcome while keeping channel operations
nonblocking. Add a per-session test gate and a deterministic regression
test for the consumer-drains-before-discard interleaving.

Fixes: #15 / 6662764 (session: don't park request loop on window-change, bound env churn)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants