Repository navigation
TUI OOM: intermittent 24-28GB memory exhaustion in v2, no trigger identified #51761
Description
Description
The TUI exhausts all system memory and is OOM-killed. Growth is linear at roughly
500MB/s-1GB/s with no GC sawtooth, reaching 24-28GB in under a minute, after which
the process is killed by the OOM killer.
I have not found a reliable trigger. It happens intermittently, sometimes
apparently on its own after the session has been open for a while. Resizing the
window (including fullscreen) often precedes an occurrence, but a resize on a
large session frequently does nothing, so I do not believe resize is a
sufficient cause. I have not established a pattern.
This is specific to v2. I hit it repeatedly on v2.0.18 and did not hit it on v1.
I have not gone back to v1 to re-verify, so treat that comparison as my
impression rather than a controlled test, but every occurrence I captured was on
the v2 build.
OpenCode version
v2.0.18 (stock release binary, official install)
Steps to reproduce
I cannot reproduce this on demand. There is no trigger I can identify or
reliably trigger, and I am not able to hand over a sequence of steps that
reliably brings it back. Sorry to whoever tries to follow this.
What I observe, offered as a pattern to correlate against rather than a recipe:
- Long session history is always present (~2150 messages in my case).
- Tool output streaming is usually in progress.
- The window freeze that precedes the growth is the only consistent early
signal I have noticed. - Resizing, including fullscreen, often happens just before an occurrence. But
resizing a large session many times in a row does frequently nothing, so I do
not think resize is a sufficient cause. It may be incidental, or it may be
one of several contributing conditions. - Once growth starts it is terminal: no recovery, OOM within about a minute.
What I actively ruled out, in a synthetic PTY harness:
- Resize alone: an empty session resized 89 times stayed flat at ~190MB.
- Resize at high rate on a long session: 12 resizes/second stayed bounded, peak
383MB, with the allocation sawtooth clearly visible and being reclaimed.
So neither resize alone nor a large transcript alone brings this on. I do not
know what the missing ingredient is.
Operating System
Arch Linux 7.2.7 (x64), Hyprland Wayland compositor, kitty terminal
Terminal
kitty, TERM=xterm-256color
Plugins
None that I'm aware of.
Mechanism
From gdb attached to a live process at 27GB. The main thread is the only busy
thread; every other thread is idle:
Thread 1 (main):
#0 mmap64 () from libc.so.6
#1 0x7f564a0c53ba ?? /tmp/.bun-1001-*.so
#2 0x7f564a0cebfb ?? /tmp/.bun-1001-*.so
#3 0x7f564a10b8a5 ?? /tmp/.bun-1001-*.so
#4 0x7f564a10a144 ?? /tmp/.bun-1001-*.so
#5 0x7f564be79e34 ?? ()
Thread 15 (HTTP Client): syscall <- idle
Thread 17-19 (Bun Pool): syscall <- idle
The main thread is in a synchronous allocation loop that never yields to the
event loop. That is the most informative part of this report: a loop that
starves the event loop cannot be reacting to user input, since input arrives
through the event loop. It has to be self-sustaining or driven by a timer.
A practical consequence: the built-in profilers cannot observe this.
SIGUSR1 (packages/cli/src/heap.ts, heap snapshot) and SIGPROF
(packages/cli/src/cpu-profile.ts, CPU profile) both dispatch and produce no
output, because their handlers run on the event loop that the loop never
reaches. gdb was the only tool that produced usable data. If you want to
profile this class of bug, the signal-based profilers will not work.
Candidate directions
Both of these are unconfirmed hypotheses, offered only to narrow the search.
-
packages/tui/src/ui/animation.tshas a module-levelsetInterval(tick, 16)
that self-perpetuates while any task remains in a module-leveltasksSet.
Each surviving task callssetValueevery tick, driving re-renders with no
user input.onCleanup(stop)is registered insidecreateAnimatable, so if
it is ever called outside a Solid owner the cleanup is a no-op and the task
can never be removed. Accumulated tasks would fit the observed
non-yielding-loop signature and the apparent randomness. I have not shown
the set actually grows unboundedly. -
The tool renderers reprocess whole accumulated output per chunk, for example
stripAnsi(props.output?.trim())in
packages/tui/src/routes/session/index.tsx. Over a long stream this is
quadratic in the output length. This would need streaming plus a long
transcript, which matches the fact that neither alone reproduces.
Known cost on the resize path
#37860 already identifies that collapseToolOutput scans complete tool output
via split("\n") and Array.from when only a short prefix is displayed, and
measures a 661.9ms -> 0.136ms improvement for the same fix shape. That is known
work and I am not proposing it again. Confirming only that the same code is
reachable from width-dependent memos in v2, so every resize re-runs the scan
across tool rows. Making that scan prefix-bounded moved peak RSS during re-layout
from 383MB to 148MB in my synthetic harness, but it did not prevent the OOM,
so I do not believe it is the cause of this issue.
Caveats
- I have no deterministic reproduction. Everything above comes from observing
live processes, plus a synthetic harness that demonstrates what does not
trigger it. - I have not confirmed either candidate direction. The gdb evidence for a
non-yielding allocation loop is solid; the attribution to a specific file is
not. - The v2-only observation is based on my own usage, not a controlled test.
Activity
github-actions commentedon Sep 28, 2026
Thanks for updating the issue with the Bug Report details and reproduction context. It now meets our issue requirements.
[-]TUI OOM: unbounded memory growth on resize with long session and live streaming[/-][+]TUI OOM: intermittent 24-28GB memory exhaustion in v2, no trigger identified[/+]naeem76 commentedon Sep 28, 2026
I can confirm this is happening to me too, on Linux Mint.
I have not yet found something reproducible yet.
aasb13 commentedon Sep 28, 2026
Update: I captured the actual memory contents this time, and it rules out most of
the obvious theories. Flagging this early so nobody repeats the dead end I just
spent a night on.
First, the trigger. It is always a window resize. Every occurrence I have seen
follows a resize, including fullscreen, roughly 50 out of 50 times. Resize is
necessary, though not sufficient: resizing repeatedly often does nothing, so
something about the state of the session at the moment of the resize matters
and I have not isolated it.
Importantly, whether a session is streaming does not appear to matter. This
happens on a session that is completely stopped and idle, with nothing running.
Captured a live process at 2401MB via gdb
dump memory, which works on thefrozen process via ptrace. The in-process profilers (
SIGUSR1heap snapshot,SIGPROFCPU profile) cannot be used here at all, because the loop starves theevent loop that dispatches their handlers.
The memory is not text and not rendered output. Zero ASCII strings at any
offset sampled. So this is not tool output, not a transcript, not markdown, not
diff output, and not an OpenTUI cell buffer. Every string-processing theory I
had, including an O(n^2)
collapseToolOutput/stripAnsireprocessinghypothesis, is wrong.
It is a growing array of small JS heap objects. Sampling 64KB windows spread
across the large anonymous regions:
The 16-byte stride is consistent, values are small integers stored in the low
byte (
00000001x2575,00000002x1006,00000004x268,00000003x255),with occasional heap pointers. Window fill ranged 1.9% to 36.9%.
That is thousands of distinct, individually allocated objects with small integer
fields, laid out contiguously by the allocator. It is explicitly not a
repeating pattern, where one value would dominate into the thousands.
Not pixels either, since a renderer buffer was a reasonable guess: a 4-byte
group histogram gives
00000001x2575,00000002x1006. A pixel buffer wouldshow thousands of distinct RGB triplets, not clustered small integers.
Main thread at capture time was a single frame inside the Bun runtime, still the
only busy thread, still not yielding to the event loop.
So: a resize, on a session that may well be idle, causes the TUI to begin
growing a collection of ~16-byte JS objects that are never released. This is
consistent with a re-layout or re-render path on resize that rebuilds per-cell
or per-record state instead of updating it in place, and the growth rate scales
with how much of the session is mounted at the new size.
That is a hypothesis. I cannot name the responsible code from a memory sample;
doing so needs a heap snapshot taken while the process is still responsive,
which this loop prevents. A candidate worth checking is the resize-driven
re-layout path in the session view, and
packages/tui/src/ui/animation.ts, whichhas a module-level
setInterval(tick, 16)that self-perpetuates while any taskremains in a module-level
tasksSet. I have not shown either growsunboundedly.
One inconsistency worth recording: an earlier capture at 21GB showed a 1GB
[anon:JSJITCode]region, implying large-scale code generation. That region didnot appear in the top regions at 2.4GB here, so I am not confident it is a
stable feature of the leak.
bbartels commentedon Sep 29, 2026
@rekram1-node Hit this as well, caused my entire machine to lock-up, this is a pretty serious issue with OpenTUI 0.5.12. Detailed issue here: anomalyco/opentui#1543
peteretelej commentedon Sep 29, 2026
Experienced this a few times, here's anaylsis the most recent:
5 OOM kills in one day (v2.0.19, long-lived tmux TUIs, 37-56 GB each)
Environment: Ubuntu 22.04.5 LTS, kernel 5.15.0-191, opencode v2.0.19, tmux 3.2a, 62 GB RAM.
On 2026-09-29 the kernel OOM-killed
opencodefive times on this host: 01:56, 13:38, 22:25, 22:34, 22:41 (local time). Each victim held 37-56 GB of anonymous RSS. All five were long-lived TUI sessions hosted inside tmux (process ages from ~8.5 hours to 2-3 days); sessions started fresh idle at 120-390 MB. One of the incidents thrashed swap hard enough that the host was unreachable for about 10 minutes.Notes to help narrow the trigger:
eventtable: opencode.db reaches 13GB+, mostlymessage.updated.1snapshots (no retention/compaction) #33356 class) -~/.local/share/opencode/totals 293 MB (db 194 MB + 10 MB WAL), so this was RAM-only growth.@opentui/core0.5.10; v2.0.17 through v2.0.20 bundle 0.5.12. Our incidents were on v2.0.19 / 0.5.12.rekram1-node commentedon Sep 29, 2026
Opentui guys are investigating ty for detail!
juliojesus commentedon Sep 30, 2026
Same signature on Windows 11 with v2.0.19 (so not Linux/Wayland-specific), model GPT-6 Astra (OpenAI, medium), with a few data points that may help narrow it down:
medium), via the built-in OpenAI provider with ChatGPT OAuth. Both affected TUIs were on this model; the unaffected sibling TUIs included other sessions on the same model, so the model alone does not trigger it.opencode.exe --session <id>from apwsh.exepane.Stop-Process -Force. That is the Windows side of [BUG] Opencode process survives terminal close — missing SIGHUP handler causes orphaned process accumulation #12767 / Memory Leak: Orphaned processes when terminal closes without explicit exit #10563.serve --serviceprocess (0 CPU, stable); the only third-party plugin (uninstalling it and letting the TUI hot-unload it changed nothing).The idle + small-session combination makes me lean towards your first hypothesis (a timer or animation loop scheduling work outside an owner) rather than tool-output reprocessing. Happy to run a debug build or collect a dump next time it happens.
hallelujah-shih commentedon Sep 30, 2026
Another data point — v2.0.19 (official binary) on Fedora 44, kernel 7.2.7, x86_64, 30 GB RAM, all TUIs hosted in tmux.
18 kills in three days
Between 2026-09-28 and 2026-09-30 the system's
earlyoom(Fedora default,-m 4) sent SIGTERM to an opencode TUI 18 times. Every victim was a TUI process, VmRSS 13.8–26.3 GB:On a 30 GB box with earlyoom at a 4%-free threshold, a TUI that starts growing at ~1 GB/s is always going to win the kill race; I can't measure the growth rate directly from these logs, but victims include processes that had been up for only a few hours.
Resize/multiplexer correlation
All of these TUIs live in tmux, so every attach/detach, pane resize, or fullscreen toggle is a terminal resize delivered to a long-lived mounted session (many with tool output / diffs in the transcript). Consistent with "resize necessary but not sufficient" and with the off-screen-diff observation — I do not control when tmux resizes hit.
What I ruled out on this host
opencode serve --serviceprocess sat at ~545 MB RSS after 17 h. The runaway processes are the TUI clients, not the service.opencode.dbis 25.8 GB, butPRAGMA freelist_countshows 76% of that is dead freelist space (only ~3 GB live data acrossmessage1.4 GB +part1.8 GB), and the service process holding it is small. (The DB bloat itself looks like SQLite auto_vacuum is disabled, database files grow indefinitely #31526 / Unbounded growth of theeventtable: opencode.db reaches 13GB+, mostlymessage.updated.1snapshots (no retention/compaction) #33356 territory — separate issue.)Happy to capture anything (gdb memory dump, tmux config, earlyoom log slice) if it helps.
Addin commentedon Sep 30, 2026
@kommander @kitlangton @jlongster @adamdotdevin @thdxr @Hona @Brendonovich
Performance work before OpenCode 2 ships
I'd like to ask that most or all of the issues below be resolved before OpenCode 2 goes stable, and to get a straight answer on something first.
Are these being taken into account?
Because the v2 tracker is moving the wrong way, and I'd like to know whether that's because these reports are being absorbed, or because they aren't.
It flipped in August and the growth nearly tripled in September. The whole tracker did the same thing — it shed ~2,500 issues a month through May, then began adding: +609 in August, +1,158 in September.
So the question isn't whether these 97 are on the list. It's whether closing reports counts as progress when the list grows faster than it shrinks. And on that: 73 of these 97 were filed since 1 August — during exactly the window the v2 tracker turned — but only 9 of those 73 carry the
2.0label. 85 of the 97 carry no label at all.Which suggests they're falling outside the v2 triage view entirely, which is the opposite of being taken into account.
The ask
I work on older machines — not ancient, but past the point where a process holding 6–7 GB of RSS and spinning a render thread at 100% goes unnoticed. On that hardware OpenCode 2 is slower for me than OpenCode 1, which is the opposite of how 2 was announced. I don't think that's a hardware problem; I think it's these issues.
Where it actually hurts
SIGUSR1/SIGPROFprofiling produces nothing on this class of bug.serveRSS ratchets to 25–32 GB while the heap snapshot shows 172 MB of live JS. Allocator-side retention, not leaked objects.Two already have work in flight, which is encouraging: #49974 covers worker saturation, SQLite contention and event fanout; #51497 covers the O(n²) session-history re-read.
What would help most
Not all 97 need to land. The pattern underneath them is much smaller than the count:
message.updatedevent #43551, edit tool: levenshtein allocates full matrix on every call #47604.eventtable — six reports, largest database 64 GB, no cleanup path — event table grows unbounded (64GB of a 73GB DB in 8 days), eventually making POST /session time out #47223, Unbounded growth of theeventtable: opencode.db reaches 13GB+, mostlymessage.updated.1snapshots (no retention/compaction) #33356, opencode.db grows to ~9 GB due to enormous message.updated.1 event snapshots #46833, opencode.db grows unbounded - event table (4.1GB after ~6 months) has no retention policy or cleanup command #47729, event table bloat from message.updated.1 causes OOM when loading old sessions #32005, Durable event table bloats from repeated turn diff payloads #48641. [FEATURE]: Session Lifecycle Management — unified storage reclamation, auto-archival, and child session cleanup #16101 asks for exactly this.Full list
Components are heuristic, matched against title and body.
OOM — process killed, app dies (17)
message.updatedevent #43551 · DBHigh CPU — busy-loops, main-thread pinning, idle burn (33)
Memory — leaks and unbounded growth (47)
opencode <script>.js/ plugin-hook subprocess leaks a fresh 13.7 MB libopentui.so into $TMPDIR (17 GB in <2 h) #49283 · Plugineventtable: opencode.db reaches 13GB+, mostlymessage.updated.1snapshots (no retention/compaction) #33356 · DBCompiled 2026-09-30 from the open-issue tracker. Happy to help reproduce or profile any of these on old hardware if that's useful — that's the environment where they bite.
peteretelej commentedon Oct 3, 2026
@simonklee issue hasn't recurred for me since 2.0.21 which I think has the opentui bump #52233. thanks!
juliojesus commentedon Oct 3, 2026
Confirming the same on Windows 11: no recurrence since v2.0.21 (OpenTUI bump in #52233; now on v2.0.22).
Thanks @simonklee.
rekram1-node commentedon Oct 8, 2026
closing since it appears to be fixed.