Skip to content

htsserver never returns from main(), it blocks on its own exit wait - #757

Merged
xroche merged 2 commits into
masterfrom
fix/webhttrack-exit-wait
Jul 27, 2026
Merged

htsserver never returns from main(), it blocks on its own exit wait#757
xroche merged 2 commits into
masterfrom
fix/webhttrack-exit-wait

Conversation

@xroche

@xroche xroche commented Jul 27, 2026

Copy link
Copy Markdown
Owner

htsserver does not return from main(). It reaches its exit wait and blocks there, and has done since 2014.

background_threads counted two unrelated things: the --ppid pinger, which only ever leaves through exit(), and each mirror thread, which does return. main() waited on background_threads - 1. With no pinger that is htsthread_wait_n(-1), a negative number of outstanding threads the counter can never reach; with a pinger it is htsthread_wait_n(0), which waits on the pinger itself. Both hang, 10 runs out of 10.

So the counter now holds only what never returns, and each wait names what it must leave running: the pinger for main(), the pinger plus itself for the mirror thread. Just dropping the - 1 is not enough, which is how I had it first. Since background_threads also counted the joinable mirror threads it equalled the live count exactly, so main() would have sailed past the wait while a mirror was still crawling and torn down under it. Review caught that one. Not counting mirrors at all is what makes both call sites correct, and it also closes a latent under-wait inside back_launch_cmd(), where two sequential mirrors left the counter excluding one thread too many and the engine-thread drain got skipped.

Measured with an html root that has no lang.def, so the server fails right after announcing and main() reaches the wait on its own: 10/10 hangs on master with and without --ppid, 0/10 with the fix. The regression case lives in tests/65_port-siblings.test, which already drives htsserver. It fails on master with exited 124, want 1. It asserts the exact status rather than "not a timeout" because a crash or an assertf abort also escapes the wait and would otherwise read as a pass. What it cannot reach from a shell is the case where a mirror is still running at exit, so that leg rests on the review's per-case reading, not on a test.

Worth noting that #752 made this worse rather than caused it. Counting a thread from the moment it is spawned closed the window the pinger used to slip through, taking the --ppid case from an intermittent hang to a certain one.

Closes #753

xroche and others added 2 commits July 27, 2026 08:11
htsthread_wait_n(background_threads - 1) subtracts one more than the count of
threads that must not be joined. Without --ppid that count is zero, so the wait
asks for a negative number of outstanding threads and the counter never gets
there; with --ppid it waits on the pinger, which by design never returns.

Wait for background_threads instead, which is what the sibling call inside
back_launch_cmd() already passes.

Closes #753

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
background_threads counted two unrelated things: the pinger, which only ever
leaves through exit(), and each mirror thread, which does return. main() then
waited on background_threads - 1, so with no pinger it asked for a negative
number of outstanding threads and the counter cannot go there, and with one it
waited on the pinger itself. Either way htsserver hung instead of returning.

Count only what never returns, and let each wait name what it must leave
running: the pinger for main(), the pinger plus itself for back_launch_cmd().
Waiting on background_threads alone would have swapped the hang for an early
return, letting main() tear down while a mirror was still crawling.

Closes #753

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
@xroche

xroche commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

Noting a second defect this PR closes, so it does not get lost if the fix is reshaped.

The counter is genuinely raced on master. webhttrack_main() increments background_threads at htsweb.c:385, right after creating the mirror thread at :384, and that mirror thread reads the same variable at :427. The write lands after the reader already exists, with no lock and nothing else ordering them, so it is an unsynchronised concurrent access to a plain int. Benign on the platforms we build for, but a data race by the memory model.

This PR removes it instead of papering over it. Afterwards there is exactly one write, at :304 on main's thread before help_server() runs, and two reads: :314 on main's own thread, and :430 on the mirror thread, which main creates later. Thread creation supplies the happens-before, so the counter is write-once-before-spawn and needs no lock. What keeps it safe is that no write happens after any thread that reads it has been created, so putting a per-mirror increment back would bring the race back with it.

@xroche
xroche merged commit 29dfd2d into master Jul 27, 2026
23 of 24 checks passed
@xroche
xroche deleted the fix/webhttrack-exit-wait branch July 27, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

webhttrack waits forever at exit when no mirror was launched (off-by-one in htsthread_wait_n)

1 participant