Skip to content

htsthread_wait() could return before a thread it should join had started - #752

Merged
xroche merged 1 commit into
masterfrom
fix/htsthread-wait-race
Jul 27, 2026
Merged

htsthread_wait() could return before a thread it should join had started#752
xroche merged 1 commit into
masterfrom
fix/htsthread-wait-race

Conversation

@xroche

@xroche xroche commented Jul 27, 2026

Copy link
Copy Markdown
Owner

process_chain, the counter htsthread_wait() polls, was raised by the child inside hts_entry_point(). A caller that spawned threads and waited straight away saw zero outstanding threads and returned at once, free to tear down state the children were about to read. hts_newthread() now raises the count before the thread can run, under the same mutex the waiter reads, and drops it again if creation failed. The invariant it buys: from the moment hts_newthread() returns success until the spawned function has returned, that thread is counted, so htsthread_wait_n(n) returns only once at most n of the threads spawned before the call are still outstanding.

httrack.c freed the option block before waiting, so the second hunk swaps those two lines. That is not just tidiness: back_launch_ftp caches opt for the life of the transfer and writes through it in file_notify() and opt->state.exit_xh, well after hts_free_opt() had already run hts_mutexfree(&opt->state.lock) and freed the struct.

Nothing installed changes, so there is no ABI question: no header, no struct, no exported prototype, and nm -D gives the same 165 symbols on both sides.

Proof is a ThreadSanitizer probe, out of tree. Children own disjoint slots so they cannot race each other, and the parent writes and frees the shared block once htsthread_wait() returns. Pinned to one CPU it fires on unfixed code in 5 runs out of 5, with half the rounds returning early, and is clean in 5 out of 5 with the fix. The control is the same probe unpinned, where the window has already closed by the time the wait runs: clean either way, which is what says the probe reports the bug and not an unconditional race. My first version of it was falsely armed, incidentally, reporting a race between the children themselves.

No in-tree regression test, which I am not happy about. Registering a -#test name needs htsselftest.c and a new tests/NN_*.test needs tests/Makefile.am, and #718 holds both. The self-test that tripped over this in the first place still joins on a counter of its own; that workaround and a proper test can go in together once #718 lands.

One thing found and left alone: htsweb.c:310 waits with htsthread_wait_n(background_threads - 1), one short of the non-joinable threads it means to exclude, so webhttrack's "no remaining port" path waits forever on a pinger thread that never returns. Master already hangs there 4 times in 10; counting at spawn makes it 10 in 10. Separate bug, separate PR.

Closes #747

process_chain was incremented by the child in hts_entry_point(), so a caller
that spawned threads and immediately called htsthread_wait() saw a zero count
and returned at once, free to tear down state the children were about to read.

Count at spawn instead, under the same mutex the waiter reads.

httrack.c freed the option block before waiting; wait first.

Closes #747

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
@xroche
xroche merged commit 9571fb9 into master Jul 27, 2026
22 checks passed
@xroche
xroche deleted the fix/htsthread-wait-race branch July 27, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

htsthread_wait() can return before a thread it should wait for has started

1 participant