Close worker-thread connections opened in _send_bulk#523
Open
jorikbraet wants to merge 1 commit into
Open
Conversation
PR ui#521 moved the email backend connection open into ThreadPool workers (`_send_email`), but `ConnectionHandler` stored connections on a `threading.local`. The `connections.close()` called from the main thread in `_send_bulk`'s finally only saw the main thread's local, so SMTP connections opened on workers were never `.quit()`-ed — they got garbage-collected later, producing `ResourceWarning: unclosed <ssl.SSLSocket ...>` and leaking FDs in long-running queue workers. Switch `ConnectionHandler` to a process-wide dict keyed by `(thread_ident, alias)` under a lock. Per-thread isolation is preserved (threads still get distinct connections), and `close()` now reaches every worker's connection. The existing intra-batch reuse from ui#521 is preserved — `test_send_bulk_reuses_open_connection` still passes (2 opens for 3 emails with THREADS_PER_PROCESS=1). Adds `test_send_bulk_closes_worker_connections` which fails on master (opens=2, closes=1) and passes after this change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PR #521 (released in 3.11.2) moved the email-backend connection open into the ThreadPool workers inside _send_bulk:
But ConnectionHandler stored its registry on a threading.local, so the connections.close() called from the main thread in _send_bulk's finally only saw the main thread's local cache. SMTP connections opened on the worker threads were never .quit()-ed. They survived until garbage collection, producing:
and leaking file descriptors in long-running send_queued_mail workers.
Fix
Switch ConnectionHandler from threading.local to a process-wide dict keyed by (threading.get_ident(), alias) under a lock. Per-thread isolation is preserved (each thread still gets its own connection, no sharing or contention), and close() called from any thread now sees and closes every worker's connection.
This preserves the intra-batch connection reuse from #521. _send_email running multiple times on the same worker thread still hits the cache rather than reopening. The existing test_send_bulk_reuses_open_connection still passes at 2 opens for 3 emails (1 main + 1 worker, reused).
I considered the smaller fix of wrapping _send_email's body in a try/finally that calls connections.close() per call. That works but defeats #521's reuse: every email pays a fresh open() (plus STARTTLS, for SMTP). The registry change is the same number of files touched and keeps the perf shape #521 was after, so it seemed the right call. Happy to switch to the per-call close if you prefer the smaller surface.
Test
Added test_send_bulk_closes_worker_connections: a thread-safe counting backend tracks open() and close() calls; the test runs _send_bulk over 5 emails with THREADS_PER_PROCESS=2 and asserts opens == closes.
Full test suite still green locally on Django 5.2 (django-admin test post_office ./).