Skip to content

Preserve installed filtering when blocklist downloads fail - #30

Open
venzellucas wants to merge 2 commits into
mossdef-org:mainfrom
venzellucas:fix/preserve-list-on-download-failure
Open

venzellucas wants to merge 2 commits into
mossdef-org:mainfrom
venzellucas:fix/preserve-list-on-download-failure

Conversation

@venzellucas

@venzellucas venzellucas commented Nov 11, 2026 •

Copy link
Copy Markdown

A failed refresh can replace a working blocklist with only locally configured/canary domains. I encountered this on 1.2.4-r4 during a WAN interruption: both enabled sources failed to download, but the resulting two iCloud canary rules passed validation and were saved to the persistent cache, replacing a list of roughly 365,000 domains.

This change defers resolver cleanup until every enabled source has downloaded and parsed successfully. A failed or missing download result aborts the refresh, removes temporary processing files, reports statusFail, and leaves the existing list, caches, and resolver configuration intact. It also propagates failures from the external dnsmasq configuration download path. Any enabled source failure cancels the refresh, including allow-list failures, to avoid installing an incomplete policy.

The existing output is retained without copying it. Download memory checks use available RAM while that output is still present; old files are released before sorting and optimization. This limits the additional memory requirement to the download phase and avoids retaining two processed lists throughout the processing stage.

Validation:

  • bash tests/run_tests.sh: all 24 test groups pass using a locally built ucode runtime, following the repository CI setup.
  • New lifecycle regressions cover total and partial download failures, malformed input, missing parallel results, simulated parallel aggregation, no-cache and first-start failures, external dnsmasq download failure, successful retry, insufficient memory, missing/empty source URLs in both download paths, CLI failure diagnostics, and release of shared dnsmasq output before sorting.
  • The original failure regressions fail against the original implementation; the new review regressions also fail against the previous PR revision.
  • bash -n tests/run_tests.sh and git diff --check pass.

The package release is bumped to 1.2.5-r4.

This is scoped to download/parsing failures and memory-budget rejection. Rollback for later processing or resolver-health failures, and transactions spanning a DNS backend change, remain outside this patch. No production-device fault injection was performed.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An enabled source with a missing URL is silently skipped rather than reported as an error, allowing the refresh to replace the installed policy with only local/canary rules instead of aborting.

3 open issues
What changed in this PR

This PR updates adblock-fast’s refresh lifecycle to preserve installed filtering and caches when downloads, parsing, or memory-budget checks fail.

Changes:

  • Defers resolver cleanup until downloads succeed and propagates failures to the start path.
  • Adds regression coverage for failed refreshes, parallel results, successful retries, and insufficient memory.
  • Exposes a test-only event-loop setter to simulate parallel downloads.
File Description
tests/​run_tests.sh Injects the event-loop setter used by parallel-download tests.
tests/​03_functional/​06_low_memory_refresh Checks that memory rejection preserves output and caches without restarting the resolver.
tests/​03_functional/​05_failed_refresh Tests refresh failures, preserved filtering, and successful recovery.
files/​lib/​adblock-fast/​adblock-fast.uc Defers cleanup, propagates download failures, and aborts failed refreshes.

Review effort: Balanced


// Do not let local/canary domains turn failed downloads into a successful
// refresh. A partial allow-list is unsafe too, so require all sources.
if (length(status_data.errors) > errors_before) {

// Downloads are complete: release the old files before the memory-heavy
// processing stage, preserving the existing peak-memory strategy.
resolver('cleanup');
Comment on lines +2806 to +2807
status_data.status = 'statusFail';
return _build_procd_data();
@stangri

stangri commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Thank you for the contribution @venzellucas, I wonder if you could address copilot's comments. Feel free to bump the release number when you do.

@venzellucas

Copy link
Copy Markdown
Author

Thank you for reviewing this, @stangri! I've addressed all three Copilot comments in 68dea5a:

  • Missing or empty URLs on enabled sources now report a download error and abort the refresh.
  • The shared dnsmasq output is explicitly removed after downloads succeed, before sorting and optimization.
  • The early failure path now reports the recorded errors and warnings through status_service('on_start_failure').

I've also bumped the package release to 1.2.5-r4, as you suggested, and added regression coverage for these cases. All 24 test groups pass locally, and the new review regressions fail against the previous PR revision.

Thanks again for your time and guidance. Happy to make any further adjustments you'd like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants