Sitemap files are never read, so URLs nothing links to are never found - #718

Merged

xroche merged 21 commits into

Jul 27, 2026

Commits on Jul 26, 2026

Read sitemap files so URLs nothing links to are found

xroche
and
claude
committed
Merge remote-tracking branch 'origin/master' into feat/sitemap
xroche
committed
sitemap: distinguish the sitemapindex log line from a urlset one

xroche
and
claude
committed
Merge remote-tracking branch 'origin/master' into feat/sitemap
xroche
committed
sitemap: fix an out-of-bounds read, tighten the parser and the tests

xroche
and
claude
committed
sitemap: state what the decompression cap actually binds on

xroche
and
claude
committed
sitemap: a child sitemap is a fetch, so filters and robots.txt must gate it

xroche
and
claude
committed
Merge remote-tracking branch 'origin/master' into feat/sitemap
xroche
committed
sitemap: gate each fetch by who asked for it, and anchor travel on the start URL

xroche
and
claude
committed
sitemap: add the fuzz harness the parser was missing, and drop truncated Sitemap: lines

xroche
and
claude
committed
fuzz: keep only the four sitemap seed inputs

xroche
and
claude
committed
selftest: bound the sitemap document builders' snprintf accumulation

xroche
and
claude
committed
Merge remote-tracking branch 'origin/master' into feat/sitemap
xroche
committed
sitemap: date the new files 2026

xroche
and
claude
committed
Merge remote-tracking branch 'origin/master' into feat/sitemap
xroche
committed
sitemap: keep the ingestion state out of htsoptstate

xroche
and
claude
committed