Expose the engine's sitemap, single-file and change-report options - #71
Merged
Conversation
The engine takes --sitemap/--sitemap-url, --single-file with a per-asset byte cap, and --changes, but nothing in the GUI could reach them. Sitemap seeding goes on the Spider page; the two output options go on Log/Index/ Cache, beside the WARC and mail-archive toggles users will weigh them against. Each one is carried through the argv build and both profile paths. Labels and tooltips read the engine's lang.def keys, so they follow the installed language instead of staying English. SetDlgItemTextLang() keeps the .rc caption when a language file predates a key, since LANGSEL() returns "" for one it does not carry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
A value restored from a profile never passes the dialog's validation, and the engine aborts the whole mirror on an argument that is over-long or that starts with a dash, which it reads as the argument being missing. Check both at argv time, where every path goes through, measuring the UTF-8 bytes the engine will actually receive rather than the ANSI characters here. The sitemap checkbox now gates its address field too, so an unticked box cannot seed a crawl through text left behind in the field, and an address given there replaces the robots.txt probe instead of adding to it, which is what its tooltip promises. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine takes
--sitemap/--sitemap-url,--single-filewith a per-asset byte cap, and--changes, but nothing in the Windows GUI could reach them. Sitemap seeding goes on the Spider page next to the robots policy it piggybacks on. The two output options go on Log/Index/Cache, where-%Malready sits as "Build a mail archive" and where the WARC toggle was moved for the same reason: those are what users will weigh them against.Labels and tooltips read the engine's
lang.defkeys, so all 30 locales get them instead of the English-only text the WARC checkbox shipped with. The byte cap needs no bounds guard of its own, since the engine quietly keeps its default on anything it cannot parse. Both pages grew taller but stay under the Limits page's 198 units, so the property sheet is unchanged.The engine side landed this morning: xroche/httrack#718, #720 and #721.