Title
zsh history parser: formats, unmetafy, multiline, caps
Summary
Implement the pure parsing layer for zsh history files: unmetafication,
simple + extended format detection, multiline entry joining, and the
size/line caps — exactly per docs/research/zsh-history-format.md.
Context
This is parsing only (bytes → entries); provider glue (date filtering,
config, warnings) is issue 10. The reference machine's history is
simple-format (no timestamps), and oh-my-zsh machines are extended-format;
both must parse. Metafied bytes (0x83 escapes) corrupt Japanese commands if
skipped — this is the most commonly missed zsh detail.
Scope
internal/source/zsh/parser.go + tests + testdata/ fixture files.
Detailed Requirements
type Entry struct { Start time.Time; Duration time.Duration; Command string; HasTimestamp bool }.
Unmetafy([]byte) []byte — exact algorithm from the research doc
(0x83 marker, next byte XOR 0x20; trailing lone 0x83 dropped). Operates
on raw bytes BEFORE any string conversion.
ParseReader(r io.Reader, caps Caps) (entries []Entry, stats Stats):
Caps{MaxLineBytes int (default 64*1024), MaxFileBytes int64 (default 128*1024*1024)}; Stats{Lines, Oversized, Malformed int; Truncated bool; ExtendedCount, SimpleCount int}.
- Streaming scan (
bufio.Reader, manual line assembly — bufio.Scanner
token limits must not abort the file: an oversized line is consumed,
counted in Oversized, and skipped).
- Per physical line: unmetafy → UTF-8 repair (
sanitize.Clean is NOT
applied here; only strings.ToValidUTF8 — commands keep their tabs
etc.; display hygiene happens downstream) → multiline joining: while
the line ends with an odd number of \, strip the final \, append
\n, read next physical line (a line starting with ": " while in
extended mode terminates a malformed continuation; count Malformed).
- Entry header:
^: (\d{10,}):(\d+);(.*)$ → extended entry
(HasTimestamp=true, Start from epoch, Duration seconds); otherwise
the whole line is a simple entry (HasTimestamp=false).
- Stop reading at
MaxFileBytes (set Truncated).
- Empty lines are skipped (not Malformed).
- No file I/O in this file beyond the
io.Reader (testability); no
goroutines.
Acceptance Criteria
Validation
go test -race -cover ./internal/source/zsh/ and 30s fuzz run output in the
PR. Reviewer check: Unmetafy table includes the byte pairs
0x83 0xC3 → 0xE3 (start of Japanese UTF-8) and lone-trailing-0x83.
Dependencies
01, 06 (only for shared constants if any; parser itself uses
strings.ToValidUTF8 — keep sanitize out of the parse path).
Non-goals
Date filtering, HISTFILE resolution, warnings, Events (all issue 10);
bash/fish formats (v2).
Design References
docs/research/zsh-history-format.md (normative format spec)
docs/DESIGN.md §7.1
Source of truth: docs/issues/09-zsh-history-parser.md (PR #1, branch docs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies: docs/ISSUE_PLAN.md (this is issue 09 of 33).
Title
zsh history parser: formats, unmetafy, multiline, caps
Summary
Implement the pure parsing layer for zsh history files: unmetafication,
simple + extended format detection, multiline entry joining, and the
size/line caps — exactly per
docs/research/zsh-history-format.md.Context
This is parsing only (bytes → entries); provider glue (date filtering,
config, warnings) is issue 10. The reference machine's history is
simple-format (no timestamps), and oh-my-zsh machines are extended-format;
both must parse. Metafied bytes (0x83 escapes) corrupt Japanese commands if
skipped — this is the most commonly missed zsh detail.
Scope
internal/source/zsh/parser.go+ tests +testdata/fixture files.Detailed Requirements
type Entry struct { Start time.Time; Duration time.Duration; Command string; HasTimestamp bool }.Unmetafy([]byte) []byte— exact algorithm from the research doc(0x83 marker, next byte XOR 0x20; trailing lone 0x83 dropped). Operates
on raw bytes BEFORE any string conversion.
ParseReader(r io.Reader, caps Caps) (entries []Entry, stats Stats):Caps{MaxLineBytes int (default 64*1024), MaxFileBytes int64 (default 128*1024*1024)};Stats{Lines, Oversized, Malformed int; Truncated bool; ExtendedCount, SimpleCount int}.bufio.Reader, manual line assembly —bufio.Scannertoken limits must not abort the file: an oversized line is consumed,
counted in
Oversized, and skipped).sanitize.Cleanis NOTapplied here; only
strings.ToValidUTF8— commands keep their tabsetc.; display hygiene happens downstream) → multiline joining: while
the line ends with an odd number of
\, strip the final\, append\n, read next physical line (a line starting with": "while inextended mode terminates a malformed continuation; count
Malformed).^: (\d{10,}):(\d+);(.*)$→ extended entry(
HasTimestamp=true, Start from epoch, Duration seconds); otherwisethe whole line is a simple entry (
HasTimestamp=false).MaxFileBytes(setTruncated).io.Reader(testability); nogoroutines.
Acceptance Criteria
extended.hist: ≥ 5 extended entries incl. one withduration > 0; timestamps parse to the exact epochs.
simple.hist: plain commands; allHasTimestamp=false.metafied.hist: contains a Japanese command stored with real0x83 metafication (fixture built by a small Go generator in
testdata/gen/main.go, committed with its output); parsed Commandequals the original Japanese string.
multiline.hist: an extended entry whose command spans 3physical lines via trailing backslashes → one Entry with two
\n.torn.hist: file ending mid-entry (no trailing newline,dangling continuation) → parsed without error; tail counted Malformed
or yielded as-is (documented choice: count Malformed, drop).
Oversized==1, followingentries still parse.
FuzzParseReaderfuzz target (seeded with all fixtures) runs 30slocally with no panics and no invalid-UTF-8 Command output.
Validation
go test -race -cover ./internal/source/zsh/and 30s fuzz run output in thePR. Reviewer check:
Unmetafytable includes the byte pairs0x83 0xC3 → 0xE3(start of Japanese UTF-8) and lone-trailing-0x83.Dependencies
01, 06 (only for shared constants if any; parser itself uses
strings.ToValidUTF8— keep sanitize out of the parse path).Non-goals
Date filtering, HISTFILE resolution, warnings, Events (all issue 10);
bash/fish formats (v2).
Design References
docs/research/zsh-history-format.md(normative format spec)docs/DESIGN.md§7.1Source of truth:
docs/issues/09-zsh-history-parser.md(PR #1, branchdocs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies:docs/ISSUE_PLAN.md(this is issue 09 of 33).