Phase 2: chunk .rxp literals to ≤63 bytes for bytecode compilation - #5
Phase 2: chunk .rxp literals to ≤63 bytes for bytecode compilation#5mgrossmann wants to merge 3 commits into
Conversation
Transpiled .rxp pages emitted one REXX string literal per HTML run. rexx370's bytecode compiler caps each constant at 63 bytes (IRXBC_STR_MAX), so ordinary HTML lines forced IRXBC_ERR_STRTOOLONG and a fallback to the token-walk interpreter on nearly every page -- a doomed compile paid on every request. line_close_lit now splits each literal into <=63 value-byte chunks joined by ||, keeping every constant compilable. Chunk boundaries are counted in value bytes (an embedded quote is one byte, written doubled); output is byte-identical to one long literal, and literals under 63 bytes are unchanged. Transpiler-only; .rexx direct execution is unaffected. Measured on a realistic page: the same transpiled output goes from interpreter fallback to a successful bytecode compile, ~38% faster per request (host). Adds 5 tests (split boundaries, boundary quote-doubling, expression composition); 17/17 host tests pass. Spec bumped to 1.2; phase2-status.md tracks state. Closes #4
…tion Adversarial stress testing found that chunking a page with >512 distinct 63-byte constants (~32 KB of distinct content, cumulative across the page) overflows rexx370's program-wide BCOM_MAX_CONSTS table. The overflow returns IRXBC_ERR_STOR, which bc_err_is_fallback() does not classify as fallback- eligible, so irx_exec_run returns it fatally -- a 500 where the unchunked page rendered via interpreter fallback. Confirmed on a realistic 50 KB page. The proper fix is upstream: classify the compile-time capacity error as fallback-eligible in rexx370 (same pattern as the 63-byte STRTOOLONG fix). PR #5 is not strictly safe to merge until that is in place.
|
Blocked on mvslovers/rexx370#212. Post-merge stress testing found that chunking a page with >512 distinct 63-byte constants (~>32 KB distinct content, cumulative — the program-wide |
|
Superseded by rexx370 and closing. rexx370 moved long-literal handling into the bytecode compiler itself: #208 ( Verified host-side against rexx370 main: the unchunked transpiler output (no PR #5) now behaves identically to the chunked output — small pages compile natively ( Closing; |
…nspiler Records the resolution of the Phase 2 "literals" item. rexx370 now handles long SAY literals in the bytecode compiler itself (#208 compile-time chunking) and falls back gracefully on fixed-table overflow (#212/#213, IRXBC_ERR_CAPACITY), so transpiled .rxp pages compile with no transpiler change. The transpiler-side chunking attempt (issue #4 / PR #5) was closed as superseded. - New doc/phase2-status.md: living Phase 2 tracker (scope, decisions, evidence). - Spec bumped to 1.2: §5 and §10 updated; transpiler stays one-literal-per-line.
Closes #4. First Phase 2 item ("literals", spec §5).
Problem
The
.rxptranspiler emitted one REXX string literal per HTML run. rexx370'sbytecode compiler caps each stored constant at 63 bytes (
IRXBC_STR_MAX), soordinary HTML lines returned
IRXBC_ERR_STRTOOLONGand the engine fell back tothe token-walk interpreter — a doomed compile attempt paid on every request.
Change
line_close_lit(src/rxptrans.c) now splits each pending literal into chunks ofat most 63 value bytes, each a quoted string joined by the existing
||.'is one value byte (writtendoubled), so the stored constant never exceeds the limit.
all pre-existing tests are unchanged. Only >63-byte literals split, and
'ab'||'c'concatenates to the same bytes..rxp);.rexxdirect execution is untouched.Supersedes the spec's
_LIT.-stem-via-vpool alternative, which is blocked on thecurrent rexx370 (
IRXEXCOMis not an installed load module).Verification
make test-host: 17/17 (5 new: split boundaries 63/64/130, boundaryquote-doubling, expression composition).
before vs. after, same rendered bytes:
[bc] exec=0 fallback=1(interpreter fallback)[bc] exec=1 fallback=0(compiles + runs bytecode)µs are host-only; the ratio is the transferable part. Details and the full
four-cell matrix in
doc/phase2-status.md.Docs
Spec bumped to 1.2 (§5 rewritten to the real mechanism, §10 phase plan
re-scoped into unblocked vs. rexx370-blocked).
doc/phase2-status.mdtracks therunning Phase 2 state.