fix(parser): recover unencoded core font subsets - #606
Open
jmdonbaba wants to merge 4 commits into
Open
Conversation
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue(s)
Closes #593
Motivation and Context
Some embedded Type1/CFF font subsets omit both
/Encodingand/ToUnicode.When this happens, the parser falls back to StandardEncoding and corrupts source
text in font-variant runs, such as bold, italic, or colored text, before it is
sent to the translation service.
This PR fixes the decoding issue reported in #593.
Summary of Changes
G<glyph-id>names in embedded subset fonts.also lack a
/ToUnicodemap.Before / After
Reproduced using
LO060_EN_46C_FV_simple.pdf, a three-page minimalreproduction extracted from the original
LO060_EN_46C_FV.pdfattached in #593.7KLV FRXUVH ZLOO SURYLGH \RX ZLWK:This course will provide you with:7DUJHW *URXSTarget GroupQRW intended as a VHOI-VWXG\ programnot intended as a self-study programThe corrected source text is produced before any translation request is sent, so
the fix is independent of the translation model.
PR Type
Breaking Changes
No breaking changes.
Contributor Checklist
Testing Instructions
uv run pytest -q.uv run ruff check babeldoc/format/pdf/new_parser/active_direct_font_backend.py tests/test_active_direct_font_backend.py.LO060_EN_46C_FV_simple.pdf, the minimal three-page reproduction extracted from the original PDF attached in PDF文件中英文字体有加粗、斜杠、颜色变成乱码或者翻译错误 #593.This course will provide you with:,Target Group, andnot intended as a self-study programis correctlyrecovered before translation.
Minimal Reproduction PDF
LO060_EN_46C_FV_simple.pdfis a three-page minimal reproduction extractedfrom the original
LO060_EN_46C_FV.pdfattached in #593.LO060_EN_46C_FV_simple.pdf
Screenshots
The following screenshots were generated from
LO060_EN_46C_FV_simple.pdf,a minimal reproduction derived from the original issue attachment.
1.
Course GoalsBefore
After
2.
Target GroupBefore
After
3.
CopyrightBefore
After
Additional Notes
The recovery path is enabled only when both
/Encodingand/ToUnicodearemissing, avoiding changes to normal PDF font decoding behavior.
Summary by cubic
Recovers ASCII decoding for unencoded Arial/Times New Roman Type1/CFF subsets by deriving a Unicode map from embedded G names only when
/Encoding,/ToUnicode, and any embedded Type1 header encoding are all absent. Previously we fell back to StandardEncoding and produced garbled text; now those subsets decode correctly without affecting other fonts.freetypeand mapsG(\d+)names (IDs 3–97 → codepoint +29) viarecover_unencoded_subset_map.FontFileand uses its encoding if present, skipping recovery.FontFile,FontFile2, orFontFile3; swallows read errors to keep fallback paths intact./Encodingand/ToUnicode, precedence of embedded encodings, and a non-core font recovery guard.Written for commit b7c939f. Summary will update on new commits.