Repository navigation
Allow line breaks between CJK characters in MTEXT layout - #1449
Conversation
| _NO_BREAK_AFTER = set("(〔[{〈《「『【〘〖〝‘“") | ||
|
|
||
|
|
||
| def is_cjk(char: str) -> bool: |
There was a problem hiding this comment.
please add a LRU cache decorator @functools.lru_cache(maxsize=1024)
|
FYI: I don't know if you used an AI, but I checked your There’s no built-in import re
_CJK_RE = re.compile(
"[" + "".join(
rf"\U{start:08x}-\U{end:08x}"
for start, end in _CJK_RANGES
) + "]"
)
def is_cjk(char: str) -> bool:
return _CJK_RE.fullmatch(char) is not NoneThis assumes single-character inputs, as your calling code does. You may not need You can also avoid checking each interior character twice by carrying its classification forward: def split_cjk_word(word: str) -> list[str]:
if not word:
return [""] # Preserve your original behavior.
parts: list[str] = []
start = 0
prev = word[0]
prev_is_cjk = is_cjk(prev)
for index in range(1, len(word)):
char = word[index]
char_is_cjk = is_cjk(char)
if (
(prev_is_cjk or char_is_cjk)
and char not in _NO_BREAK_BEFORE
and prev not in _NO_BREAK_AFTER
):
parts.append(word[start:index])
start = index
prev = char
prev_is_cjk = char_is_cjk
parts.append(word[start:])
return partsThis reduces classification calls for CJK-heavy text, though your original short-circuiting can skip some calls, so the overall speedup is workload-dependent. One important distinction: detecting these ranges is not the same as implementing Unicode line breaking. The ranges include non-CJK characters, such as fullwidth Latin letters, and omit some newer ideograph extensions. If you need standards-based wrapping rather than this particular heuristic, use an implementation of Unicode UAX #14, such as ICU’s line-break iterator through PyICU. |
Fixes #1389
CJK text has no spaces, so a whole run of Chinese/Japanese/Korean characters became one
Textcell and could not wrap at the column width. The MTEXT layout engine now splits a word into parts wherever a break is allowed and joins them with a zero width breakableSpace: between CJK characters and between CJK and other text, but not before closing punctuation like、。」or after opening punctuation like「(. Words without CJK characters stay one cell.With the example from the issue,
MTextExplodenow produces two lines instead of one line of ~17 drawing units for a width of 4.75. As each cell becomes its own TEXT entity, exploded CJK text gets one TEXT per character (the same way Latin text gets one TEXT per word).Added
tests/test_10_issues/test_issue_1389_cjk_mtext_wrap.py.pytest tests/test_05_tools tests/test_07_render tests/test_08_addons tests/test_10_issuespasses (pure Python,EZDXF_DISABLE_C_EXT=1), formatted with black.