Skip to content

Allow line breaks between CJK characters in MTEXT layout - #1449

Merged
mozman merged 2 commits into
mozman:masterfrom
zzoraemon043:fix/cjk-mtext-wrap
Oct 20, 2026
Merged

mozman merged 2 commits into
mozman:masterfrom
zzoraemon043:fix/cjk-mtext-wrap

Conversation

@zzoraemon043

Copy link
Copy Markdown
Contributor

Fixes #1389

CJK text has no spaces, so a whole run of Chinese/Japanese/Korean characters became one Text cell and could not wrap at the column width. The MTEXT layout engine now splits a word into parts wherever a break is allowed and joins them with a zero width breakable Space: between CJK characters and between CJK and other text, but not before closing punctuation like 、。」 or after opening punctuation like 「(. Words without CJK characters stay one cell.

With the example from the issue, MTextExplode now produces two lines instead of one line of ~17 drawing units for a width of 4.75. As each cell becomes its own TEXT entity, exploded CJK text gets one TEXT per character (the same way Latin text gets one TEXT per word).

Added tests/test_10_issues/test_issue_1389_cjk_mtext_wrap.py. pytest tests/test_05_tools tests/test_07_render tests/test_08_addons tests/test_10_issues passes (pure Python, EZDXF_DISABLE_C_EXT=1), formatted with black.

_NO_BREAK_AFTER = set("(〔[{〈《「『【〘〖〝‘“")


def is_cjk(char: str) -> bool:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a LRU cache decorator @functools.lru_cache(maxsize=1024)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@mozman
mozman merged commit 9fda903 into mozman:master Oct 20, 2026
12 checks passed
@mozman

mozman commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

FYI: I don't know if you used an AI, but I checked your split_cjk_word() function with GPT-6.1 Sol to see if there are some optimizations possible: tldr - not really - nothing to change, I guess the cached is_cjk() is good enough

There’s no built-in str.iscjk(). For exactly the ranges you’ve defined, a compiled standard-library regex is an alternative worth benchmarking: it performs the character-class check in C rather than iterating over ranges in Python.

import re

_CJK_RE = re.compile(
    "[" + "".join(
        rf"\U{start:08x}-\U{end:08x}"
        for start, end in _CJK_RANGES
    ) + "]"
)

def is_cjk(char: str) -> bool:
    return _CJK_RE.fullmatch(char) is not None

This assumes single-character inputs, as your calling code does. You may not need lru_cache here; benchmark with and without it using representative text. Your existing cached version could already be faster when characters repeat frequently.

You can also avoid checking each interior character twice by carrying its classification forward:

def split_cjk_word(word: str) -> list[str]:
    if not word:
        return [""]  # Preserve your original behavior.

    parts: list[str] = []
    start = 0
    prev = word[0]
    prev_is_cjk = is_cjk(prev)

    for index in range(1, len(word)):
        char = word[index]
        char_is_cjk = is_cjk(char)

        if (
            (prev_is_cjk or char_is_cjk)
            and char not in _NO_BREAK_BEFORE
            and prev not in _NO_BREAK_AFTER
        ):
            parts.append(word[start:index])
            start = index

        prev = char
        prev_is_cjk = char_is_cjk

    parts.append(word[start:])
    return parts

This reduces classification calls for CJK-heavy text, though your original short-circuiting can skip some calls, so the overall speedup is workload-dependent.

One important distinction: detecting these ranges is not the same as implementing Unicode line breaking. The ranges include non-CJK characters, such as fullwidth Latin letters, and omit some newer ideograph extensions. If you need standards-based wrapping rather than this particular heuristic, use an implementation of Unicode UAX #14, such as ICU’s line-break iterator through PyICU. unicodedata.east_asian_width() is not an equivalent replacement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: MTEXT Explode fails to wrap continuous CJK text (width overflow)

2 participants