fix: preserve unchanged paragraph compositions

[hazugi2004]

Related Issue(s)

#610

Description

post_translate_paragraph compared the translated str with the TranslateInput object, so the unchanged-output branch could never run for a normal input. This patch compares with translate_input.unicode instead.

When a translator intentionally returns the source text unchanged, BabelDOC now returns False from post-processing and keeps the paragraph's original composition objects rather than rebuilding and re-typesetting them.

A focused unit regression verifies the return value, tracker output, unicode, and composition object identity. No prompt, public API, dependency, or generated IL file changes are included.

PR Type

• bug
• test

Contributor Checklist

• I’ve fully read and understood the CONTRIBUTING guide
• My changes follow the project’s code style and guidelines
• I’ve linked the related issue above
• I’ve added the focused regression test
• The new test and Ruff checks passed locally
• I understand that this small proof-of-concept patch may be rewritten by maintainers

Testing Instructions

pytest tests/test_il_translator.py -q
ruff check babeldoc/format/pdf/document_il/midend/il_translator.py tests/test_il_translator.py

Local result: 1 test passed; Ruff reported all checks passed.

Additional Notes

The downstream evidence came from a local copyrighted scientific manuscript, but this PR includes and requires no source PDF or translated output.

---

Summary by cubic

Preserves a paragraph’s existing composition when the translator returns the source text unchanged. Old: unchanged paragraphs were rebuilt and marked changed; New: we detect unchanged text after normalizing 20+ punctuation runs, return False from post-processing, keep the original composition, set tracker output, and mark placeholder_full_match.

• Compares translated_text to translate_input.unicode after collapsing [. 。…，]{20,} to "." using shared EXCESSIVE_PUNCTUATION_RE in both il_translator and il_translator_llm_only to unify normalization and token counts.
• Adds a focused unit test for the normalized-unchanged path; no prompt, public API, dependency, or IL changes.

^{Written for commit be8b888. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 2 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="babeldoc/format/pdf/document_il/midend/il_translator.py">

<violation number="1" location="babeldoc/format/pdf/document_il/midend/il_translator.py:1000">
P3: The normalization pattern `[. 。…，]{20,}` is now duplicated a third time (it already exists at il_translator.py:1268 and twice in il_translator_llm_only.py:754/763). Keep the regex in one shared module-level constant so the unchanged-output comparison stays in sync with the pre/post normalization applied by the two callers.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: The normalization pattern [. 。…，]{20,} is now duplicated a third time (it already exists at il_translator.py:1268 and twice in il_translator_llm_only.py:754/763). Keep the regex in one shared module-level constant so the unchanged-output comparison stays in sync with the pre/post normalization applied by the two callers.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/document_il/midend/il_translator.py, line 1000:

<comment>The normalization pattern `[. 。…，]{20,}` is now duplicated a third time (it already exists at il_translator.py:1268 and twice in il_translator_llm_only.py:754/763). Keep the regex in one shared module-level constant so the unchanged-output comparison stays in sync with the pre/post normalization applied by the two callers.</comment>

<file context>
@@ -997,7 +997,12 @@ def post_translate_paragraph(
         """Post-translation processing: update paragraph with translated text."""
         tracker.set_output(translated_text)
-        if translated_text == translate_input.unicode:
+        normalized_input = re.sub(
+            r"[. 。…，]{20,}",
+            ".",
</file context>

[hazugi2004]

Fixed in be8b888. The repeated-punctuation rule is now the module-level EXCESSIVE_PUNCTUATION_RE in il_translator; the unchanged-output comparison, standard translator cleanup, and both LLM-only normalization sites all use that single compiled regex. The now-unused re import was removed from il_translator_llm_only.py. The focused regression and Ruff checks both pass.