feat(update): preserve developer-owned regions across updates (#2184)

[DrKat0m]

Templates often ship files with dummy content or TODO markers that the developer replaces right after generation. Previously copier update treated those regions like any other template-owned content, so a change to the placeholder in the template could clobber the developer's work or raise a needless merge conflict.

Template authors can now wrap a developer-owned region in literal marker comments that survive rendering:

# copier:preserve:start <optional-id>
...developer-owned content...
# copier:preserve:end <optional-id>

During an update the content between the markers is captured from the current project and injected into every intermediate render (old render, new render, and the freshly rendered destination) so all three merge sides agree on the region. The merge therefore keeps the developer's content conflict-free, while the marker lines and surrounding content keep updating from the template as usual.

• Added copier/_preserve.py with the marker parsing, capture and restore primitives.
• Wired capture/restore into Worker._apply_update, scoping the scan to template-managed files discovered in the old render.
• Documented the feature in docs/updating.md.
• Added unit and integration tests in tests/test_preserve.py.

Closes #2184

[DrKat0m]

Hi @sisp, whenever you have a chance, could you please take a look at this PR? I'd really appreciate any feedback on the approach (especially the choice to keep the region conflict free by aligning all three merge sides rather than post processing conflict output) and on the marker syntax. Happy to iterate on anything. Thank you!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.32%. Comparing base (02e1574) to head (677ce4f).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2762      +/-   ##
==========================================
+ Coverage   97.29%   97.32%   +0.02%     
==========================================
  Files          60       61       +1     
  Lines        7584     7668      +84     
==========================================
+ Hits         7379     7463      +84     
  Misses        205      205              

Flag	Coverage Δ
unittests	97.32% <100.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[DrKat0m]

Hi @sisp, just a gentle follow up whenever you have some time. I'd really appreciate it if you could take a look at this PR and share any feedback you have. In particular, I'd love to know your thoughts on the overall approach and whether the marker syntax feels appropriate. Thank you!

[sisp]

Just a quick thought on this (because I'm on vacation 🌴 ): If the preserve markers must remain in the rendered files (as you document), then they must be valid comments of the respective file format. This means, a hardcoded #-comment won't work for files like .js, .ts, .go, .c, etc. because this isn't how comments work in those languages. Like, e.g., Prettier's error suppression comments, we might need a comprehensive set of marker variants depending on this file format / programming language. Unlike Prettier, Copier supports an open set of file formats – because Copier renders any text file no matter the underlying format. So I'm not sure yet how to tackle this best.

[sisp]

Ideally, the markers aren't part of a rendered file, then we could use Jinja syntax and be language-agnostic. Also, use of Copier wouldn't leak into rendered files.

Perhaps we could omit developer-owned sections from old and new copies that are created by the update algorithm? This way, the 3-way merge should ignore changes in the template in those sections. #2376 might make this even easier.

We could try this quickly by exporting _copier_operation also in the file render context and conditionally render a developer-owned block like this:

{%- if _copier_operation != 'update' %}
...
{%- endif %}

If this works, I'd prefer a custom tag like

{%- ignore %}
...
{%- endignore %}

for better DX.

[DrKat0m]

Thanks for the feedback - enjoy the vacation! 🌴 You're right that hardcoded comment markers can't work across an open set of formats, so I reworked this the way you suggested.

Dropped the hardcoded markers in favor of a Jinja tag -

{% ignore -%}
def greeting() -> str:
return "dummy" # developer replaces this after generation
{%- endignore %}

It's a small extension compiling to {% if _copier_operation != 'update' %}…{% endif %} - rendered on copier copy, omitted from the internal renders during copier update. Being pure Jinja, nothing leaks into rendered files and it's fully language agnostic.

However, since the region is omitted from both merge sides, the developer's content re-applies as a diff hunk, clean when there's stable context around the block, but if the template edits a line directly adjacent to it, it degrades to a normal merge conflict.