update_frontmatter regenerates the whole frontmatter block: keys alphabetised and one-line lists exploded

[BattermanZ]

This was generated by AI during triage.

Summary

update_frontmatter does not edit a Note's frontmatter, it regenerates it. Changing one unrelated key rewrites the whole block: every key is re-sorted alphabetically, and one-line YAML lists are exploded into block lists. The Note's body is preserved byte for byte, but everything between the --- markers is discarded and re-emitted.

Callers reach for this tool to set one field. They get a reformatted frontmatter block on every Note they touch.

Reproduction

Add one key to a Note with a typical Obsidian-style frontmatter block:

---
tags: [type/project, status/active, domain/plans, topic/kids]
created: 2026-03-18
---

Call update_frontmatter with {"status": "active"}. Result:

---
created: 2026-03-18
status: active
tags:
- type/project
- status/active
- domain/plans
- topic/kids
---

Two changes nobody asked for. created and tags swapped places because the keys are now in alphabetical order, and the one-line tag list became a five-line block list.

This is not a regression report from the field. The behavior is asserted by the repository's own test suite: update_note_frontmatter_merges_top_level_keys_and_keeps_the_body_byte_for_byte expects title: Home / tags:\n - alpha / nested: to come back as nested: / status: / tags:\n- alpha / title: Home. The test comment records the cause without treating it as a defect: "serde_json maps serialize deterministically (keys sorted)".

Cause

The merge parses the frontmatter block into a serde_json::Map, applies the updates, and serializes the whole map back through serde_yaml_ng. Two things follow.

serde_json::Map is not built with preserve_order here, so it is a sorted map and the author's key order is lost on the way in. And serde_yaml_ng emits sequences in block form, so any flow-style list is converted regardless of how it was written.

The write itself is careful. It rewrites exactly the span between the markers and leaves the body untouched. The loss happens in the round trip through the map, not in the file handling.

Why it matters

The damage scales with how much you use the tool. On a Vault of 553 Notes where 380 carry a one-line tags: list, any agent doing routine frontmatter maintenance reformats each Note it touches. The git diff for a one-field change is a rewritten frontmatter block, which makes the actual change hard to see and makes git blame on frontmatter close to useless.

It also means the tool cannot be used for bulk work. Anything that touches frontmatter across many Notes produces a diff whose size is unrelated to the size of the change.

Suggested direction

Preserve what the author wrote, and touch only the keys the caller named.

• Keep the existing key order. Keys the caller did not mention should come back in their original positions; genuinely new keys appended at the end is the obvious rule.
• Keep each value's existing YAML style. A one-line list stays a one-line list, a block list stays a block list.
• Leave indentation, quoting, and comments alone where a key was not touched.

Whether that is achievable by swapping in an order-preserving map and a style-aware emitter, or needs surgical span edits per key, is an implementation question worth deciding before the work starts. The order-preserving map fixes half of it cheaply (serde_json's preserve_order feature) and is worth checking first, but it does not address flow versus block style.

Worth grilling before it goes to an agent. Deciding whether comments inside frontmatter must survive, and what happens to a block the tool cannot parse, will shape the approach.

Related

• No way to rename a tag across a Vault — the one vault-wide refactor left to the caller #242 (Vault-wide rename_tag) hit this during triage. That operation deliberately avoids this machinery and edits tags in place, precisely so a 263-Note rename produces a one-word-per-Note diff. It does not fix this issue and does not depend on it.

Environment

Reproduced against development at 1c485ca, Linux, cargo test --lib.

[BattermanZ]

This was generated by AI during triage.

Agent Brief

Category: bug
Summary: update_frontmatter must edit only the frontmatter keys the caller names and leave every other byte of the block exactly as the author wrote it. Record the general rule as ADR-21 in the same change.

Current behavior

Setting one key regenerates the whole frontmatter block. The block is parsed into a serde_json::Map, which is a sorted map, so the author's key order is gone before anything is written. The merged map is then serialized through serde_yaml_ng, whose emitter writes every sequence in block form, so a one-line list becomes a multi-line one. The Note's body survives byte for byte; everything between the --- markers does not.

Reproduced on development at 1c485ca. Given:

---
tags: [type/project, status/active, domain/plans, topic/kids]
created: 2026-03-18
---

a call setting {"status": "active"} writes:

---
created: 2026-03-18
status: active
tags:
- type/project
- status/active
- domain/plans
- topic/kids
---

Neither the reordering nor the exploded list was requested.

The scale is not marginal. Across the maintainer's working vault, the demo vaults, and both shipped sample vaults, 530 of the 531 Notes carrying frontmatter use a one-line list, so effectively every frontmatter write reformats the Note it touches. Comments inside a block, YAML anchors, block scalars, and duplicate keys occur zero times across the same 531 Notes.

The current behavior is not accidental. It is described in the merge function's own doc comment and pinned by two tests, one asserting the exact reformatted bytes as a canary for an emitter swap. Both are now wrong and must be rewritten as part of this change.

Desired behavior

Only the named keys change. After a successful call, every byte of the frontmatter block that does not belong to a key the caller named is identical to what was there before: key order, one-line versus multi-line lists, indentation, quoting style, comments, and blank lines. Values are not re-emitted, so a value written as 1.50 stays 1.50 and a value written as "123" keeps its double quotes. This is the same promise the Note body already has, extended to the block.

A key the caller names keeps its author's shape. Replacing the contents of a one-line list writes a one-line list; replacing a multi-line list writes a multi-line list. Where the caller changes the kind of the value, for instance replacing a list with a single string, there is no prior shape to inherit and the plain form is written.

New keys are appended at the end of the block, and lists are written on one line. This is the house style for anything an agent adds. Inserting a new key in sorted position is specifically wrong; sorted insertion is how the block came to be alphabetised in the first place.

The caller cannot choose a style. Do not add a parameter, flag, or convention letting a caller request one-line or multi-line output. The rules above are deliberately predictable rather than configurable.

Three refusal rules, and no others.

• A frontmatter block that does not parse as YAML at all: refuse the whole call and leave the file untouched, as the current implementation already does, with the existing "fix it directly in the vault" guidance.
• A key the caller named that cannot be located or replaced unambiguously, for example because that key appears twice in the block: refuse the whole call, touch nothing, and return a structured error naming the key and the reason. "Cannot edit key tags: it appears twice in this note's frontmatter, so which one to change is ambiguous" is the standard to hit. A refusal that does not name the key is not acceptable.
• A key the caller did not name that is odd or ambiguous: make the edit anyway. The existing duplicate-key quality warning stays, so the caller still learns about it.

A call never half-applies. Either every named key is written or none is.

Deletion, creation, and stripping keep their current semantics. An explicit null still deletes a key, and deleting it removes the lines it owned without disturbing its neighbours. A Note with no frontmatter block still gets one created. Deleting the last remaining key still removes the block entirely rather than leaving an empty --- pair. An empty update map and a create-from-nulls-only are still refused.

Optimistic concurrency is unchanged. expected_content_hash is still required and still checked before anything is read or written, and the returned hash is still the hash of the bytes actually written.

ADR-21

Record a new ADR in the architecture decision records, accepted, in the same change that makes it true. The decision: text the caller did not supply is not Hatchdoor's to rewrite. A write that replaces a Note's whole content may normalise what it was given, and the existing line-ending and trailing-newline normalisation is the illustration of that; a write that names a part of a Note leaves everything it did not name byte for byte.

The record needs the index-table row with its one-line "what not to break", and should cite this issue as the worked example of what the rule prevents. It refines ADR-03, which already routes every write through one layer, by giving that layer a promise about what it may do to a file. It supersedes nothing.

Follow the repository's append-only ADR process and template rather than inventing a format.

Key interfaces

• The frontmatter merge function in the shared write layer (update_note_frontmatter) is where this lives. Its signature does not need to change: it still takes the Note entry, a map of top-level keys to set or delete, and the expected content hash. What changes is that it stops round-tripping the block through a map and starts locating and replacing the spans the named keys occupy.
• Expose the in-place editing as a plain function in the module that owns frontmatter writes, callable by other write operations. Do not build a trait, a seam, or an extension point for it: ADR-13 rules out speculative abstractions, and there are exactly two callers, one of which does not exist yet.
• The refusal for an unlocatable named key should use the same error variant the invalid-YAML refusal already uses, so callers branching on the structured error code see one category of "this note needs a human", with the message carrying the specific key.
• WriteOutcome's quality warnings keep their current meaning and are still populated for a duplicate key.
• The MCP update_frontmatter tool description currently reads "Shallow top-level YAML merge into an existing note's frontmatter, leaving the body untouched", which understates what is now true. It should say that only the keys named are changed and the rest of the block keeps the author's formatting, and it should say the call is refused when a named key cannot be edited unambiguously. This is the only part of the advertised contract a caller ever reads, so it has to be right.
• serde_json's preserve_order feature is not the fix and should not be enabled. It is a crate-wide feature that changes every map in the codebase, and it addresses key order only; the one-line versus multi-line list problem has no feature flag behind it.

Acceptance criteria

• Setting one key on a Note whose frontmatter is tags: [a, b, c] followed by created: 2026-03-18 produces a file whose only difference is the added key, with tags still on one line and still first
• A comment line inside a frontmatter block survives a write that does not name a key on that line
• A value written as 1.50 is still 1.50, and a value written as "123" still carries its double quotes, after a write naming a different key
• Replacing the value of an existing one-line list writes a one-line list; replacing an existing multi-line list writes a multi-line list
• A key that did not previously exist is appended after the last existing key, and a new list-valued key is written on one line
• Deleting a key with an explicit null removes exactly the lines that key owned
• A Note whose frontmatter block contains the same key twice is refused when that key is named, with an error naming the key, and the file is unchanged on disk
• The same Note is written successfully when a different, unambiguous key is named, and still reports the duplicate-key quality warning
• A frontmatter block that is not valid YAML is still refused with the file unchanged
• A Note with no frontmatter block still gains one, and deleting the last key still removes the block
• The two existing tests that assert alphabetised, block-style output are rewritten to assert the preserving behavior, and the byte-exact emitter canary is either replaced with a byte-exact assertion of the new behavior or removed with a note saying why
• ADR-21 exists, is accepted, appears in the ADR index table with a "what not to break" line, and cites this issue — recorded as ADR-22: ADR-21 was claimed by the saved-queries decision from Feature request: Obsidian Bases-compatible live query views inside notes #272, merged before this branch was cut
• The MCP update_frontmatter tool description states the preservation rule and the refusal condition
• node scripts/check-module-map.mjs passes, and the module map is updated if any production source file moved or a contract changed

Out of scope

• rename_tag itself, which is No way to rename a tag across a Vault — the one vault-wide refactor left to the caller #242. That issue is blocked on this one and will call the in-place editor this change introduces; do not implement any part of it here.
• Whole-content write normalisation, which is Note write tools normalise content without saying so, so content_hash does not match what the caller sent #260. Line-ending and trailing-newline handling on update_note is deliberate and stays exactly as it is. ADR-21 is worded to permit it.
• Any change to how frontmatter is read, parsed for the index, or projected by get_frontmatter. This is a write-path change only.
• Adding a way for a caller to express a formatting preference, or to write or read comments in a frontmatter block.
• Repairing existing Notes that a previous update_frontmatter call already reformatted. There is no migration; the fix is forward-looking.
• Enabling serde_json's preserve_order.

[BattermanZ]

Implemented on fix/frontmatter-in-place-edit, commit 507f3f8, carried into development by #280.

The block is now edited as text rather than regenerated. Every acceptance criterion in the brief is ticked above against live evidence, not just tests: the run drove the real MCP endpoint on a locally running build and read each result back off disk, 20 of 20 checks passing.

Two things worth knowing beyond what was asked:

The ADR is recorded as ADR-22, not ADR-21. ADR-21 was claimed by the saved-queries decision from #272, which merged before this branch was cut. Same decision, same content, next free number.

Two pre-existing defects in the same function were fixed here, both surfaced by the live pass and the review rather than by the brief. Deleting the last key stripped the block but left the newline that ends the closing marker behind, so every stripped note gained a blank first line, which the old test's ends_with assertion never saw. And a CRLF note came out with mixed line endings, because this write path never runs content through the whole-note line ending normalisation.

One deliberate limitation, documented in ADR-22 and pinned by a test: the editor reparses its own output and compares it against the merge that was asked for, refusing on a mismatch. That backstop can refuse a call over a key the caller did not name, when a block is laid out in a way the key scanner reads differently from a YAML parser, an indented top-level mapping being the example. Refusing beats writing a file whose meaning has quietly changed.

Noted for a separate ticket, not touched here: frontmatter_warnings matches on "---\n", so a CRLF note never gets a duplicate-key quality warning.