Skip to content

feat: add {content_hash} naming placeholder (AI生成) - #106

Closed
pluveto wants to merge 3 commits into
mainfrom
feat/issue-71-content-hash
Closed

pluveto wants to merge 3 commits into
mainfrom
feat/issue-71-content-hash

Conversation

@pluveto

@pluveto pluveto commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Closes #71.

{fname_hash} is MD5 of the file stem. Same-name files with different bytes still collided. This adds a content digest that KeyPolicy interpolates; Artifact answers the digest and KeyPolicy does not open files.

Placeholders

  • {content_hash} — MD5 hex (lowercase) of the file bytes
  • {content_hash4} / {content_hash8} — first 4 / 8 hex digits (same truncation as {fname_hash})
  • Aliases: {contenthash}, {contenthash4}, {contenthash8}

If the naming template (or hmac_format) uses these and the artifact has no readable bytes, KeyPolicy::apply returns MissingContent. It does not fall back to {fname_hash}.

Example:

naming = "{year}/{month}/{content_hash8}_{stem}{ext}"

三剑客(deliberate)

Eric Evans:
质疑:{fname_hash} 是「名字的散列」,不是「文件的散列」。若把内容散列做成 fname_hash 的别名,或在缺字节时静默回退,统一语言就裂了——issue 里「同名不同内容会覆盖」正是因为把两种身份当成了一种。
选项:内容摘要是 Artifact 上的派生值;KeyPolicy 处于命名上下文,只向 Artifact 要摘要,不读盘。算法与 fname_hash 保持同一套 MD5 hex:这是命名防撞,不是安全边界;换 SHA-256 会在同一上下文里出现两种 hash 而无领域理由。
底线:模板要 {content_hash*} 而 Artifact 答不出内容,必须是一次明确失败。

John Ousterhout:
质疑:若 KeyPolicy 自己 fs::read,命名模块变浅——调用方还要知道路径、IO、何时该读。若在 from_path 时急切散列,不用该占位符的上传也付钱。
选项:A) KeyPolicy 读文件;B) Artifact 构造时散列;C) Artifact 回答 content_digest,按需读并缓存,KeyPolicy 只在模板(或 hmac_format)出现占位符时询问。
怎样算做完:{content_hash} / {content_hash4} / {content_hash8} 与 fname_hash 截断规则一致;无字节则 apply 报错;同 stem 不同字节得到不同 key;hmac 向量不过期;文档表补三行。错误在使用点(apply)消灭,镜像 MissingHmacKey。

Alan Kay:
质疑:Artifact 现在只是 name+size+path 的数据袋;「内容是谁」被泄漏到策略对象。对象应封装状态,对外只收消息。
选项:artifact.content_digest() 是唯一入口。有 path 则读字节(OnceLock 缓存,因为现有 App 会对同一 Artifact apply 两次且不能改 app.rs);无 path 则拒绝该消息。KeyPolicy 不打开文件、不看 PathBuf。截断是插值的事,不必先抽 ContentDigest 新类型——与现有 fname_hash 字符串切片同一消息形状。

三个人的讨论

Evans 同意 Kay:摘要必须是 Artifact 的回答,否则命名上下文会穿透到文件系统。Ousterhout 认为 Kay 的缓存可以接受(双次 apply 是既成事实),但反对抽出 ContentDigest 新类型——fname_hash 没有对称类型,多一个浅包装。Evans 曾倾向值对象,被 Ousterhout 以「完成标准不需要」否决后同意用 String hex,文档写明 MD5。三人否决「缺字节时回退 fname_hash」和「KeyPolicy 直接读盘」。MD5 对碰撞攻击脆弱,但本上下文是远程对象键防覆盖,不是完整性证明;换算法会分裂占位符语义——保留 MD5。

收敛结论

已收敛。推荐路径:Artifact::content_digest()(MD5 hex lowercase,有 path 则读一次并缓存);KeyPolicy 检测到 {content_hash*} / {contenthash*} 后询问;失败映射 MissingContent / ContentIo。被否决:KeyPolicy 读盘;构造期急切散列;SHA-256;静默回退 fname_hash。无 Go 0.2 前向兼容负担。

决策与后续

已决定:MD5、按需、Artifact 回答、apply 失败、文档三行、不改 app/source。hmac 旧向量与新测试共存。

How to verify

cargo fmt
cargo clippy --workspace --locked --all-targets -- -D warnings
cargo test --workspace --locked
cargo test --locked -p upgit-core --test naming

Expected tests: same stem / different bytes → different {content_hash} keys; {fname_hash} still name-only; missing bytes + {content_hash} → error; existing HMAC vector unchanged.

{fname_hash} is MD5 of the stem, so same-name files overwrite each other.
Artifact now answers a content digest; KeyPolicy interpolates
{content_hash}, {content_hash4}, and {content_hash8} (MD5 hex lowercase).
Templates that need a digest fail when the artifact has no readable bytes.
Comment thread crates/upgit-core/src/key_policy.rs Outdated
Comment on lines +228 to +230
fn uses_content_hash(s: &str) -> bool {
s.contains("{content_hash") || s.contains("{contenthash")
}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个函数应当内敛lambda

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已内联:删除 uses_content_hash,在 apply 里用闭包判断模板 / hmac_format 是否需要 content hash。

2777635

Comment thread docs/README.zh-CN.md Outdated
Comment on lines +292 to +297
| -- `{fname_hash}` | -- `{fname}`(不含扩展名)的 MD5 散列值 |
| -- `{fname_hash4}` | -- `{fname}` 的 MD5 散列值,取前 4 位 |
| -- `{fname_hash8}` | -- `{fname}` 的 MD5 散列值,取前 8 位 |
| -- `{content_hash}` | -- 文件内容的 MD5 散列值 |
| -- `{content_hash4}` | -- 文件内容的 MD5 散列值,取前 4 位 |
| -- `{content_hash8}` | -- 文件内容的 MD5 散列值,取前 8 位 |

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不如都用 xxHash32 算法吧

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{fname_hash} 与 {content_hash}(含 4/8 截断)都改为 xxHash32(seed 0)。hex 为 8 位 {:08x},{…8} 是完整摘要,{…4} 取前 4 位。hmac 仍是 HMAC-SHA256。upgit-core 改用 xxhash-rust(xxh32),去掉 md-5。

2777635

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

并非所有源都能提供content,因此content_hash可能为空,千万不要用空串去生成哈希值。应当调用方显式提供fallback

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

空字节不再参与散列。KeyPolicy::apply(artifact, at, content_hash_fallback: Option<&str>):有非空内容时 xxHash32;没有可读内容时用调用方传入的 fallback(原样插入,不哈希)。未提供 fallback 才 MissingContent。Publisher / App 显式传入 Some(artifact.stem())。

2777635

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已 OOP 化:resolve_content_hash 收到 KeyPolicy 上;HMAC 的 hex 编码收到 HmacSpec::hex_lower。文件级游离函数已去掉。

fcbd360

Comment thread crates/upgit-core/src/artifact.rs Outdated
Comment on lines +125 to +133
pub(crate) fn hex_lower(bytes: &[u8]) -> String {
const LUT: &[u8; 16] = b"0123456789abcdef";
let mut out = String::with_capacity(bytes.len() * 2);
for &b in bytes {
out.push(LUT[(b >> 4) as usize] as char);
out.push(LUT[(b & 0x0f) as usize] as char);
}
out
}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这显然不是 artifact.rs‎ 的职责。放到一个单独的包里,或者如果只使用一次的话,直接内联lambda。

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

哈希和 hex_lower 已从 artifact.rs 拿走。Artifact 只回答 bytes()(无 path 则为 Ok(None)),去掉 OnceLock / content_digest / NoContent。xxHash32 在 KeyPolicy 里用同一个闭包算 fname_hash 和 content_hash。

2777635

{fname_hash} and {content_hash} are xxHash32 (seed 0). Artifact answers
bytes only; KeyPolicy hashes. Callers pass content_hash_fallback so
missing or empty content is never hashed.
resolve_content_hash is a KeyPolicy method. HMAC hex encoding lives on
HmacSpec. No file-level helper functions remain.
@pluveto

pluveto commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

Refreshing the PR head after rebase onto main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

文件名格式模板 增加 文件自身hash值

1 participant