Repository navigation
Conversation
`Trustmark::encode` cloned the full-resolution input twice and ran the crop/resize/normalize preprocessing a second time just to subtract the same tensor from the encoder output. `apply_residual` then converted and cloned the input to RGBA32F several times, converted the upscaled residual to RGBA32F, and copied the result back into yet another float image. Build the model input once from a borrowed `ModelImage` (new `TryFrom<&ModelImage>`; the owned conversion delegates to it), pass the tensor to the encoder by reference and reuse it for the residual. Convert the input to RGB32F or RGBA32F once, consuming it, and add the upscaled residual to its color channels in place, so at most two full-size float images are alive. The float operations and their order are unchanged, so outputs are bit-identical; a unit test pins the residual arithmetic, alpha handling and output formats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Trustmark::encodedid the same full-resolution work twice, andapply_residualmade more full-size float copies than it needed:encodecloned the full-resolution image twice and ran crop → resize → normalize twice to build the same input tensor: once for the encoder and once just to subtract it from the encoder's output. With the antialiased resize in fix(rust): antialias downscaling to the model input size #13, that duplicate pass costs more.apply_residualconverted the input to RGBA32F, cloned it, converted the upscaled residual to RGBA32F, converted the result again, and then copied it in withimageops::replace. That kept several full-size float images alive at once.Change
The public API is unchanged.
encode:TryFrom<&ModelImage>. The old owned conversion now delegates to it, sodecodeis unchanged.apply_residual:Bit-for-bit equivalence
A scratch harness hashed the output pixel type and all of its raw f32 bytes, then decoded the result. All 53/53 cases were identical to the base branch and decoded to the right watermark:
ghost.png(RGBA),ufo_240.jpg,ripley.jpg, a 3672×2448 upscale, 4032×1500 (wide), 1500×4032 (tall) and a 3072×3072 RGBA upscale;Measurements
Release build, variant Q, base and new runs interleaved with alternating order. The machine was shared and heavily loaded; the cleanest pass ran at a load average of about 8–9.
ufo_240,ghost) show no measurable change. The expected saving there is under 1 ms.Peak memory during encode, measured from a warmed-up model with VmHWM reset just before the call (stable across runs):
Tests
cargo fmt --checkand clippy with-D warningsare clean.cargo test --workspace: 28 lib tests and the doc test pass.apply_residual_to_color_channelspins the residual arithmetic, saturation, alpha passthrough and output formats. It also passes against the base implementation, so it describes existing behaviour.git merge-treeis clean, and the merged tree builds and passes 30 lib tests plus the doc test;roundtriptest helper, which fix(rust): place B/C residual on the center crop of wide/tall images #14 already conflicts on in the same way.Not run: the criterion
encodebench, and timings on the tree merged with #13.🤖 Generated with Claude Code