Repository navigation
Preserve single-object processing history during concatenation - #373
Closed
sylvesterkaczmarek wants to merge 1 commit into
Closed
sylvesterkaczmarek wants to merge 1 commit into
sylvesterkaczmarek wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GitHub Issue: #372
Description
Fixes #372.
Normalize a single-object
history_jsonattribute to a one-element list inretrieve_history, and return an empty list for an absent attribute. Both the CLI collector and Harmony adapter extend their output history with this helper's result. Returning a dictionary caused them to append its property names and lose the corresponding provenance values.Existing history arrays retain their order and contents. Malformed JSON still raises at the reader. This does not change concatenation, numerical values, download handling, or the new operation record. The helper return annotation and documentation now describe the record-list contract used by its callers.
Local test steps
Added eight cases covering singleton records, arrays, absent/empty histories, invalid JSON, mixed-record collection, the actual module CLI in a subprocess, and the Harmony adapter. Five cases fail against unchanged production code; three controls pass.
python -m pytest tests/unit -q --disable-warnings --cov=stitchee --cov-report=term-missing: 31 passed, 91% package statement coverage.ruff check .: passed.git diff --check: passed.A separate baseline run including
tests/localreported five failures because its untracked TEMPO/CERES data directories are not supplied in this checkout. Those local-data tests are not claimed as passing. Credentialed integration tests were not run.Overview of integration done
The new CLI and adapter cases concatenate real synthetic NetCDF files, read the resulting history and data back, and verify byte-for-byte input preservation. The adapter uses actual local-file staging and real STAC objects; only downloads and collection-source lookup are supplied locally. The existing unit suite also exercises paged and unpaged Harmony CLI catalogs.
Tested on macOS with Python 3.12.11 against develop at
581d7d418a5f76a891fdbfebf3a9dd9687b02cea. No deployed Harmony service, live Earthdata downloads, or AWS resources were used. No dependency or workflow changes. This is separate from the string-input validation fix in #370.PR Acceptance Checklist
CHANGELOG.mdupdated.