Record artifact labels - #5
Merged
Merged
Conversation
A workflow author writes labels: {subject: batch_017} on an artifact and
a reader can group by subject without parsing filenames or consulting a
per-domain sheet.
Recorded verbatim on both the declared and the resolved artifact, never
interpreted, string keys and values only. Omitted when empty so an
unlabelled run reads exactly as before.
Read through getattr: BaseArtifact.labels is approved upstream but not
released, so this ships now and starts working when a release carries it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A workflow author writes this on an artifact:
and it reaches the record verbatim, so a reader can answer "everything derived from subject batch_017" with a lookup instead of parsing filenames or consulting a sheet that uses a different vocabulary in every field of study.
What changed
labels_of()reads an artifact's labels, applied at both points that describe an artifact: the resolved entry in each task record, and the declared entry indefinition.json.Recorded verbatim and never interpreted. String keys and values only, because a reader indexes on them and a value it cannot compare is worse than an absent one. Omitted entirely when empty, so an unlabelled run's records are byte-identical to before.
Why it reads through getattr
BaseArtifact.labelsis approved upstream in temple-compute/horus-runtime#181 but not in a release yet. Reading defensively means this ships now, reports no labels on today's engine, and starts working the moment a release carries the field. A recorder that raised on an older engine would fail the run it exists to observe.Tests
Seven added, 67 total, coverage 93.40%.
Five unit tests on the reader: an engine without the field, an unlabelled artifact, strings surviving, non-string values filtered out, and something that is not a mapping at all.
Two end to end: labels reach the record, and an unlabelled run gains no empty keys. The first patches
labels_ofbecause the pinned engine has no field to set; the patch goes away when the dependency moves.Also
Known limits now name the upstream PR each one waits on.
sourceis stillnull, but temple-compute/horus-runtime#184 is merged and_copy_sourcealready reads throughgetattr, so that one also fixes itself on the next release with no change here.