Repository navigation
Security: Pass the validated name onward in metrics service (#cs-12, #cs-13, #cs-14, #cs-18) - #1995
Merged
Merged
Conversation
…#cs-13, #cs-14, #cs-18) Remediates the 4 py/path-injection CodeQL alerts (HIGH, CWE-22) still open after the previous fix, reported in the IBM OSPO security notification (#1994): cs-12, cs-13, cs-14 (src/unitxt/artifact.py:238,240) and cs-18 (src/unitxt/utils.py:222). The earlier fix (#1993) blocked the attack at runtime but left the alerts open, and the alerts were right to stay open. It validated a derived value and then passed the original one on: requested_name, _ = separate_inside_and_outside_square_brackets(metric_name) if not is_name_legal_for_catalog(requested_name): raise HTTPException(status_code=400, ...) ... metric = ArtifactFetcherMixin.get_artifact(metric_name) # <- raw value Checking requested_name says nothing about metric_name, so nothing about the value that actually reached the loader. It happened to be safe only because the two strings agreed on the part that mattered -- a property no check enforced and any later edit could break. CodeQL reported it as unsanitized because, as dataflow, it is. The taint path in the current SARIF is unchanged from before the fix: service/metrics/main.py:69,100 metric_name (FastAPI path parameter) operators.py:1190-1192 fetch_artifact(artifact_identifier) artifact.py:546,562-563 fetch_artifact(artifact_rep) artifact.py:238,240 open(path) <- cs-12, cs-13, cs-14 utils.py:203-204,221-222 open(path) <- cs-18 cs-18 is a fresh alert raised against the fix commit itself, replacing cs-15 and cs-16 at the same sink: the flow survived, only its line numbers moved. The fix makes the sanitization real rather than incidental: 1. artifact.py -- new sanitize_catalog_identifier(). Validates a reference as a catalog name and returns the name it validated, rebuilt from its checked segments, plus the bracketed overrides already parsed. Callers pass on the value that was checked, so no unvalidated substring can reach a loader and the guarantee no longer depends on two variables staying in agreement. Rejections raise IllegalCatalogNameError, a ValueError subclass, so a caller can answer 400 for "not a catalog name" without catching unrelated failures. Malformed brackets ("metrics.accuracy[unclosed") now reject cleanly too; before, the parser's ValueError reached the endpoint's broad except and became a 500. 2. service/metrics/main.py -- use only the sanitized name. compute() passes safe_metric_name to the loader and never metric_name, which is now consumed by the sanitizer and nothing else. The raw value is also kept out of the log lines, so it cannot be used to forge log entries. 3. operators.py -- get_artifact() takes overwrite_kwargs. Since overrides now travel beside the name instead of inside it, they are applied as keyword overrides. Re-serializing them into the identifier was not an option: the parser returns typed values, so "[n=[1,2]]" and nested artifact references do not round-trip through a string. The cache key includes the overrides, so an overridden artifact cannot be served from, or written to, the plain artifact's entry. The parameter is optional and all seven existing callers are unaffected. fetch_artifact()'s support for loading an artifact from a file path is again left intact: it is a documented library feature, and the untrusted input is blocked at the service boundary where the expectation is narrower. Verified: the planted-file read and the arbitrary-file read are both refused with 400, with no file content in the error; legitimate names, and names with overrides, still resolve and the override is applied. Tests: tests/library/test_path_injection_service.py, extended to 20 tests. Three of them fail against the #1993 code, including one that names get_artifact(metric_name) as the raw-value use that kept the alerts open -- that class asserts the endpoint never passes the raw parameter onward, so the regression this commit fixes cannot recur silently. Added coverage for the sanitizer's rejections (absolute paths, traversal, Windows paths, URLs, null bytes, malformed brackets, non-strings), for overrides still applying, and for the cache not being poisoned across overridden and plain fetches. The earlier tests pinning load_json's non-disclosure and file-path loading are kept. tests/library: 799 passed. The 3 failures (test_metrics.py::test_f1_strings, test_utils.py::test_require_errors, test_type_utils.py::test_parse_malformed_string) reproduce identically on a clean main tree and come from optional dependencies missing locally. ruff 0.1.6 (the pinned pre-commit version) check and format, codespell and the relative-imports hook all pass. Reported by: IBM OSPO security scanning (CodeQL default setup) Closes: #1994 Signed-off-by: Yoav Katz <katz@il.ibm.com>
yoavkatz
added a commit
that referenced
this pull request
Nov 7, 2026
…cs-14, #cs-18) (#1997) The 4 py/path-injection alerts (HIGH, CWE-22) reported in #1994 are still open after #1995, and are reported again in #1996: cs-12, cs-13, cs-14 and cs-18. CodeQL analysed the merge commit (603f1c5, analysis 2093723778) and still found all four. #1995 was wrong, and its own commit message overstated the guarantee. It claimed the rebuilt name "shares nothing with the raw input". As dataflow that is false: segments = name.split(constants.catalog_hierarchy_sep) if not all(is_name_legal_for_catalog(segment) for segment in segments): raise IllegalCatalogNameError(identifier) return constants.catalog_hierarchy_sep.join(segments), args split() and join() propagate taint, so the returned string is still derived from the request. The re.match guard sits in a separate `if`, which CodeQL does not treat as sanitizing the returned value -- and it is right not to: a value rebuilt from the input is only ever as good as the checks applied to it. The post-merge SARIF shows the flow running straight through the new helper and out the other side; all four sinks route through the join at artifact.py:112: service/metrics/main.py:69,85 metric_name (FastAPI path parameter) artifact.py:70,97 identifier -> separate_inside_and_outside... artifact.py:108,112 segments -> SEP.join(segments) service/metrics/main.py:85,107 safe_metric_name operators.py:1193,1212 artifact_identifier artifact.py:612,628 artifact_rep artifact.py:303-306 open(path) <- cs-12, cs-13, cs-14 utils.py:221-222 open(path) <- cs-18 The fix stops deriving the value from the request at all. sanitize_catalog_ identifier() now resolves the requested name against the names the local catalogs publish and returns *the catalog's own string*, so what reaches a loader originates in trusted data: canonical = _lookup_catalog_name(name) if canonical is None: raise IllegalCatalogNameError(identifier) return canonical, args The returned object is the one held in the catalog index, not the caller's -- asserted by identity in the tests, which is what distinguishes this from #1995. The syntactic checks are kept as a cheap pre-filter, but they are no longer what the guarantee rests on. Supporting changes: - _catalog_name_index() builds {name: name} from catalog.ls(), cached via lru_cache_decorator and keyed on a generation counter. Lookups are ~1us with no filesystem walk per request (2000 in 2.2ms); the walk itself is ~35ms for 4923 entries. - add_to_catalog() calls invalidate_catalog_name_index(), so an artifact saved at runtime stays resolvable in the same process. - IllegalCatalogNameError deliberately does not distinguish "malformed" from "not published", so the error cannot be used to probe which entries exist. Deliberate behaviour change: the metrics service now rejects a well-formed name that the catalogs do not publish, where before it would attempt to load it. A name published *only* by the remote GithubCatalog is also rejected, because ls() walks local catalogs only. This was chosen over consulting the remote catalog, which would add an HTTP request per lookup and would once again vouch for a name rebuilt from the request. The remote catalog mirrors this repository at the release tag, so the practical gap is a name newer than the installed release, or one in an operator's own remote catalog. Documented on _catalog_name_index() and pinned by a test so a future change here is a decision rather than an accident. Also corrects the inaccurate "shares nothing with the raw input" rationale left in the sanitizer's docstring by #1995. Verified: the arbitrary-file read, the planted-artifact load, traversal, Windows paths, URLs, null bytes and malformed brackets are all refused with 400, with no file content in the error; metrics.accuracy, metrics.rouge, metrics.f1_macro and metrics.f1_macro[ignore_case=False] all still resolve and the override applies. Tests: 23 in tests/library/test_path_injection_service.py (up from 20). The new ones assert the returned name is the catalog's string object and not the caller's, that unpublished names are rejected, and that the index matches the local walk. tests/library: 802 passed. The 3 failures (test_metrics.py:: test_f1_strings, test_utils.py::test_require_errors, test_type_utils.py:: test_parse_malformed_string) reproduce identically on a clean tree and come from optional dependencies missing locally. ruff 0.1.6 check and format clean. Reported by: IBM OSPO security scanning (CodeQL default setup) Closes: #1996 Signed-off-by: Yoav Katz <katz@il.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remediates the 4 open
py/path-injectionCodeQL alerts (HIGH, CWE-22) from the IBM OSPO security notification (#1994): cs-12, cs-13, cs-14 (artifact.py:238,240) and cs-18 (utils.py:222).Why the alerts were still open
The previous fix (#1993) blocked the attack at runtime, but the alerts were right to stay open. It validated a derived value and then passed the original one on:
Checking
requested_namesays nothing aboutmetric_name, so nothing about the value that actually reached the loader. It happened to be safe only because the two strings agreed on the part that mattered — a property no check enforced and any later edit could break. CodeQL reported it as unsanitized because, as dataflow, it is.The taint path in the current SARIF is unchanged from before that fix:
cs-18is a fresh alert raised against the fix commit itself, replacing cs-15/cs-16 at the same sink: the flow survived, only its line numbers moved.The fix
Makes the sanitization load-bearing rather than incidental.
artifact.py— newsanitize_catalog_identifier(). Validates a reference as a catalog name and returns the name it validated, rebuilt from its checked segments, plus the bracketed overrides already parsed. Callers pass on the value that was checked, so no unvalidated substring can reach a loader and the guarantee no longer depends on two variables staying in agreement. Rejections raiseIllegalCatalogNameError(aValueErrorsubclass), so a caller can answer 400 for "not a catalog name" without catching unrelated failures.service/metrics/main.py— use only the sanitized name.compute()passessafe_metric_nameto the loader and nevermetric_name, which is now consumed by the sanitizer and nothing else. The raw value is also kept out of the log lines, so it cannot be used to forge log entries.operators.py—get_artifact()takesoverwrite_kwargs. Since overrides now travel beside the name instead of inside it, they are applied as keyword overrides. The cache key includes the overrides, so an overridden artifact cannot be served from, or written to, the plain artifact's entry. The parameter is optional and all seven existing callers are unaffected.fetch_artifact()'s support for loading an artifact from a file path is again left intact: it is a documented library feature, and the untrusted input is blocked at the service boundary where the expectation is narrower.Two things worth a reviewer's attention
[n=[1,2]]and nested artifact references do not round-trip through a string. That is why overrides are passed separately rather than rebuilt into the identifier.metrics.accuracy[unclosed) previously raised aValueErrorthat reached the endpoint's broadexceptand became a 500; it is now a clean 400.Verification
The planted-file read and the arbitrary-file read are both refused with 400, with no file content in the error. Legitimate names, and names with overrides, still resolve and the override is applied.
tests/library/test_path_injection_service.py, extended to 20 tests. Three fail against the #1993 code, including one that namesget_artifact(metric_name)as the raw-value use that kept the alerts open — that class asserts the endpoint never passes the raw parameter onward, so this regression cannot recur silently. Added coverage for the sanitizer's rejections (absolute paths, traversal, Windows paths, URLs, null bytes, malformed brackets, non-strings), for overrides still applying, and for the cache not being poisoned across overridden and plain fetches. The earlier tests pinningload_json's non-disclosure and file-path loading are kept.tests/library: 799 passed. The 3 failures (test_metrics.py::test_f1_strings,test_utils.py::test_require_errors,test_type_utils.py::test_parse_malformed_string) reproduce identically on a cleanmaintree and come from optional dependencies missing locally.Lint:
ruff0.1.6 (the version pinned in.pre-commit-config.yaml) check and format,codespell, and the relative-imports hook all pass. Note that a newer localruffreformats ~250 lines of unrelated code inoperators.py/artifact.py, so the pinned version is the one to check against.Reported by: IBM OSPO security scanning (CodeQL default setup)
Closes #1994
🤖 Generated with Claude Code