Skip to content

Security: Pass the validated name onward in metrics service (#cs-12, #cs-13, #cs-14, #cs-18) - #1995

Merged
yoavkatz merged 1 commit into
mainfrom
security/path-injection-cs-12-18
Nov 4, 2026
Merged

yoavkatz merged 1 commit into
mainfrom
security/path-injection-cs-12-18

Conversation

@yoavkatz

@yoavkatz yoavkatz commented Nov 4, 2026

Copy link
Copy Markdown
Member

Remediates the 4 open py/path-injection CodeQL alerts (HIGH, CWE-22) from the IBM OSPO security notification (#1994): cs-12, cs-13, cs-14 (artifact.py:238,240) and cs-18 (utils.py:222).

Why the alerts were still open

The previous fix (#1993) blocked the attack at runtime, but the alerts were right to stay open. It validated a derived value and then passed the original one on:

requested_name, _ = separate_inside_and_outside_square_brackets(metric_name)
if not is_name_legal_for_catalog(requested_name):   # checks requested_name
    raise HTTPException(status_code=400, ...)
...
metric = ArtifactFetcherMixin.get_artifact(metric_name)   # passes the raw value

Checking requested_name says nothing about metric_name, so nothing about the value that actually reached the loader. It happened to be safe only because the two strings agreed on the part that mattered — a property no check enforced and any later edit could break. CodeQL reported it as unsanitized because, as dataflow, it is.

The taint path in the current SARIF is unchanged from before that fix:

service/metrics/main.py:69,100  metric_name (FastAPI path parameter)
operators.py:1190-1192          fetch_artifact(artifact_identifier)
artifact.py:546,562-563         fetch_artifact(artifact_rep)
artifact.py:238,240             open(path)            <- cs-12, cs-13, cs-14
utils.py:203-204,221-222        open(path)            <- cs-18

cs-18 is a fresh alert raised against the fix commit itself, replacing cs-15/cs-16 at the same sink: the flow survived, only its line numbers moved.

The fix

Makes the sanitization load-bearing rather than incidental.

  1. artifact.py — new sanitize_catalog_identifier(). Validates a reference as a catalog name and returns the name it validated, rebuilt from its checked segments, plus the bracketed overrides already parsed. Callers pass on the value that was checked, so no unvalidated substring can reach a loader and the guarantee no longer depends on two variables staying in agreement. Rejections raise IllegalCatalogNameError (a ValueError subclass), so a caller can answer 400 for "not a catalog name" without catching unrelated failures.

  2. service/metrics/main.py — use only the sanitized name. compute() passes safe_metric_name to the loader and never metric_name, which is now consumed by the sanitizer and nothing else. The raw value is also kept out of the log lines, so it cannot be used to forge log entries.

  3. operators.py — get_artifact() takes overwrite_kwargs. Since overrides now travel beside the name instead of inside it, they are applied as keyword overrides. The cache key includes the overrides, so an overridden artifact cannot be served from, or written to, the plain artifact's entry. The parameter is optional and all seven existing callers are unaffected.

fetch_artifact()'s support for loading an artifact from a file path is again left intact: it is a documented library feature, and the untrusted input is blocked at the service boundary where the expectation is narrower.

Two things worth a reviewer's attention

  • Re-serializing the overrides into the name would have been a bug. The parser returns typed values, so [n=[1,2]] and nested artifact references do not round-trip through a string. That is why overrides are passed separately rather than rebuilt into the identifier.
  • Fixes a latent 500. Malformed brackets (metrics.accuracy[unclosed) previously raised a ValueError that reached the endpoint's broad except and became a 500; it is now a clean 400.

Verification

The planted-file read and the arbitrary-file read are both refused with 400, with no file content in the error. Legitimate names, and names with overrides, still resolve and the override is applied.

tests/library/test_path_injection_service.py, extended to 20 tests. Three fail against the #1993 code, including one that names get_artifact(metric_name) as the raw-value use that kept the alerts open — that class asserts the endpoint never passes the raw parameter onward, so this regression cannot recur silently. Added coverage for the sanitizer's rejections (absolute paths, traversal, Windows paths, URLs, null bytes, malformed brackets, non-strings), for overrides still applying, and for the cache not being poisoned across overridden and plain fetches. The earlier tests pinning load_json's non-disclosure and file-path loading are kept.

tests/library: 799 passed. The 3 failures (test_metrics.py::test_f1_strings, test_utils.py::test_require_errors, test_type_utils.py::test_parse_malformed_string) reproduce identically on a clean main tree and come from optional dependencies missing locally.

Lint: ruff 0.1.6 (the version pinned in .pre-commit-config.yaml) check and format, codespell, and the relative-imports hook all pass. Note that a newer local ruff reformats ~250 lines of unrelated code in operators.py/artifact.py, so the pinned version is the one to check against.

Reported by: IBM OSPO security scanning (CodeQL default setup)

Closes #1994

🤖 Generated with Claude Code

…#cs-13, #cs-14, #cs-18)

Remediates the 4 py/path-injection CodeQL alerts (HIGH, CWE-22) still open after
the previous fix, reported in the IBM OSPO security notification (#1994):
cs-12, cs-13, cs-14 (src/unitxt/artifact.py:238,240) and cs-18
(src/unitxt/utils.py:222).

The earlier fix (#1993) blocked the attack at runtime but left the alerts open,
and the alerts were right to stay open. It validated a derived value and then
passed the original one on:

    requested_name, _ = separate_inside_and_outside_square_brackets(metric_name)
    if not is_name_legal_for_catalog(requested_name):
        raise HTTPException(status_code=400, ...)
    ...
    metric = ArtifactFetcherMixin.get_artifact(metric_name)   # <- raw value

Checking requested_name says nothing about metric_name, so nothing about the
value that actually reached the loader. It happened to be safe only because the
two strings agreed on the part that mattered -- a property no check enforced and
any later edit could break. CodeQL reported it as unsanitized because, as
dataflow, it is. The taint path in the current SARIF is unchanged from before
the fix:

  service/metrics/main.py:69,100  metric_name (FastAPI path parameter)
  operators.py:1190-1192          fetch_artifact(artifact_identifier)
  artifact.py:546,562-563         fetch_artifact(artifact_rep)
  artifact.py:238,240             open(path)            <- cs-12, cs-13, cs-14
  utils.py:203-204,221-222        open(path)            <- cs-18

cs-18 is a fresh alert raised against the fix commit itself, replacing cs-15 and
cs-16 at the same sink: the flow survived, only its line numbers moved.

The fix makes the sanitization real rather than incidental:

1. artifact.py -- new sanitize_catalog_identifier().

   Validates a reference as a catalog name and returns the name it validated,
   rebuilt from its checked segments, plus the bracketed overrides already
   parsed. Callers pass on the value that was checked, so no unvalidated
   substring can reach a loader and the guarantee no longer depends on two
   variables staying in agreement. Rejections raise IllegalCatalogNameError, a
   ValueError subclass, so a caller can answer 400 for "not a catalog name"
   without catching unrelated failures.

   Malformed brackets ("metrics.accuracy[unclosed") now reject cleanly too;
   before, the parser's ValueError reached the endpoint's broad except and
   became a 500.

2. service/metrics/main.py -- use only the sanitized name.

   compute() passes safe_metric_name to the loader and never metric_name, which
   is now consumed by the sanitizer and nothing else. The raw value is also kept
   out of the log lines, so it cannot be used to forge log entries.

3. operators.py -- get_artifact() takes overwrite_kwargs.

   Since overrides now travel beside the name instead of inside it, they are
   applied as keyword overrides. Re-serializing them into the identifier was not
   an option: the parser returns typed values, so "[n=[1,2]]" and nested
   artifact references do not round-trip through a string. The cache key
   includes the overrides, so an overridden artifact cannot be served from, or
   written to, the plain artifact's entry. The parameter is optional and all
   seven existing callers are unaffected.

fetch_artifact()'s support for loading an artifact from a file path is again
left intact: it is a documented library feature, and the untrusted input is
blocked at the service boundary where the expectation is narrower.

Verified: the planted-file read and the arbitrary-file read are both refused
with 400, with no file content in the error; legitimate names, and names with
overrides, still resolve and the override is applied.

Tests: tests/library/test_path_injection_service.py, extended to 20 tests.
Three of them fail against the #1993 code, including one that names
get_artifact(metric_name) as the raw-value use that kept the alerts open --
that class asserts the endpoint never passes the raw parameter onward, so the
regression this commit fixes cannot recur silently. Added coverage for the
sanitizer's rejections (absolute paths, traversal, Windows paths, URLs, null
bytes, malformed brackets, non-strings), for overrides still applying, and for
the cache not being poisoned across overridden and plain fetches. The earlier
tests pinning load_json's non-disclosure and file-path loading are kept.

tests/library: 799 passed. The 3 failures (test_metrics.py::test_f1_strings,
test_utils.py::test_require_errors, test_type_utils.py::test_parse_malformed_string)
reproduce identically on a clean main tree and come from optional dependencies
missing locally. ruff 0.1.6 (the pinned pre-commit version) check and format,
codespell and the relative-imports hook all pass.

Reported by: IBM OSPO security scanning (CodeQL default setup)
Closes: #1994

Signed-off-by: Yoav Katz <katz@il.ibm.com>
@yoavkatz
yoavkatz merged commit 603f1c5 into main Nov 4, 2026
16 of 23 checks passed
yoavkatz added a commit that referenced this pull request Nov 7, 2026
…cs-14, #cs-18) (#1997)

The 4 py/path-injection alerts (HIGH, CWE-22) reported in #1994 are still open
after #1995, and are reported again in #1996: cs-12, cs-13, cs-14 and cs-18.
CodeQL analysed the merge commit (603f1c5, analysis 2093723778) and still
found all four.

#1995 was wrong, and its own commit message overstated the guarantee. It claimed
the rebuilt name "shares nothing with the raw input". As dataflow that is false:

    segments = name.split(constants.catalog_hierarchy_sep)
    if not all(is_name_legal_for_catalog(segment) for segment in segments):
        raise IllegalCatalogNameError(identifier)
    return constants.catalog_hierarchy_sep.join(segments), args

split() and join() propagate taint, so the returned string is still derived from
the request. The re.match guard sits in a separate `if`, which CodeQL does not
treat as sanitizing the returned value -- and it is right not to: a value
rebuilt from the input is only ever as good as the checks applied to it. The
post-merge SARIF shows the flow running straight through the new helper and out
the other side; all four sinks route through the join at artifact.py:112:

  service/metrics/main.py:69,85   metric_name (FastAPI path parameter)
  artifact.py:70,97               identifier -> separate_inside_and_outside...
  artifact.py:108,112             segments -> SEP.join(segments)
  service/metrics/main.py:85,107  safe_metric_name
  operators.py:1193,1212          artifact_identifier
  artifact.py:612,628             artifact_rep
  artifact.py:303-306             open(path)            <- cs-12, cs-13, cs-14
  utils.py:221-222                open(path)            <- cs-18

The fix stops deriving the value from the request at all. sanitize_catalog_
identifier() now resolves the requested name against the names the local
catalogs publish and returns *the catalog's own string*, so what reaches a
loader originates in trusted data:

    canonical = _lookup_catalog_name(name)
    if canonical is None:
        raise IllegalCatalogNameError(identifier)
    return canonical, args

The returned object is the one held in the catalog index, not the caller's --
asserted by identity in the tests, which is what distinguishes this from #1995.
The syntactic checks are kept as a cheap pre-filter, but they are no longer what
the guarantee rests on.

Supporting changes:

- _catalog_name_index() builds {name: name} from catalog.ls(), cached via
  lru_cache_decorator and keyed on a generation counter. Lookups are ~1us with
  no filesystem walk per request (2000 in 2.2ms); the walk itself is ~35ms for
  4923 entries.
- add_to_catalog() calls invalidate_catalog_name_index(), so an artifact saved
  at runtime stays resolvable in the same process.
- IllegalCatalogNameError deliberately does not distinguish "malformed" from
  "not published", so the error cannot be used to probe which entries exist.

Deliberate behaviour change: the metrics service now rejects a well-formed name
that the catalogs do not publish, where before it would attempt to load it. A
name published *only* by the remote GithubCatalog is also rejected, because ls()
walks local catalogs only. This was chosen over consulting the remote catalog,
which would add an HTTP request per lookup and would once again vouch for a name
rebuilt from the request. The remote catalog mirrors this repository at the
release tag, so the practical gap is a name newer than the installed release, or
one in an operator's own remote catalog. Documented on _catalog_name_index() and
pinned by a test so a future change here is a decision rather than an accident.

Also corrects the inaccurate "shares nothing with the raw input" rationale left
in the sanitizer's docstring by #1995.

Verified: the arbitrary-file read, the planted-artifact load, traversal, Windows
paths, URLs, null bytes and malformed brackets are all refused with 400, with no
file content in the error; metrics.accuracy, metrics.rouge, metrics.f1_macro and
metrics.f1_macro[ignore_case=False] all still resolve and the override applies.

Tests: 23 in tests/library/test_path_injection_service.py (up from 20). The new
ones assert the returned name is the catalog's string object and not the
caller's, that unpublished names are rejected, and that the index matches the
local walk. tests/library: 802 passed. The 3 failures (test_metrics.py::
test_f1_strings, test_utils.py::test_require_errors, test_type_utils.py::
test_parse_malformed_string) reproduce identically on a clean tree and come from
optional dependencies missing locally. ruff 0.1.6 check and format clean.

Reported by: IBM OSPO security scanning (CodeQL default setup)
Closes: #1996

Signed-off-by: Yoav Katz <katz@il.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔒 [IBM OSPO Security Notification] — IBM/unitxt

1 participant