Skip to content

feat(mech): report who runs a mech and how it has behaved (KYA phase 2) - #263

Merged
DIvyaNautiyal07 merged 9 commits into
mainfrom
feat/mech-operator-report
Oct 12, 2026
Merged

DIvyaNautiyal07 merged 9 commits into
mainfrom
feat/mech-operator-report

Conversation

@DIvyaNautiyal07

@DIvyaNautiyal07 DIvyaNautiyal07 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What

Phase 2 of KYA, the reader side. A buyer's agent can look up who runs a mech and how it has behaved before it pays.

  • ToolManager.mech_report(mech_address, metadata) extends terms_report (every existing key unchanged) with:
    • operator: the domain the manifest claims, as domain and url, with domain_verified true only when https://<domain>/.well-known/agent-registration.json lists the chain's ERC-8004 identity registry with the mech's agent id, and verification_note when not. The name is claimed_name with a name_note that it is unchecked. The agent id is read from the service registry indexer and is not the Olas service id except by coincidence (optimism 195 is agent 516, polygon 21 is 25, base 112 is 114).
    • benchmarks: each tool's https link, as the operator's claim, with benchmark_note. No figure.
    • delivery: received_requests, self_delivered and delivered_by_others (the subgraph's receivedRequests, selfDeliveredFromReceived, deliveredByOthersFromReceived), since (ISO 8601 UTC) and age_days from the CreateMech block, with a note that the counts cover on-chain and settled off-chain requests, not split by path.
    • payment: the method, with no price. The contract quotes the charge at request time; maxDeliveryRate is only a ceiling.
  • ToolManager.mech_report_or_terms(mech_address, metadata) is the report to read before paying: mech_report when the manifest and record can be read, otherwise the terms with report_note. Connect can drop its own copy of this fallback when it bumps the pin; mech_report itself stays fail-closed.
  • mechx mech info --chain-config <chain> --address <mech> prints that report as JSON.
  • registry_subgraph_url per chain in mechs.json, with MECHX_REGISTRY_SUBGRAPH_URL as override.
  • Version 0.24.0.
  • tomte 0.7.3, merged in from chore: bump tomte to 0.7.3 #264: on the 0.7.0 pin the Security step fails on an nltk advisory in the scanner's own env.

Why these shapes

  • Fail closed: an unreadable manifest raises ManifestError, no partial report. Any failure in the agent-id lookup or the domain check leaves the domain unverified with a note, and is tried again next call rather than cached as a no.
  • The manifest is operator-written. Text is type-checked, stripped, capped and rejected on control characters; the domain must be a bare lowercase hostname ending in a top-level domain; links must parse as https with a host. Nothing in it is followed.
  • The proof is fetched from the exact host with redirects disabled, and a 3xx gets its own note. The body is capped at 64 KB and each read times out after 5 s.
  • The report's subgraph reads time out after 10 s each.
  • Mech.owner is the creation-tx sender and is never updated on transfer, so it is not reported.

Not in this PR

  • mech list columns, in their own PR if wanted.
  • The owner's other mechs, from the service owner on the registry indexer, as a follow-up.

Known data gaps

  • No live Valory manifest carries operator or benchmark yet.
  • www.valory.xyz/.well-known/agent-registration.json returns 404, and its generator lists service ids rather than ERC-8004 agent ids, so Valory mechs read as unverified until it is regenerated and deployed.

🤖 Generated with Claude Code

DIvyaNautiyal07 and others added 2 commits October 9, 2026 12:11
A buyer's agent can now read, before it pays, who a mech's manifest says
runs it, whether that domain backs the claim, what the operator claims per
tool, and the mech's on-chain delivery record.

- `ToolManager.mech_report(mech_address, metadata)` extends `terms_report`
  with `operator` (name, domain, `domain_verified`, a note when not
  verified), `benchmarks` per tool as link-only claims, `delivery`
  (receivedRequests, selfDeliveredFromReceived,
  deliveredByOthersFromReceived, age from the CreateMech block) and
  `payment` (method, no price). An unreadable manifest raises
  `ManifestError` rather than yielding a partial report.
- The domain check fetches `/.well-known/agent-registration.json` from the
  exact host without following redirects and matches the chain's ERC-8004
  identity registry with the mech's agent id, read from the service
  registry indexer (`registry_subgraph_url` per chain,
  `MECHX_REGISTRY_SUBGRAPH_URL` override). The agent id is not the service
  id except by coincidence. An unavailable proof is unverified and fetched
  again next time, never cached as a no.
- Manifest text is data: hostnames are lowercase ASCII labels matched in
  full, URLs are parsed and must be https with a host, text is stripped,
  capped and rejected on control characters.
- `mechx mech info --chain-config <chain> --address <mech>` prints the
  report; `mechx mech list` gains the delivery counters, creation date and
  claimed operator as columns. The counters cover on-chain and settled
  off-chain requests and are labelled so.
- tomte pinned to 0.7.3 from PyPI in CI, the dev group and the tox
  template pin; it carries the fleet baseline for the scanner's own
  dependencies that the Security step was failing on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@OjusWiZard OjusWiZard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things in the domain proof handling, each checked by running it at d002711:

  • mech_client/domain/operator.py, hostname: hex-form IPv4 such as 0x7f.0x1 passes, because the last label contains a letter (x), and socket.getaddrinfo("0x7f.0x1", 443) resolves it to 127.0.0.1. Rejecting 0x-prefixed or all-digit labels, or requiring an alphabetic or xn-- last label, would match the "IPv4 kept out" intent. Worth a test.
  • operator.py, _agent_id_of: str.isdigit() is true for "²" and for very long digit strings, and int() then raises ValueError. isascii() and isdecimal() with a length cap would avoid it.
  • operator.py, fetch_domain_proof: response.json() on a deeply nested body (e.g. "[" * 100000) raises RecursionError, which the except ValueError does not catch.
  • Neither of those is caught by verify_domain / claimed_operator, so in mech list the exception comes back out of executor.map in _claimed_operator_labels and the whole table fails for every mech on the chain. Having claimed_operator treat any error as unverified would contain it.
  • fetch_domain_proof: requests.get(..., timeout=5) without stream=True reads the whole body with no size cap, and the timeout is per read, not total. Streaming with a small cap (say 64 KB) and an overall deadline would bound it.

Smaller:

  • mech_cmd.py, query_mech_creations: a failure there now fails all of mech list; a blank "Since" column would degrade more gently.
  • The new mech list columns sit before Metadata Link, Terms and Operator, which shifts those columns for anything reading by position.
  • Several docstrings in operator.py (Benchmark, https_url, fetch_domain_proof, proof_matches), the comment block at operator.py:34-36 and the comment at manager.py:99-101 carry rationale or restate the code; one line each would do.
  • The PR body names the delivery keys in camelCase (receivedRequests), while mech_report returns snake_case (received_requests).

The rest reads well: redirects off with 3xx as not found, unavailable kept separate from not verified, wrongly typed manifest fields handled, terms_report keys unchanged. Unit tests pass (1146).

…ailure

- A hostname's last label must be a top-level domain: letters only or
  punycode. This keeps every IPv4 spelling out, including the hex form a
  resolver accepts.
- A proof's agent id is read only from ASCII decimal digits, capped at the
  width of a uint256, so non-ASCII digits and overlong strings never reach
  int().
- The proof body is streamed under a 64 KB cap with a deadline covering the
  whole fetch, and parsed with recursion failures treated as not a proof.
- Any error inside the domain check leaves the claim unverified with a
  note; it no longer escapes `claimed_operator` into `mech list`.
- A failed creation-time query leaves the Since column blank instead of
  failing the listing.
- The new `mech list` columns come after the existing ones, so readers
  keyed by position see no shift.
- Trimmed the operator docstrings and comments to the invariant.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@DIvyaNautiyal07

Copy link
Copy Markdown
Contributor Author

Thanks, all fixed in 1665b5f. The last label now has to be letters or punycode, which shuts out hex and decimal IPv4. Agent ids are ASCII decimal only, capped at uint256 width. The proof is streamed with a 64 KB cap and a deadline over the whole fetch, and a RecursionError on parse counts as not a proof. Any error inside the domain check now leaves the claim unverified, so it can't take down mech list, and a failed creation query just blanks Since. New columns moved to the end, docstrings trimmed, and the PR body now uses the snake_case keys.

@bennyjo bennyjo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One blocker: the unchecked operator name shows next to a verified domain, plus two questions on indexer timeouts and redirects, because each one changes what a buyer's agent sees before it pays.

Comment thread mech_client/domain/tools/manager.py Outdated
Comment thread mech_client/infrastructure/subgraph/queries.py
Comment thread mech_client/domain/operator.py
- The operator block leads with the domain, which is the only part the
  proof checks. The name is reported as `claimed_name` with a `name_note`
  saying it is unchecked, and no longer sits next to `domain_verified` as
  if the check covered it. `mech list` renders the domain with its status,
  then the name the operator calls itself.
- A 3xx from the proof URL is its own verdict, with a note that the
  manifest must name the exact host that serves the proof, instead of
  reading as no proof.
- The subgraph reads a report makes (record, creation times, ERC-8004
  agent id) time out after 10 seconds each, and a timed-out record read is
  a SubgraphError. The full listing query keeps the client default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bennyjo
bennyjo self-requested a review October 9, 2026 11:20

@OjusWiZard OjusWiZard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, re-checked at 33d99a7 by running each case: hex and decimal IPv4 are rejected, agent ids are ASCII decimal only, nested JSON is not a proof, the 64 KB cap holds (also on decoded gzip), and errors in the domain check leave it unverified without taking down mech list.

One thing left: the deadline doesn't bound a slow drip. _read_proof_body only checks time.monotonic() > deadline after iter_content(PROOF_CHUNK_BYTES) yields, and timeout=5 is per read. Against a local server that answers 200 and then sends one byte every 0.5s, fetch_domain_proof had not returned after 25s, so mech info hangs and mech list waits on that row. A smaller chunk size alone won't cap it; it needs a wall-clock bound, e.g. running the fetch in a worker and taking future.result(timeout=PROOF_FETCH_TIMEOUT). The CHANGELOG line about a deadline over the whole fetch would then hold.

Separately, on valory-xyz/connect#85 I've asked for its mech_report wrapper (indexer record lookup, fallback to the terms, report_note) to move here, the same way mech_terms did. Since this PR is unreleased, it could land in 0.24.0.

Minor: the "Any failure here leaves the domain unverified" comment appears at manager.py:99-101 and manager.py:139, with a near-copy in mech_cmd.py; once is enough.

- The proof body is read with `HTTPResponse.read1`, which returns as soon
  as any bytes arrive, and the deadline is checked after every read. With
  `iter_content` a body sent a byte at a time never reached the check.
- The caller waits on a capped worker pool for at most
  PROOF_FETCH_TIMEOUT seconds; a fetch it stopped waiting for ends at its
  own next read, so it frees its thread and never holds up exit.
- urllib3 errors raised by `read1` count as unavailable, like requests
  errors. `urllib3>=2.6.0,<3` is now a direct dependency: `read1` with
  decoding arrived in 2.2.0 and its decompressed reads are bounded from
  2.6.0.
- `ToolManager.mech_report_or_terms` is the report to read before paying:
  the full report when the manifest and record can be read, otherwise the
  terms with `report_note`. It replaces the wrapper Pearl Connect carried.
  `mech_report` stays fail-closed.
- One copy of the "any failure leaves the domain unverified" comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DIvyaNautiyal07

Copy link
Copy Markdown
Contributor Author

Thanks. The drip is bounded now: the proof is read with read1, which returns on every arrival, and the worker checks the deadline after each read, while the caller waits on a capped pool for at most 5s. Closing the response from the waiting thread doesn't work, it blocks on the reader's lock, so the worker stops itself instead. A local drip server now returns at 5.0s and the process exits at about 6s, and a 200 MB gzip bomb still decodes in 8 KB reads. That needs urllib3 2.6+, so it's now a direct dependency. The Connect wrapper is here as ToolManager.mech_report_or_terms, with mech_report left fail-closed for mech info. Duplicate comment removed.

@bennyjo bennyjo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for turning the three points from my last review around quickly. They're fixed at 2f69239, and so is the slow-drip bound Ojus found.

What blocks this now is size. The PR is +3,665/-391, and the KYA ask for this repo is narrower. I sorted every function, class, test class, and doc change against that ask, which is to resolve the mech's ERC-8004 agent id, check the operator domain against the proof, and report the operator, the owner's other mechs, age, price or payment method, the delivery record labeled settled-only, and a benchmark link per tool, with no karma shown as reliability. Line counts include tests and docs.

Needed for the spec (+1,620/-14)

  • Agent id lookup, query_erc8004_agent_id and query_service_agent_id (+193)
  • Registry subgraph URL per chain and its env override (+110)
  • Proof match and verdict, proof_matches, verify_domain, the four proof states, and their notes (+392)
  • Reading the operator block (+77)
  • Report assembly, mech_report, delivery_record, the settled-only note, and ManifestError (+307/-1)
  • Payment method in place of price (+72)
  • Mech record query with the 10 s timeout, for age and delivery counts (+403/-13)
  • claimed_operator (+37) and the docs for the report (+29)

Needed, but larger than the spec requires (+1,316/-15, about 375 in simpler form)

  • Proof fetcher with a thread pool, a read1 loop, and a direct urllib3 pin (+358). Once the list columns are out, a slow drip can only stall the dripping operator's own report, so a plain request with a timeout and a 64 KB cap would do, about 100.
  • Three sanitizers and four length caps (+264). One hostname check that still rejects IPv4 and one bounded-text check would do, about 75.
  • Benchmarks with metric and window validation (+169). The spec asks for a link per tool, about 45.
  • Two layers of failure handling plus the claimed name (+159). One try in claimed_operator would do, about 65, and it fixes the wrong note a mech with no ERC-8004 identity gets.
  • mech info with a 50-line custom renderer (+366/-15). Printing the report as JSON would do, about 90.

Not needed for the spec (+729/-362)

  • Five new mech list columns with a per-row agent-id lookup and proof fetch (+262/-47)
  • query_mech_creations and the extra listing fields (+151/-2)
  • Rewrite of the existing listing tests onto _mech_row, which is 300 of the 391 deleted lines (+181/-300)
  • mech_report_or_terms, which nothing in this repo calls (+115)
  • tomte moving from the git tag to PyPI 0.7.3 (+15/-10)
  • Version bump to 0.24.0 (+5/-3)

Missing from the spec

  • The owner's other mechs isn't in the report. Mech.owner is the creation sender, so it needs another source, such as the service owner on the registry indexer this PR already reads.

Smaller version

Keeping the first group and the simpler forms of the second comes to about 2,000 added lines, a little over half of this diff.

Proposed split

  1. This PR keeps the KYA report, meaning mech_report, the agent-id lookup, the proof check, and mech info, with the simpler forms above.
  2. A separate PR adds the mech list columns and their queries, if we still want them once the report is in use. That's where the threaded fetcher earns its place, because one slow row stalls the whole table.
  3. A chore PR moves the tomte pin.
  4. mech_report_or_terms lands with the Connect switch-over that calls it.
  5. The version bump goes wherever we cut the release, in this PR or a chore PR, as we've done both ways before.
  6. A follow-up PR adds the owner's other mechs.

Smaller PRs are easier to review and to revert, and they keep each change to what we asked for.

Comment thread mech_client/cli/commands/mech_cmd.py Outdated
Comment thread pyproject.toml
Comment thread tests/unit/cli/test_mech_cmd.py Outdated
Comment thread mech_client/domain/operator.py Outdated
Comment thread mech_client/domain/operator.py Outdated
Comment thread mech_client/domain/operator.py Outdated
Comment thread mech_client/domain/tools/manager.py Outdated
Comment thread mech_client/cli/commands/mech_cmd.py Outdated
Comment thread mech_client/domain/tools/manager.py
DIvyaNautiyal07 and others added 2 commits October 9, 2026 21:12
The report on one mech before paying is what this change is for; the rest
moves to its own PRs.

- `mech list` and its tests are back to main. The new columns, the
  creation-time query and the listing fields go with a separate PR if the
  report shows they are wanted.
- The proof fetch is a plain request again: no redirects, a 64 KB cap
  and a 5 s timeout per read. A host that trickles its body stalls only
  the report on its own mech. The worker pool, `read1` and the urllib3
  pin are gone.
- One bounded-text check and one hostname check; the https check is that
  text check plus a parse. One text length cap.
- `benchmarks` maps each tool to its https link.
- One failure layer in `claimed_operator`: no ERC-8004 identity gets its
  own note, and a failed lookup or check reads as could not be checked.
- `mech info` prints the report as JSON.
- `mech_report_or_terms` lands with the Connect switch-over, and the
  tomte bump in its own chore PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0.7.3 carries the fleet baseline for the advisory in the safety scanner's
own dependencies, which is what the Security step reports on 0.7.0. The
scanned env installs nothing but the scanner, so this repo cannot clear it
otherwise.

0.7.3 is on PyPI, so this also resolves the standing TODOs and pins by
version rather than by git tag, in the workflow installs, the dev group
and the tox template pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DIvyaNautiyal07

Copy link
Copy Markdown
Contributor Author

Thanks, agreed on the split. This PR is now the report only: agent-id lookup, proof check, mech_report, and mech info printing JSON, at +2,565/-19 with tests (fe19e37). The list columns, query_mech_creations and the listing test rewrite are out and the listing tests match main. mech_report_or_terms comes back with the Connect switch-over. tomte moves to its own chore PR, #264, which needs to land first because the safety step fails on the old pin. The version bump stays here so this PR is the release. The owner's other mechs goes in a follow-up, from the service owner on the registry indexer.

@bennyjo
bennyjo self-requested a review October 9, 2026 16:46

@OjusWiZard OjusWiZard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at fe19e37; the narrowed report looks good and tests pass (1154). One request: please keep mech_report_or_terms here.

It doesn't need a caller in mech-client to belong in it, because it's public API. That's already the pattern: terms_report came in with 93957e6 with no caller in this repo ("so integrations get … from one place"), and ToolManager.get_offchain_url has none on main today.

mech-client exists so a developer can add mech support to their software, and Connect is one consumer of that. The record lookup, the fallback to the terms and report_note are mech logic, so they should come from here. If it stays in Connect, the next consumer that talks to mechs either duplicates it or moves it upstream then. Landing it now also lets valory-xyz/connect#85 drop its own copy when it bumps the pin, rather than waiting for a later pair of PRs.

…aying

`ToolManager.mech_report_or_terms` returns `mech_report` when the manifest
and the mech's record can be read, and otherwise the terms alone with
`report_note` saying why. The record lookup and the fallback to the terms
are mech logic, so integrations get them from mech-client rather than each
carrying a copy. `mech_report` stays fail-closed for `mech info`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DIvyaNautiyal07 added a commit to valory-xyz/connect that referenced this pull request Oct 9, 2026
- `mech_tools` and `mech_request` call mech-client's
  `ToolManager.mech_report_or_terms` directly, and connect/mech_report.py
  is gone. The record lookup, the fallback to the terms and `report_note`
  are mech logic and now live in one place upstream.
- mech-client is pinned to valory-xyz/mech-client#263 at da08f07, where
  the operator's domain is its identity and the name is `claimed_name`,
  unchecked, and benchmarks are one link per tool.
- The real-client guard checks that shape with a streaming proof fake, and
  two more guards run the real fallback for an unreadable record and an
  unreadable manifest. The other tests cover only what connect does: which
  mech and document it passes, the merge, the order before any allowance,
  and that a request still goes out after a fallback.
- The skill tells the agent to name the operator's domain and whether it
  is verified, and to treat the name as the operator's own claim. README,
  CLAUDE.md and the MCP tool descriptions say the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DIvyaNautiyal07

Copy link
Copy Markdown
Contributor Author

Agreed, it's back in da08f07: ToolManager.mech_report_or_terms returns the full report when the manifest and record can be read, otherwise the terms with report_note, and mech_report stays fail-closed for mech info. @bennyjo this reverses what I said in your thread, since it's public API for integrations like terms_report, and it lets connect#85 drop its own copy when it bumps the pin.

@OjusWiZard OjusWiZard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for keeping mech_report_or_terms here. Re-checked at 1957b08: an unreadable manifest or record falls back to the terms with report_note, otherwise it's the full report, and tests pass (1161).

@bennyjo bennyjo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All nine points from my last review are resolved at 1957b08, and I'm fine with mech_report_or_terms staying as public API since connect#85 already calls it, and with the tomte bump riding along as its own commit, so this is good to merge with the owner's other mechs as the follow-up you named.

@DIvyaNautiyal07
DIvyaNautiyal07 merged commit c119ad0 into main Oct 12, 2026
30 checks passed
@DIvyaNautiyal07
DIvyaNautiyal07 deleted the feat/mech-operator-report branch October 12, 2026 08:22
DIvyaNautiyal07 added a commit to valory-xyz/connect that referenced this pull request Oct 12, 2026
0.24.0 is released, so the git source on valory-xyz/mech-client#263 goes
and the lock resolves the published package. The release carries no
library change since the pinned commit, only its CI tooling bump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants