Repository navigation
feat(mech): report who runs a mech and how it has behaved (KYA phase 2) - #263
Conversation
A buyer's agent can now read, before it pays, who a mech's manifest says runs it, whether that domain backs the claim, what the operator claims per tool, and the mech's on-chain delivery record. - `ToolManager.mech_report(mech_address, metadata)` extends `terms_report` with `operator` (name, domain, `domain_verified`, a note when not verified), `benchmarks` per tool as link-only claims, `delivery` (receivedRequests, selfDeliveredFromReceived, deliveredByOthersFromReceived, age from the CreateMech block) and `payment` (method, no price). An unreadable manifest raises `ManifestError` rather than yielding a partial report. - The domain check fetches `/.well-known/agent-registration.json` from the exact host without following redirects and matches the chain's ERC-8004 identity registry with the mech's agent id, read from the service registry indexer (`registry_subgraph_url` per chain, `MECHX_REGISTRY_SUBGRAPH_URL` override). The agent id is not the service id except by coincidence. An unavailable proof is unverified and fetched again next time, never cached as a no. - Manifest text is data: hostnames are lowercase ASCII labels matched in full, URLs are parsed and must be https with a host, text is stripped, capped and rejected on control characters. - `mechx mech info --chain-config <chain> --address <mech>` prints the report; `mechx mech list` gains the delivery counters, creation date and claimed operator as columns. The counters cover on-chain and settled off-chain requests and are labelled so. - tomte pinned to 0.7.3 from PyPI in CI, the dev group and the tox template pin; it carries the fleet baseline for the scanner's own dependencies that the Security step was failing on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
OjusWiZard
left a comment
There was a problem hiding this comment.
A few things in the domain proof handling, each checked by running it at d002711:
mech_client/domain/operator.py,hostname: hex-form IPv4 such as0x7f.0x1passes, because the last label contains a letter (x), andsocket.getaddrinfo("0x7f.0x1", 443)resolves it to127.0.0.1. Rejecting0x-prefixed or all-digit labels, or requiring an alphabetic orxn--last label, would match the "IPv4 kept out" intent. Worth a test.operator.py,_agent_id_of:str.isdigit()is true for"²"and for very long digit strings, andint()then raisesValueError.isascii() and isdecimal()with a length cap would avoid it.operator.py,fetch_domain_proof:response.json()on a deeply nested body (e.g."[" * 100000) raisesRecursionError, which theexcept ValueErrordoes not catch.- Neither of those is caught by
verify_domain/claimed_operator, so inmech listthe exception comes back out ofexecutor.mapin_claimed_operator_labelsand the whole table fails for every mech on the chain. Havingclaimed_operatortreat any error as unverified would contain it. fetch_domain_proof:requests.get(..., timeout=5)withoutstream=Truereads the whole body with no size cap, and the timeout is per read, not total. Streaming with a small cap (say 64 KB) and an overall deadline would bound it.
Smaller:
mech_cmd.py,query_mech_creations: a failure there now fails all ofmech list; a blank "Since" column would degrade more gently.- The new
mech listcolumns sit beforeMetadata Link,TermsandOperator, which shifts those columns for anything reading by position. - Several docstrings in
operator.py(Benchmark,https_url,fetch_domain_proof,proof_matches), the comment block atoperator.py:34-36and the comment atmanager.py:99-101carry rationale or restate the code; one line each would do. - The PR body names the delivery keys in camelCase (
receivedRequests), whilemech_reportreturns snake_case (received_requests).
The rest reads well: redirects off with 3xx as not found, unavailable kept separate from not verified, wrongly typed manifest fields handled, terms_report keys unchanged. Unit tests pass (1146).
…ailure - A hostname's last label must be a top-level domain: letters only or punycode. This keeps every IPv4 spelling out, including the hex form a resolver accepts. - A proof's agent id is read only from ASCII decimal digits, capped at the width of a uint256, so non-ASCII digits and overlong strings never reach int(). - The proof body is streamed under a 64 KB cap with a deadline covering the whole fetch, and parsed with recursion failures treated as not a proof. - Any error inside the domain check leaves the claim unverified with a note; it no longer escapes `claimed_operator` into `mech list`. - A failed creation-time query leaves the Since column blank instead of failing the listing. - The new `mech list` columns come after the existing ones, so readers keyed by position see no shift. - Trimmed the operator docstrings and comments to the invariant. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Thanks, all fixed in 1665b5f. The last label now has to be letters or punycode, which shuts out hex and decimal IPv4. Agent ids are ASCII decimal only, capped at uint256 width. The proof is streamed with a 64 KB cap and a deadline over the whole fetch, and a RecursionError on parse counts as not a proof. Any error inside the domain check now leaves the claim unverified, so it can't take down |
bennyjo
left a comment
There was a problem hiding this comment.
One blocker: the unchecked operator name shows next to a verified domain, plus two questions on indexer timeouts and redirects, because each one changes what a buyer's agent sees before it pays.
- The operator block leads with the domain, which is the only part the proof checks. The name is reported as `claimed_name` with a `name_note` saying it is unchecked, and no longer sits next to `domain_verified` as if the check covered it. `mech list` renders the domain with its status, then the name the operator calls itself. - A 3xx from the proof URL is its own verdict, with a note that the manifest must name the exact host that serves the proof, instead of reading as no proof. - The subgraph reads a report makes (record, creation times, ERC-8004 agent id) time out after 10 seconds each, and a timed-out record read is a SubgraphError. The full listing query keeps the client default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OjusWiZard
left a comment
There was a problem hiding this comment.
Thanks, re-checked at 33d99a7 by running each case: hex and decimal IPv4 are rejected, agent ids are ASCII decimal only, nested JSON is not a proof, the 64 KB cap holds (also on decoded gzip), and errors in the domain check leave it unverified without taking down mech list.
One thing left: the deadline doesn't bound a slow drip. _read_proof_body only checks time.monotonic() > deadline after iter_content(PROOF_CHUNK_BYTES) yields, and timeout=5 is per read. Against a local server that answers 200 and then sends one byte every 0.5s, fetch_domain_proof had not returned after 25s, so mech info hangs and mech list waits on that row. A smaller chunk size alone won't cap it; it needs a wall-clock bound, e.g. running the fetch in a worker and taking future.result(timeout=PROOF_FETCH_TIMEOUT). The CHANGELOG line about a deadline over the whole fetch would then hold.
Separately, on valory-xyz/connect#85 I've asked for its mech_report wrapper (indexer record lookup, fallback to the terms, report_note) to move here, the same way mech_terms did. Since this PR is unreleased, it could land in 0.24.0.
Minor: the "Any failure here leaves the domain unverified" comment appears at manager.py:99-101 and manager.py:139, with a near-copy in mech_cmd.py; once is enough.
- The proof body is read with `HTTPResponse.read1`, which returns as soon as any bytes arrive, and the deadline is checked after every read. With `iter_content` a body sent a byte at a time never reached the check. - The caller waits on a capped worker pool for at most PROOF_FETCH_TIMEOUT seconds; a fetch it stopped waiting for ends at its own next read, so it frees its thread and never holds up exit. - urllib3 errors raised by `read1` count as unavailable, like requests errors. `urllib3>=2.6.0,<3` is now a direct dependency: `read1` with decoding arrived in 2.2.0 and its decompressed reads are bounded from 2.6.0. - `ToolManager.mech_report_or_terms` is the report to read before paying: the full report when the manifest and record can be read, otherwise the terms with `report_note`. It replaces the wrapper Pearl Connect carried. `mech_report` stays fail-closed. - One copy of the "any failure leaves the domain unverified" comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks. The drip is bounded now: the proof is read with |
bennyjo
left a comment
There was a problem hiding this comment.
Thanks for turning the three points from my last review around quickly. They're fixed at 2f69239, and so is the slow-drip bound Ojus found.
What blocks this now is size. The PR is +3,665/-391, and the KYA ask for this repo is narrower. I sorted every function, class, test class, and doc change against that ask, which is to resolve the mech's ERC-8004 agent id, check the operator domain against the proof, and report the operator, the owner's other mechs, age, price or payment method, the delivery record labeled settled-only, and a benchmark link per tool, with no karma shown as reliability. Line counts include tests and docs.
Needed for the spec (+1,620/-14)
- Agent id lookup,
query_erc8004_agent_idandquery_service_agent_id(+193) - Registry subgraph URL per chain and its env override (+110)
- Proof match and verdict,
proof_matches,verify_domain, the four proof states, and their notes (+392) - Reading the operator block (+77)
- Report assembly,
mech_report,delivery_record, the settled-only note, andManifestError(+307/-1) - Payment method in place of price (+72)
- Mech record query with the 10 s timeout, for age and delivery counts (+403/-13)
claimed_operator(+37) and the docs for the report (+29)
Needed, but larger than the spec requires (+1,316/-15, about 375 in simpler form)
- Proof fetcher with a thread pool, a
read1loop, and a direct urllib3 pin (+358). Once the list columns are out, a slow drip can only stall the dripping operator's own report, so a plain request with a timeout and a 64 KB cap would do, about 100. - Three sanitizers and four length caps (+264). One hostname check that still rejects IPv4 and one bounded-text check would do, about 75.
- Benchmarks with metric and window validation (+169). The spec asks for a link per tool, about 45.
- Two layers of failure handling plus the claimed name (+159). One
tryinclaimed_operatorwould do, about 65, and it fixes the wrong note a mech with no ERC-8004 identity gets. mech infowith a 50-line custom renderer (+366/-15). Printing the report as JSON would do, about 90.
Not needed for the spec (+729/-362)
- Five new
mech listcolumns with a per-row agent-id lookup and proof fetch (+262/-47) query_mech_creationsand the extra listing fields (+151/-2)- Rewrite of the existing listing tests onto
_mech_row, which is 300 of the 391 deleted lines (+181/-300) mech_report_or_terms, which nothing in this repo calls (+115)- tomte moving from the git tag to PyPI 0.7.3 (+15/-10)
- Version bump to 0.24.0 (+5/-3)
Missing from the spec
- The owner's other mechs isn't in the report.
Mech.owneris the creation sender, so it needs another source, such as the service owner on the registry indexer this PR already reads.
Smaller version
Keeping the first group and the simpler forms of the second comes to about 2,000 added lines, a little over half of this diff.
Proposed split
- This PR keeps the KYA report, meaning
mech_report, the agent-id lookup, the proof check, andmech info, with the simpler forms above. - A separate PR adds the
mech listcolumns and their queries, if we still want them once the report is in use. That's where the threaded fetcher earns its place, because one slow row stalls the whole table. - A chore PR moves the tomte pin.
mech_report_or_termslands with the Connect switch-over that calls it.- The version bump goes wherever we cut the release, in this PR or a chore PR, as we've done both ways before.
- A follow-up PR adds the owner's other mechs.
Smaller PRs are easier to review and to revert, and they keep each change to what we asked for.
The report on one mech before paying is what this change is for; the rest moves to its own PRs. - `mech list` and its tests are back to main. The new columns, the creation-time query and the listing fields go with a separate PR if the report shows they are wanted. - The proof fetch is a plain request again: no redirects, a 64 KB cap and a 5 s timeout per read. A host that trickles its body stalls only the report on its own mech. The worker pool, `read1` and the urllib3 pin are gone. - One bounded-text check and one hostname check; the https check is that text check plus a parse. One text length cap. - `benchmarks` maps each tool to its https link. - One failure layer in `claimed_operator`: no ERC-8004 identity gets its own note, and a failed lookup or check reads as could not be checked. - `mech info` prints the report as JSON. - `mech_report_or_terms` lands with the Connect switch-over, and the tomte bump in its own chore PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0.7.3 carries the fleet baseline for the advisory in the safety scanner's own dependencies, which is what the Security step reports on 0.7.0. The scanned env installs nothing but the scanner, so this repo cannot clear it otherwise. 0.7.3 is on PyPI, so this also resolves the standing TODOs and pins by version rather than by git tag, in the workflow installs, the dev group and the tox template pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, agreed on the split. This PR is now the report only: agent-id lookup, proof check, |
OjusWiZard
left a comment
There was a problem hiding this comment.
Re-checked at fe19e37; the narrowed report looks good and tests pass (1154). One request: please keep mech_report_or_terms here.
It doesn't need a caller in mech-client to belong in it, because it's public API. That's already the pattern: terms_report came in with 93957e6 with no caller in this repo ("so integrations get … from one place"), and ToolManager.get_offchain_url has none on main today.
mech-client exists so a developer can add mech support to their software, and Connect is one consumer of that. The record lookup, the fallback to the terms and report_note are mech logic, so they should come from here. If it stays in Connect, the next consumer that talks to mechs either duplicates it or moves it upstream then. Landing it now also lets valory-xyz/connect#85 drop its own copy when it bumps the pin, rather than waiting for a later pair of PRs.
…aying `ToolManager.mech_report_or_terms` returns `mech_report` when the manifest and the mech's record can be read, and otherwise the terms alone with `report_note` saying why. The record lookup and the fallback to the terms are mech logic, so integrations get them from mech-client rather than each carrying a copy. `mech_report` stays fail-closed for `mech info`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `mech_tools` and `mech_request` call mech-client's `ToolManager.mech_report_or_terms` directly, and connect/mech_report.py is gone. The record lookup, the fallback to the terms and `report_note` are mech logic and now live in one place upstream. - mech-client is pinned to valory-xyz/mech-client#263 at da08f07, where the operator's domain is its identity and the name is `claimed_name`, unchecked, and benchmarks are one link per tool. - The real-client guard checks that shape with a streaming proof fake, and two more guards run the real fallback for an unreadable record and an unreadable manifest. The other tests cover only what connect does: which mech and document it passes, the merge, the order before any allowance, and that a request still goes out after a fallback. - The skill tells the agent to name the operator's domain and whether it is verified, and to treat the name as the operator's own claim. README, CLAUDE.md and the MCP tool descriptions say the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t/mech-operator-report
|
Agreed, it's back in da08f07: |
OjusWiZard
left a comment
There was a problem hiding this comment.
Thanks for keeping mech_report_or_terms here. Re-checked at 1957b08: an unreadable manifest or record falls back to the terms with report_note, otherwise it's the full report, and tests pass (1161).
bennyjo
left a comment
There was a problem hiding this comment.
All nine points from my last review are resolved at 1957b08, and I'm fine with mech_report_or_terms staying as public API since connect#85 already calls it, and with the tomte bump riding along as its own commit, so this is good to merge with the owner's other mechs as the follow-up you named.
0.24.0 is released, so the git source on valory-xyz/mech-client#263 goes and the lock resolves the published package. The release carries no library change since the pinned commit, only its CI tooling bump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What
Phase 2 of KYA, the reader side. A buyer's agent can look up who runs a mech and how it has behaved before it pays.
ToolManager.mech_report(mech_address, metadata)extendsterms_report(every existing key unchanged) with:operator: the domain the manifest claims, asdomainandurl, withdomain_verifiedtrue only whenhttps://<domain>/.well-known/agent-registration.jsonlists the chain's ERC-8004 identity registry with the mech's agent id, andverification_notewhen not. The name isclaimed_namewith aname_notethat it is unchecked. The agent id is read from the service registry indexer and is not the Olas service id except by coincidence (optimism 195 is agent 516, polygon 21 is 25, base 112 is 114).benchmarks: each tool's https link, as the operator's claim, withbenchmark_note. No figure.delivery:received_requests,self_deliveredanddelivered_by_others(the subgraph'sreceivedRequests,selfDeliveredFromReceived,deliveredByOthersFromReceived),since(ISO 8601 UTC) andage_daysfrom theCreateMechblock, with a note that the counts cover on-chain and settled off-chain requests, not split by path.payment: the method, with no price. The contract quotes the charge at request time;maxDeliveryRateis only a ceiling.ToolManager.mech_report_or_terms(mech_address, metadata)is the report to read before paying:mech_reportwhen the manifest and record can be read, otherwise the terms withreport_note. Connect can drop its own copy of this fallback when it bumps the pin;mech_reportitself stays fail-closed.mechx mech info --chain-config <chain> --address <mech>prints that report as JSON.registry_subgraph_urlper chain inmechs.json, withMECHX_REGISTRY_SUBGRAPH_URLas override.Why these shapes
ManifestError, no partial report. Any failure in the agent-id lookup or the domain check leaves the domain unverified with a note, and is tried again next call rather than cached as a no.Mech.owneris the creation-tx sender and is never updated on transfer, so it is not reported.Not in this PR
mech listcolumns, in their own PR if wanted.Known data gaps
operatororbenchmarkyet.www.valory.xyz/.well-known/agent-registration.jsonreturns 404, and its generator lists service ids rather than ERC-8004 agent ids, so Valory mechs read as unverified until it is regenerated and deployed.🤖 Generated with Claude Code