Repository navigation
Fix WebKit export approval: verify landings against the WebKit repository - #273
DanielRyanSmith wants to merge 2 commits into
Conversation
bugs.webkit.org now reports the 'who' field of bug history as a bare login name (e.g. 'ews-feeder') rather than a full email address (e.g. 'ews-feeder@webkit.org'). This caused WebKit-exported PRs to no longer be detected as reviewed, and therefore never approved. Accept either form when checking whether a bug was resolved FIXED by ews-feeder or commit-queue.
bugs.webkit.org now reports only the local part of email addresses (e.g. 'ews-feeder') to logged-out API callers. The local part is not a unique identity: anyone can register e.g. 'ews-feeder@example.com', resolve their own bug as FIXED, and so have a PR titled 'WebKit export of <bug>' approved without review. Treat the 'who' check as a pre-filter only, and confirm the landing by finding the landing bot's 'Committed <ref>' comment and checking with commits.webkit.org that the referenced commit is on main and its message references the bug.
|
The bot is not approving WebKit PRs until we deploy a new version with a change like this |
We could presumably also just fix how we authenticate, and get the actual emails? This seems to have regressed with WebKit/WebKit@189ce80.
There are no follow-ups linked. This does seem very LLMish, and I will remind you of the Policy for LLM (AI) Use. |
gsnedders
left a comment
There was a problem hiding this comment.
I'm gonna have a very quick go at fixing this, based on my earlier change in the larger PR, inc. fixing auth.
Live check: ran webkit.flags() against real bugs.webkit.org and commits.webkit.org for all 49 open "WebKit export of" PRs in web-platform-tests/wpt.
We should add/update fixtures in test/fixtures to verify this, versus just updating our unit tests?
| logger = require('../logger'); | ||
|
|
||
| // Accounts that resolve a bug as FIXED when a reviewed patch lands in WebKit. | ||
| const WEBKIT_LANDING_BOTS = ["ews-feeder", "commit-queue"]; |
There was a problem hiding this comment.
Let's actually store these by their actual usernames, for when we actually fix authentication:
| const WEBKIT_LANDING_BOTS = ["ews-feeder", "commit-queue"]; | |
| const WEBKIT_LANDING_BOTS = ["ews-feeder@webkit.org", "commit-queue@webkit.org"]; |
| // Matches the comment left by WebKit's landing bots, e.g. | ||
| // "Committed 324168@main (004e01c34435): <https://commits.webkit.org/324168@main>" | ||
| // "Committed r266816: <https://trac.webkit.org/changeset/266816>" | ||
| const LANDING_COMMENT_PATTERN = /Committed (\d+@main|r\d+)(?: \(([0-9a-f]{7,40})\))?/g; |
There was a problem hiding this comment.
There's an interesting inconsistency here, insofar as we allow any branch for SVN but only main for git.
The history of comments is something like:
https://bugs.webkit.org/show_bug.cgi?id=209237, commit-queue@webkit.org:
Patch
Clearing flags on attachment: 393868
Committed r258643: https://trac.webkit.org/changeset/258643
https://bugs.webkit.org/show_bug.cgi?id=208861, ews-feeder@webkit.org:
Committed r258210: https://trac.webkit.org/changeset/258210
All reviewed patches have been landed. Closing bug and clearing flags on attachment 393151 [details].
https://bugs.webkit.org/show_bug.cgi?id=223117, ews-feeder@webkit.org:
Committed r274424: https://commits.webkit.org/r274424
All reviewed patches have been landed. Closing bug and clearing flags on attachment 423037 [details].
Eight bugs, including https://bugs.webkit.org/show_bug.cgi?id=230351, ews-feeder@webkit.org:
Committed r286611 (244935@trunk): https://commits.webkit.org/244935@trunk
All reviewed patches have been landed. Closing bug and clearing flags on attachment 446169 [details].
And https://bugs.webkit.org/show_bug.cgi?id=247043 contains two examples, ews-feeder@webkit.org:
Committed 256627@main (8f70e6bf626c): https://commits.webkit.org/256627@main
Reviewed commits have been landed. Closing PR #6300 and removing active labels.
Plus:
Committed 252432.920@safari-7614-branch (768d89b728fc): https://commits.webkit.org/252432.920@safari-7614-branch
Reviewed commits have been landed. Closing PR #245 and removing active labels.
Though we can have things only on a branch on their own bug.
I might be tempted to use a regex more like:
| // Matches the comment left by WebKit's landing bots, e.g. | |
| // "Committed 324168@main (004e01c34435): <https://commits.webkit.org/324168@main>" | |
| // "Committed r266816: <https://trac.webkit.org/changeset/266816>" | |
| const LANDING_COMMENT_PATTERN = /Committed (\d+@main|r\d+)(?: \(([0-9a-f]{7,40})\))?/g; | |
| // Matches the comment left by WebKit's landing bots, e.g. | |
| // "Committed 324168@main (004e01c34435): <https://commits.webkit.org/324168@main>" | |
| // "Committed 252432.920@safari-7614-branch (768d89b728fc): <https://commits.webkit.org/252432.920@safari-7614-branch>" | |
| // "Committed r286611 (244935@trunk): <https://commits.webkit.org/244935@trunk>" | |
| // "Committed r274424: <https://commits.webkit.org/r274424>" | |
| // "Committed r266816: <https://trac.webkit.org/changeset/266816>" | |
| const LANDING_COMMENT_PATTERN = /^Committed ([0-9]+(?:\.[0-9]+)*@[^\000-\040 ~^:?*[\\]+|r[0-9]+)[ :]/gm; |
| var bugURLPattern = new RegExp( | ||
| `(?:https?://bugs\\.webkit\\.org/show_bug\\.cgi\\?id=|https?://webkit\\.org/b/)${issue}(?!\\d)`); |
There was a problem hiding this comment.
This doesn't change
wpt-pr-bot/lib/metadata/index.js
Line 114 in 784d204
https?://webkit\\.org/b/ alternation is pointless.
We also only have a single bug matching that, which suggests we might not need it: https://github.com/web-platform-tests/wpt/pulls?q=is%3Apr+webkit.org%2Fb%2F+in%3Atitle
| for (var match of comment.text.matchAll(LANDING_COMMENT_PATTERN)) { | ||
| var ref = match[1], abbreviatedHash = match[2]; | ||
| try { | ||
| var commit = await commitsWebkit.get(ref); |
There was a problem hiding this comment.
If we want to ensure the abbreviatedHash points at something suitable, and not just trust the canonical identifier, then we should pass it in here, rather than the canonical identifier?
| throw new Error(`Invalid WebKit commit reference: ${ref}`); | ||
| } | ||
| const response = await exports.fetch(`${BASE_URL}/${ref}/json`, { | ||
| headers: { "User-Agent": "wpt-pr-bot" }, |
See #274 |
|
I'm going to go ahead and drop this PR in favor of #274 . Thanks for taking a look at this! |
wpt-pr-bot stopped approving WebKit export PRs. After investigating, it looks like the format of the response from the bugs.webkit.org API changed. This change fixes the review check to allow the PRs to continue to be auto-approved.
More investigation details (generated):
Details
## Summarywpt-pr-bot stopped approving WebKit-exported PRs on
2026-10-16. There was no deploy; the serving version dates
from 2026-09-08.
WebKit now lands patches without
review+flags, so the bot treats an exportas reviewed when the WebKit bug is resolved
FIXEDbyews-feederorcommit-queue. The check comparedhistory_item.whoagainst full emailaddresses (
ews-feeder@webkit.org). bugs.webkit.org now applies Bugzilla'semail filtering for logged-out API callers, so
whocontains only the part ofthe address before the
@(ews-feeder). The bot calls the API withoutlogging in (see follow-ups). Because nothing matched,
webkit.flags.reviewedstayed
falseand no exports were approved.The fix
The bot now confirms the landing with the WebKit repository itself:
whocheck acceptsews-feeder/commit-queue, with or without@webkit.org(other domains and different casing are rejected). It is nowonly a cheap pre-filter.
webkit.landed()reads the bug's comments and finds the landing bot'scomment in either format:
Committed 324168@main (004e01c34435): <https://commits.webkit.org/324168@main>Committed r266816: <https://trac.webkit.org/changeset/266816>(legacySVN format)
https://commits.webkit.org/<ref>/json(new
lib/commits-webkit.js) and requires:branch === "main"comment has one
(
https://bugs.webkit.org/show_bug.cgi?id=Norhttps://webkit.org/b/N,not followed by another digit)
Only WebKit committers can land on
main, so this can't be spoofed by anarbitrary Bugzilla user. The ordering behaviour of the existing review+ /
commit-queue+ flag logic is unchanged.
Commit references are parsed from comment text, so
commits-webkit.get()accepts only
^\d+@main$or^r\d+$before building a URL. Errors fromcommits.webkit.org are logged and treated as "not landed"; the poller retries
every 15 minutes.
Why not just accept
ews-feeder?The part of an email before the
@is not unique. Anyone can register aBugzilla account such as
ews-feeder@example.com, and it would also show up asews-feeder. Accepting the bare name would let anyone get a PR approvedwithout review:
ews-feeder@<anything>.see_also.Testing
test/webkit.js(26 tests):missing
whoother bugs and bug-number prefixes, comments with no landing reference
test/commits-webkit.js(10 tests): fetching, 404 →null, errors, andrejection of invalid or path-traversal references.
test/get-metadata.js: the existing WebKit 201401 test now stubs the commentand commit lookups. Its expected output is unchanged.
commits-webkit.js100%,webkit.js91%. Theremaining lines are covered by
get-metadata.webkit.flags()against real bugs.webkit.org andcommits.webkit.org for all 49 open "WebKit export of" PRs in
web-platform-tests/wpt.
reviewed: true.reviewed: falseresult is a WebKit bug that is still open, sothere are no false negatives.
Notes
treated as reviewed before this change (the bot trusted any landing-bot
FIXED), so this PR doesn't change that.
commits.webkit.org request, for each open export whose bug was resolved by a
landing bot.
After deploy
The poller runs every 15 minutes and approval is one-way, so stuck exports
(including #63879 and #63886) should be approved on the next poll.