archive: do not leak URL credentials in user visible messages - #706
archive: do not leak URL credentials in user visible messages#706mahaase wants to merge 3 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #706 +/- ##
==========================================
+ Coverage 89.23% 89.25% +0.02%
==========================================
Files 50 50
Lines 16450 16458 +8
==========================================
+ Hits 14679 14690 +11
+ Misses 1771 1768 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
jkloetzke
left a comment
There was a problem hiding this comment.
Please make this two commits for the fixes. One that prunes the credentials from the logs and one that splits the username/password and host name differently. These are two independent changes that should get their own commit.
The documentation commit can be dropped. Users of Bob probably don't care how HTTP basic auth works.
| return url | ||
| # rsplit: urlparse() treats the *last* '@' as the delimiter, so a password | ||
| # with an unencoded '@' must not confuse us into cutting at the first one. | ||
| return url._replace(netloc=url.netloc.rsplit('@', 1)[1]) |
There was a problem hiding this comment.
AFAICT, this is an undocumented, private method. Just do it like _getURL() previously.
The HTTP basic authentication credentials are part of the archive URL. All
three places that turn that URL back into a string for display used the raw
netloc, which still carries the "user:password@" part:
* getArchiveName() feeds _namedErrorString(), so the credentials were
printed on *every* error message -- including the perfectly ordinary
"artifact not found" that occurs for each package on a cache miss. No
verbosity flag needed.
* _remoteName() is the "details" of the DOWNLOAD/UPLOAD/MAP-SRC/CACHE-BID/
CACHE-FPR/MAP-FPRNT status lines, shown with -v.
* getArchiveUri() is printed by "bob archive".
These messages routinely end up in build logs and CI consoles.
WebDav._getURL() already stripped the credentials before putting the URL on
the wire, so this only ever affected the display strings. Factor that logic
out into getNetLoc() and use it in the three spots above as well.
The optional 'name' archive setting was no workaround: _remoteName() and
getArchiveUri() do not consult it.
The user info is separated from the host by the *last* '@' of the network
location. That is what urlparse() does (it uses rpartition('@')) and what
RFC 3986 mandates, because '@' is allowed unencoded in the user info.
getNetLoc() cut at the first one instead. For a password containing an
unencoded '@' that produced a bogus host: the request URL got the remainder
of the password as its authority, so the request failed -- and the leftover
password fragment was shown in the resulting message.
Use rsplit('@', 1) so that the host matches the one urlparse() reports.
testNoCredentialsInMessages checks the three user visible strings derived from the archive URL: the password and the '@' delimiter must be gone while the host must survive. Reverting archive.py alone makes all three subtests fail. TestGetNetLoc covers the helper itself: URL without credentials, plain removal, a password containing an unencoded '@' and an IPv6 literal host.
7bc796d to
9ddcefe
Compare
Problem
The HTTP basic authentication credentials of the
httparchive backend arepart of the URL, as documented. All three places that turn that URL back into
a string for display use the raw
netloc, which still carries theuser:password@part:getArchiveName()(archive.py:877)_namedErrorString()(archive.py:401)_remoteName()(archive.py:902)detailsof theDOWNLOAD/UPLOAD/MAP-SRC/CACHE-BID/CACHE-FPR/MAP-FPRNTstatus lines-vgetArchiveUri()(archive.py:966)bob archive(cmds/archive.py:32)The first one is the bad one.
_namedErrorString()is used for every errormessage, including the perfectly ordinary "artifact not found" that happens for
each package on a cache miss. So a plain
bob buildon a cold cache prints thepassword once per package:
These messages routinely end up in build logs, CI consoles and pasted-into-an-issue
snippets.
The optional
namearchive setting is not a workaround:_remoteName()andgetArchiveUri()never consult it.Note this is a display-only issue.
WebDav._getURL()already strips thecredentials before the URL goes on the wire; they are sent in the
Authorizationheader (webdav.py:47-55).Fix
Factor the stripping that
WebDav._getURL()already does into astripUserInfo()helper and use it in the three places above as well.It takes and returns a parsed URL and preserves the result type, so both the
urlsplit()result used inwebdav.pyand theurlparse()result used inarchive.pywork.Drive-by: cut at the last
@, not the firstThe existing implementation used
netloc.split('@')[1].urlparse()delimitsthe user info at the last
@, so the two disagree when the passwordcontains an unencoded
@:Percent encoding is documented as required, so this was latent rather than a
live bug, but there is no reason to get it wrong.
Tests
TestHttpBasicAuthArchive.testNoCredentialsInMessages— asserts all threemethods drop the password and the
@, and still contain the host.TestStripUserInfo— no credentials (identity), removal,@in thepassword, IPv6 literal,
urlsplit()type preservation.Reverting
archive.pyalone makes the first test fail on all three subtests;test_archive+test_webdavare green with the fix (71 tests).