feat(xds): per-stream call-credentials hook for the xDS transport#2705
Conversation
Per grpc#2444, credential-gated xDS control planes (e.g. GCP Traffic Director / `google_default`) require two things the transport did not provide: per-stream call credentials on the ADS stream, and a secure channel to carry them. Their bootstrap `server_uri` is often scheme-less (e.g. `trafficdirector.googleapis.com:443`), which parses with no scheme — so `Endpoint` has no signal to negotiate TLS, the ADS connection never establishes a secure session, and routing times out with "route config not yet available". xds-client: - Add a minimal `CallCredentials` trait mirroring grpc-go's `PerRPCCredentials`. The tonic transport attaches it on each (re)connect, only over a secure channel. - Construction is builder-based: `TonicTransportBuilder::with_call_credentials` and `with_channel(Channel, secure)` for a pre-built channel. - On the secure path, prepend `https://` to `server_uri` in `TonicTransportBuilder::build` when it's missing the scheme. tonic-xds: - Promote `google_default` to a first-class `ChannelCredentialType::GoogleDefault` (gRFC A27). - Add `XdsChannelConfig::with_call_credentials(...)`, so that applications can give a token source without relying on `grpc` or `grpc-google` crates at this time. examples: - Add the `channel_with_google_default` tonic-xds example (reusing the testutil greeter), demonstrating two ADC token sources. BREAKING CHANGE: * Relocate `TonicTransport::from_channel` to `TonicTransportBuilder::with_channel`, and * Adds a variant to the `Error` enum
|
|
|
@YutaoMa PTAL! |
| #[error("stream error: {0}")] | ||
| Stream(#[from] tonic::Status), | ||
|
|
||
| /// Call credentials failed, or require a secure transport. |
There was a problem hiding this comment.
Let's remove the feature gating on this error variant, non-additive features in Rust can be a footgun: implicit feature enablement with other dependencies can break compilation.
While we are at it, let's also add #[non_exhaustive] to the Error, it's a miss on my end to not mark it so previously. That'll help with future extensions like this.
There was a problem hiding this comment.
Done. Dropped the #[cfg(feature = "transport-tonic")] on CallCredentials and added #[non_exhaustive] to Error. See 447e849.
| /// Generates the authentication metadata for a specific call. | ||
| async fn get_request_metadata( | ||
| &self, | ||
| metadata: &mut tonic::metadata::MetadataMap, |
There was a problem hiding this comment.
Is there any specific reason to prefer out-param style API? I'd recommend keeping this simple and idiomatic by returning an owned map.
There was a problem hiding this comment.
Ah saw the grpc link comment above, seems like it's to keep the style consistent with grpc. In that case yeah it's good to keep it as is.
There was a problem hiding this comment.
Yeah, thanks for confirmation!
| /// | ||
| /// Attached on each (re)connect, only when the channel is secure. | ||
| #[tonic::async_trait] | ||
| pub trait CallCredentials: Send + Sync + std::fmt::Debug + 'static { |
There was a problem hiding this comment.
Could you re-export this type in tonic-xds so user of tonic-xds doesn't need to explicitly depend on xds-client to use this feature? (xds-client is intended to stay as a low-level ADS transport library so gRPC service developers don't have to depend on it explicitly). Also I suggest naming it TonicCallCredentials here: once we support grpc transport in xds-client, similar API will need to be added for GrpcCallCredentials, keeping the name distinguishable helps with future extensions.
There was a problem hiding this comment.
Done. Renamed and re-exported. See 447e849.
bed4264 to
47030ae
Compare
|
@YutaoMa PTAL! |
- Added #[non_exhaustive] to pub enum Error, and remove #[cfg(feature = "transport-tonic")] from `CallCredentials` error entry. - Rename `CallCredentials` as `TonicCallCredentials`, and let `tonic-xds` re-export it to public.
47030ae to
447e849
Compare
| } | ||
|
|
||
| #[test] | ||
| fn selected_credential_unknown_type_skipped() { |
There was a problem hiding this comment.
Looks like we can combine this test with "selected_credential_none_supported()"
…done by `selected_credential_none_supported`.
gu0keno0
left a comment
There was a problem hiding this comment.
PTAL. Thanks for the contribution. I'd like to simplify the states of "secure", "channel" and "per call creds" and avoid invalid states from builder directly.
| /// | ||
| /// let builder = TonicTransportBuilder::new().with_channel(channel, true); | ||
| /// ``` | ||
| pub fn with_channel(mut self, channel: Channel, secure: bool) -> Self { |
There was a problem hiding this comment.
Is this only needed for test? Maybe gate it by #[cfg(test)]
The reason is that "secure" is not tied to "channel". I wish Tonic had channel.is_secure() -> bool, yet since it doesn't have it, better not to expose ambituity.
There was a problem hiding this comment.
It was originally TonicTransport::from_channel function (link), and I brought it here to make it work with with_call_credentials; thinking again upon all your comments, it'd be better to move back to TonicTransport::from_channel to to avoid ambiguous cases rather than gating by #[cfg(test)]. When call-credential is required, the users will be able to make the channel using TonicChannelBuilder instead.
| if let Some(creds) = &self.call_creds | ||
| && creds.requires_secure_transport() | ||
| && !self.secure |
There was a problem hiding this comment.
It's probably better to check this during TonicTransport build time. If secure == false, always set creds to None.
| #[derive(Clone, Debug)] | ||
| pub struct TonicTransport { | ||
| channel: Channel, | ||
| secure: bool, |
There was a problem hiding this comment.
I think we can drop this if we do the check during TonicTransport build time.
…onicTransport::from_channel` to avoid invalid states.
Reflected to f44998e. As said reply, |
|
|
||
| /// Fetches ADC tokens directly from `google-cloud-auth`. | ||
| #[derive(Debug)] | ||
| struct AdcTonicCallCredentials { |
There was a problem hiding this comment.
I think we might want to support google credentials out-of-box in tonic-xds' xds bootstrap handling in the future, yet that's out of scope for this PR, so no need to do it here.
Motivation
Per #2702, credential-gated xDS control planes (e.g. GCP Traffic Director /
google_default) require two things the transport did not provide: per-stream call credentials on the ADS stream, and a secure channel to carry them. Their bootstrapserver_uriis often scheme-less (e.g.trafficdirector.googleapis.com:443), which parses with no scheme — soEndpointhas no signal to negotiate TLS, the ADS connection never establishes a secure session, and routing times out with "route config not yet available".Solution
xds-client:
TonicCallCredentialstrait mirroring grpc-go'sPerRPCCredentials. The tonic transport attaches it on each (re)connect, only over a secure channel.TonicTransportBuilder::with_call_credentials, where channel security is derived fromwith_tls_configand validated at build time to avoid sending credentials to insecure channel.https://toserver_uriinTonicTransportBuilder::buildwhen it's missing the scheme.tonic-xds:
google_defaultto a first-classChannelCredentialType::GoogleDefault(gRFC A27).XdsChannelConfig::with_call_credentials(...), so that applications can give a token source without relying ongrpcorgrpc-googlecrates at this time.examples:
channel_with_google_defaulttonic-xds example (reusing the testutil greeter), demonstrating two ADC token sources.BREAKING CHANGE: