Environment
- stackql v0.10.582 (c926a33), Windows and Linux, local file registry
- Found building the rebuilt
snowflake provider; reproduced against both a live API and a local mock server
Summary
Link-header pagination (RFC 5988 Link: <url>; rel="next") is broken under both possible config shapes:
- The natural config for the scheme - a
responseToken only - deadlocks the binary: every list SELECT hangs indefinitely after the first response, whether or not a Link header is present.
- Adding a
requestToken (semantically meaningless for this scheme - there is no token, only a URL) unblocks the loop, but the URL extracted by defaultLinkHeaderTransformer is then injected verbatim as the request token query parameter value instead of being used as the next request URL.
Repro 1: responseToken-only config hangs
Service-level config (root x-stackQL-config in the service doc):
x-stackQL-config:
pagination:
responseToken:
key: link
location: header
Any list SELECT against the service hangs forever (no timeout, no error). Reproduced:
- against a live API that returns 200 + JSON with NO Link header
- against a mock that returns a correctly formatted header (
<url>; rel="next", single space after the semicolon per the linksNextRegex in internal/anysdk/pagination.go)
- with and without a response transform on the method
Expected: either traverse the chain (header present) / return the single page (header absent), or fail fast at config validation if a responseToken-only pagination block is unsupported. A hang should never be the outcome of a config that passes the published stackql-config.schema.json.
Repro 2: with a requestToken, the next URL is sent as a query param value
x-stackQL-config:
pagination:
requestToken:
key: fromName
location: query
responseToken:
key: link
location: header
defaultLinkHeaderTransformer correctly extracts the next-page URL from rel="next", but the follow-up request goes out as:
GET /api/v2/databases?fromName=http%3A%2F%2Flocalhost%3A63218%2Fapi%2Fv2%2Fdatabases%3FfromName%3DDB3
i.e. the entire URL is injected as the fromName value. Expected: the extracted URL replaces the request URL for the next page (the whole point of link-header pagination). Wire capture is from a mock-server integration suite that logs every request; happy to share the harness.
Notes
- Token-style pagination is unaffected: the
k8s provider's continue token config works correctly on the same binary, so this is specific to the link-header path's consumer.
- docs/provider_spec.md documents
algorithm: link_header_next for this pattern, but that identifier is not recognized anywhere in code (it merely diverts from the defaultLinkHeaderTransformer fast path into the generic header transformer, which behaves identically) - worth reconciling docs and code in the same fix.
- Practical impact for
snowflake today is nil (the vendor specs declare Link headers but the live control plane does not emit them), so that provider ships with no pagination config. Any provider whose API genuinely paginates via Link headers hits this immediately.
Environment
snowflakeprovider; reproduced against both a live API and a local mock serverSummary
Link-header pagination (RFC 5988
Link: <url>; rel="next") is broken under both possible config shapes:responseTokenonly - deadlocks the binary: every list SELECT hangs indefinitely after the first response, whether or not aLinkheader is present.requestToken(semantically meaningless for this scheme - there is no token, only a URL) unblocks the loop, but the URL extracted bydefaultLinkHeaderTransformeris then injected verbatim as the request token query parameter value instead of being used as the next request URL.Repro 1: responseToken-only config hangs
Service-level config (root
x-stackQL-configin the service doc):Any list SELECT against the service hangs forever (no timeout, no error). Reproduced:
<url>; rel="next", single space after the semicolon per thelinksNextRegexin internal/anysdk/pagination.go)Expected: either traverse the chain (header present) / return the single page (header absent), or fail fast at config validation if a responseToken-only pagination block is unsupported. A hang should never be the outcome of a config that passes the published
stackql-config.schema.json.Repro 2: with a requestToken, the next URL is sent as a query param value
defaultLinkHeaderTransformercorrectly extracts the next-page URL fromrel="next", but the follow-up request goes out as:i.e. the entire URL is injected as the
fromNamevalue. Expected: the extracted URL replaces the request URL for the next page (the whole point of link-header pagination). Wire capture is from a mock-server integration suite that logs every request; happy to share the harness.Notes
k8sprovider'scontinuetoken config works correctly on the same binary, so this is specific to the link-header path's consumer.algorithm: link_header_nextfor this pattern, but that identifier is not recognized anywhere in code (it merely diverts from thedefaultLinkHeaderTransformerfast path into the generic header transformer, which behaves identically) - worth reconciling docs and code in the same fix.snowflaketoday is nil (the vendor specs declareLinkheaders but the live control plane does not emit them), so that provider ships with no pagination config. Any provider whose API genuinely paginates viaLinkheaders hits this immediately.