fix(hlx6): Poll api.aem.live for changes

[bosschaert]

Description

This PR only contain changes related to the api.aem.live backend, nothing changes for the admin.da.live backend as this functionality already works there.

If changes have been made to a document outside of DA, ensure to pick them up. This is done by polling for such changes every 5 seconds.

Note, this PR should only be merged after https://github.com/adobe/helix-api-service/pull/469 has been merged and deployed.

Related Issue

Fixes #177
Fixes #https://github.com/adobe/helix-api-service/issues/453

How Has This Been Tested?

• Locally with the real api.aem.live
• New unit tests

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• I have signed the Adobe Open Source CLA.
• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[kptdobe]

Review

Reviewed at 17aed61. Lint is clean and npm test passes (253 tests, two runs). The polling lifecycle (single in-flight HEAD, obsolete-response guards, teardown) is carefully handled and well tested. There is one high-severity issue I'd fix before merging; the rest are follow-ups or nits.

High

1. An external change can still be overwritten after the invalidation reconnect.
After an invalidation the client keeps its local Y.Doc and reconnects (y-websocket default), sending in SyncStep2 any updates the server hasn't seen: edits typed during the reconnect, or updates in flight when the socket closed. If they arrive within the 1 s restore window, clientHasUpdated (shareddoc.js#L883) skips the reload, so the doc stays on the old content plus those edits. Meanwhile current and ydoc.etag come from the fresh GET (the external version), and lastsync is re-anchored to it (#L928). The next debounced save POSTs the stale document with If-Match: <new ETag>, which passes, and the external change is lost. So "reconnecting sessions reload the source document" (README.md#L46) doesn't hold in this case. The only reconnect test (shareddoc.test.js#L3964) covers identical content with no client update.

Possible fixes: close the sockets with a dedicated close code so da-live drops its local Y.Doc before reconnecting. As a server-side safety net, persist an invalidation marker so the next session doesn't save while it still holds pre-invalidation state (e.g. when the clientHasUpdated shortcut skipped the reload). Suggested test: invalidate, reconnect, send a client update within 1 s, and assert no POST.

Medium

2. Polling keeps the Durable Object awake. The 5 s setInterval (#L997) prevents the DO from hibernating while a Helix doc is open, so each open document is billed for duration continuously and sends ~17k HEAD requests/day to api.aem.live. Has a push from helix-api-service to /api/v1/syncadmin (as da-admin does) been considered? Otherwise, back off while the doc is idle.

3. Polling auth doesn't recover. The HEAD always uses the first connection's token (#L655), and 401/403 (#L673) are logged and retried with the same token. Once that token expires, change detection stops for the rest of that connection's lifetime, logging a warning every 5 s. Fall back to the other connections' tokens, or close the stale connection (e.g. with the existing 4401 code) so the client reconnects with a fresh token.

4. A refused save is acked as successful. The new "no verified ETag" refusal (#L558) is caught inside persistence.update, so the flush ack (#L1196) reports ok although nothing was saved. Swallowing save errors is pre-existing, but this refusal can last a whole session, so a client relying on the flush ack is told the content is saved when it isn't.

Low

5. IS_HELIX is a local-testing switch in production code paths. Set to true/local, it makes isHelixDoc true for every doc, which also grants all collaborators read,write (edge.js#L432) regardless of what da-admin reports. Consider honoring it only in the dev environment. Copying it onto every ydoc (#L1108) so the ydoc can double as env is also fragile.

6. The da-admin ETag tracking (#L614-L619) says it feeds the HEAD check, but polling is only scheduled for Helix docs. The same applies to the da-admin auth branch in checkEtag (#L655).

7. Nits

• The log line changed from "Invalidate from Admin received" to "Invalidate document cache" (#L395); check whether any log queries or alerts match the old text.
• 30f386b and fca1222 add and then revert the same package.json line; worth squashing.
• Several tests rely on internal fields (etagUnverified, etagCheckAbortController, pendingPuts) rather than observable behavior, which makes them brittle.

Dependency on adobe/helix-api-service#469

The POST response must return the same ETag that a later HEAD returns (ignoring W/), and If-Match must accept the tag with W/ stripped. Otherwise every save causes a poll mismatch or a 412, and the session reloads repeatedly.

---

Reviewed with GitHub Copilot