Replies: 1 comment 2 replies
|
So the only unclear thing from the proposal is what the intended API is for the upload-only mode. Since maintaining backwards compatibility might mean you have to supply the PowerSync service endpoint. Or do we envision making it an optional parameter and doing some type/override guarding? I think I would prefer something in that line instead of adding a dedicated function to API to handle that. |
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TL;DR: This proposes to split the
PowerSyncBackendConnectorinterface into two superinterfaces: One providing credentials (for streaming downloads) and one responsible for uploading changes. We then allow connecting in upload or download-only mode.Current status:
Implementation progress:
BackendConnectortrait powersync-native#46.Current ‘backend connector’ architecture
The specifics vary slightly based on the specific SDK, but generally when wiring up the PowerSync Client SDK, you have to define a “backend connector” class with 2 methods:
fetchCredentials()which needs to return an object containing:token: JWT for authentication with PowerSync Service. Developers can either reuse the existing JWT from their authentication provider, or mint a new JWT specific for PowerSync, by calling their own backend APIendpoint: PowerSync Service instance URLuploadData()which gets entries from the upload queue, uploads them to the developer/customer’s backend, and marks those transactions as complete so that they are removed from the upload queueProblem statements
There are several paper cuts involved with the tight coupling of everything in the ‘backend connector’.
fetchCredentials()as part of the “backend connector” does not make sense semantically.fetchCredentials()anduploadData()are both part of a “backend connector” makes the concepts more complicated to explain in our documentation (and makes structuring the documentation more complicated)connect()on thePowerSyncDatabasein order to pass a “backend connector” with anuploadData()implementation. You cannot upload client-side mutations to your backend without connecting to a PowerSync Service instance.fetchCredentials()(PowerSync Service instance endpoint URL and custom backend URL) can cause confusion.uploadData(), it would be better to standardize terminology to refer to the entries in the upload queue as “mutations” everywhere in our code/APIs and documentation. “Mutations” is the most clear description we can use.uploadData()doesn’t align with any of those.Goals
uploadData()— standardize on “mutations”fetchCredentials()— in many/most cases it won’t require fetching from your backend, if you reuse an existing JWTProposal
Note: The internal document lists additional proposals, this is a summary based on discussions in comments.
While preserving backwards compatibility, eliminate the concept of a 'backend connector' in favor of these interfaces (or, where appropriate for an SDK, callback functions):
Depending on the SDK, we then add overloads with separate parameters on
connect(), and add a download-only mode:Calling
connect(backendConnector)continues to work.The API changes depend on the target language:
connect, but we can replace thePowerSyncBackendConnectorparameter with a supertype.Additional considerations
Local mutations that are never uploaded
What do we do with a database that has pending
ps_cruditems and is then connected in download-only mode? It wouldn't be able to apply checkpoints, and we only surface that as log items which are easy to overlook.To make this error state easier to observe, maybe we should also add these errors to the sync status.
Upgrading a partial connection to a full connection
Consider a connect call only providing credentials for downloads (no mutation upload callback) followed by another call only providing a mutation uploader (and no separate credentials).
In this state, should the database be fully connected (using the initial set of credentials for downloads and the second call for uploads), or should only the last connect call have any effect (meaning that the database is upload-only?).
Making
connect()calls "add up" feels nicer, but is not consistently possible when e.g. different options like retry delays are passed to the calls. So only letting the last call have an effect is more consistent.Requesting checkpoints in download-only mode
In upload-only mode, the database has no credentials to connect to the PowerSync service and thus can't request write checkpoints / checkpoint requests. We'd upload everything from
ps_crudand then keep the checkpoint gate ati64::max.In this state, if the database is then connected in download-only mode, should it request a checkpoint? Conceptually, requesting checkpoints is part of the upload process but it probably makes sense to run that once in a download-only connection to make sure we can apply checkpoints.
All reactions