Skip to content

Reset projection without downtime #253

Closed

Description

@jlabedo

Description
During the projection reset you see stale data on your queries. As you accumulate more and more events in your stream, the reset takes longer and has an impact on the end user.
The feature would be to have a reset without downtime (or very low).

Example
while resetting, a new table could be built while the reset is ongoing, the old table would continue to be updated by events. When the reset is complete, delete old table and rename the new one.

Activity

  1. dgafka commented on Oct 26, 2023

    @dgafka
    Member

    New table would mean queries needs to be changed and projection initialization would need to be aware that it should create for the rebuild.
    The current rebuild method is not ideal, is just solution provided out of the box by Prooph abstraction.
    I personally think there should be no rebuild method at all, and only register/unregister projection in given version.

    What I mean by that is enabling blue/green projection deployment.
    To make it work, we would need to allow registering given projection in specific version.
    Version would be passed to projection initialize and projection handlers via metadata. This way they would know, how to produce tables and where to project.

    After projection is built in new version, developer can verify if all is fine and if so do the switch to read from new tables. Old projection can then be unregistered.
    The only caveat is, that we would need to keep information of registered projection versions so we know that we call the projection for each of the available version. This can be done in the code tho.

    #[Projection("balance")]
    #[ProjectionVersions([1, 2])]
    class BalanceProjection
  2. dgafka commented on Oct 26, 2023

    @dgafka
    Member

    And I think that's more an discussion ticket than issue, mate :)

  3. locked and limited conversation to collaborators on Oct 26, 2023
  4. converted this issue into a discussion #254 on Oct 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions