FLOGO-19484: add support/sqltx for ambient DB transaction propagation - #300
Open
awakchau-tibco wants to merge 1 commit into
Open
awakchau-tibco wants to merge 1 commit into
awakchau-tibco wants to merge 1 commit into
Conversation
awakchau-tibco
added a commit
to project-flogo/flow
that referenced
this pull request
Sep 3, 2026
…of core and flow The module required bare `core v1.6.22` and `flow v1.6.28`. Neither tag exists yet, so the module could not be resolved, built or tested by anyone -- and `go.sum` could not be generated at all, because there was nothing to hash. Worse, it broke consumers in a way that was hard to read. The SQL connectors also require a bare `core v1.6.22`, and a pseudo-version like `v1.6.22-0.<ts>-<sha>` is a PRE-RELEASE of v1.6.22, so semver sorts it BELOW the bare tag. MVS therefore discarded any working pseudo-version a consumer supplied, selected the bare tag, and failed with "unknown revision v1.6.22". The only way out was a `replace` in every consuming build. Both are now pinned to the commits on this PR's branch: core v1.6.22-0.20260902065001-9e288508778b project-flogo/core#300 flow v1.6.28-0.20260902065149-820199433c22 the runtime commit of this PR flow is pinned at the runtime commit rather than this branch's head deliberately: that commit is what actually provides StartTransactionalSubFlow, and it does not move as further commits land on the branch. The module now builds, vets and tests green with NO replace directives, resolving core and flow from the remote like any ordinary dependency, and go.sum carries real hashes. AT RELEASE: once core v1.6.22 and flow v1.6.28 are tagged, replace both pseudo-versions with the plain tags and re-run `go mod tidy`.
Adds the carrier that lets a transactional subflow hand a live *sql.Tx to the
activities running inside it, plus connection.GetId to resolve a connection
manager back to its registered id.
support/sqltx
Keeps ONE context key holding an immutable map[connID]*Handle, replaced
copy-on-write. A single context can therefore carry several independent
transactions (one per connection), and two flow instances running in parallel
never observe each other's registry. Propagate() returns the destination
context unchanged when the source carries no registry, so the non-
transactional path allocates nothing.
Handle serialises access to the one *sql.Tx that database/sql pins to a single
physical connection. Three locks with distinct jobs:
mu the operation lock connectors take around a statement
stmtsMu guards the prepared-statement memo, done flag and warn-once set
opMu guards the token-keyed in-flight cancel funcs
Splitting them keeps CancelInFlight() usable while mu is held by the very
operation being cancelled, which is what makes rollback-on-timeout work.
PrepareCached prepares ON THE TRANSACTION, never on the pool. Preparing on
*sql.DB would need a second pooled connection while the transaction pins the
first, which deadlocks outright at maxOpenConnection=1. The memo is bounded at
128 statements and reports saturation once, so a flow generating unbounded
distinct SQL degrades instead of growing without limit.
support/connection.GetId
Linear scan over the registry mirroring the existing IsShared, guarded by
recover() because manager implementations are not required to be comparable.
A map[Manager]string index was rejected: it panics at app startup as soon as
one registered manager has an uncomparable dynamic type.
Tests cover the registry copy-on-write semantics, concurrent handle use, memo
saturation, cancel/finish races and the ErrTxFinished path, using an in-repo
fake driver so no database is required.
awakchau-tibco
force-pushed
the
FLOGO-19484-transactional-subflow
branch
from
September 11, 2026 08:24
9e28850 to
e7c210a
Compare
awakchau-tibco
added a commit
to project-flogo/flow
that referenced
this pull request
Sep 11, 2026
…of core and flow The module required bare `core v1.6.22` and `flow v1.6.28`. Neither tag exists yet, so the module could not be resolved, built or tested by anyone -- and `go.sum` could not be generated at all, because there was nothing to hash. Worse, it broke consumers in a way that was hard to read. The SQL connectors also require a bare `core v1.6.22`, and a pseudo-version like `v1.6.22-0.<ts>-<sha>` is a PRE-RELEASE of v1.6.22, so semver sorts it BELOW the bare tag. MVS therefore discarded any working pseudo-version a consumer supplied, selected the bare tag, and failed with "unknown revision v1.6.22". The only way out was a `replace` in every consuming build. Both are now pinned to the commits on this PR's branch: core v1.6.22-0.20260902065001-9e288508778b project-flogo/core#300 flow v1.6.28-0.20260902065149-820199433c22 the runtime commit of this PR flow is pinned at the runtime commit rather than this branch's head deliberately: that commit is what actually provides StartTransactionalSubFlow, and it does not move as further commits land on the branch. The module now builds, vets and tests green with NO replace directives, resolving core and flow from the remote like any ordinary dependency, and go.sum carries real hashes. AT RELEASE: once core v1.6.22 and flow v1.6.28 are tagged, replace both pseudo-versions with the plain tags and re-run `go mod tidy`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FLOGO-19484 — Support for DB transactions using a subflow
A subflow can be marked transactional against a chosen SQL connection. The transaction
BEGINs when the subflow starts and is COMMITted or ROLLed BACK automatically from the subflow's
outcome — there is no explicit commit/rollback activity to place in the flow.
It is generic across SQL Server, MySQL, Oracle and PostgreSQL through
database/sql. Eachsubflow invocation gets its own transaction, parallel requests never share one, and the
transaction reaches the activities inside the subflow through the go context.
How it fits together
The design point worth knowing: the connection manager is a process-wide singleton, so the
transaction cannot live on it. An earlier approach that handed out a "transaction-enabled"
manager was dropped for exactly this reason — it would have leaked one request's transaction into
every other request on the same connection. The context is the only per-invocation carrier
available, which is why
sqltxkeys off it.Merge order — this is 1 of 7 PRs
project-flogo/core#300support/sqltx,connection.GetIdtibco/wi-contrib#197project-flogo/flow#224activity/subflowmodulev1.6.22tibco/wi-mssql#50v1.6.22tibco/wi-mysql#48v1.6.22tibco/wi-oracledb#40v1.6.22tibco/wi-postgres#35v1.6.22Release sequence:
v1.6.20andv1.6.21already exist and do not containsupport/sqltx, so the dependentmodules pin
v1.6.22deliberately: an unresolvable version fails loudly, whereas pinning anexisting tag would fail with a confusing "undefined: sqltx" instead.
What this PR adds
support/sqltx— the carrier that lets a transactional subflow hand a live*sql.Txto theactivities running inside it — plus
connection.GetId, which resolves a manager back to itsregistered id.
Nothing in core calls this yet. It is the shared vocabulary the flow runtime and the four SQL
connectors are written against, so it has to land first.
+1352 / −0 across 7 files. No existing file changes behaviour;
registry.goonly gains afunction.
support/sqltxOne context key, holding an immutable
map[connID]*Handle, replaced copy-on-write.A single context can therefore carry several independent transactions (one per connection), and
two flow instances running in parallel never observe each other's registry.
Propagate(src, dst)returnsdstidentically whensrccarries no registry, so thenon-transactional path — which is every existing flow — allocates nothing and is unaffected.
Handleserialises access to the single*sql.Tx.database/sqlpins a transaction to onephysical connection, so two activities on parallel branches of the same subflow must not use it
concurrently. Three locks, with deliberately separate jobs:
muLock()/TryLockFor()stmtsMuopMuThey are separate so
CancelInFlight()stays callable whilemuis held by the very operationbeing cancelled. Collapsing them into one mutex deadlocks rollback-on-timeout.
PrepareCachedprepares on the transaction, never on the pool. This is a correctnessrequirement, not a preference:
db.Prepareneeds a second pooled connection while thetransaction pins the first, so at
maxOpenConnection: 1it deadlocks outright. Preparing on theTx removes the pool from the picture entirely.
The memo is bounded at 128 statements and reports saturation exactly once, so a flow that
generates unbounded distinct SQL degrades rather than growing without limit.
connection.GetIdA linear scan over the registry, mirroring the existing
IsShared, guarded byrecover().A
map[Manager]stringindex would be O(1) and was rejected: manager implementations are notrequired to be comparable, and inserting one with an uncomparable dynamic type panics at app
startup. A scan over a registry that is written once at startup and then only read is not worth
that risk.
Testing
go test ./support/...—support/sqltxandsupport/connectiongreen.Covered: registry copy-on-write semantics, concurrent handle use, memo saturation, cancel/finish
races, and the
ErrTxFinishedpath. An in-repo fakedriver.Drivermeans no database isneeded to run them.
Pre-existing failure, unrelated to this PR
support.TestURLStringToFilePathfails on Windows (\vs/in the expected path). Verifiedidentical at
origin/masterin a detached worktree — it is not affected by this change.Reviewer notes
support/connection/registry.go, and only additively.sqltx.Withoutis currently unused. It is the inverse ofWithHandleand exists for asuppression case that did not materialise — happy to drop it if you would rather not carry it.