[simolus3]

This replaces the custom connection pool implementation based on isolate messages with the sqlite3_connection_pool package.

That package manages a SQLite connection pool (one writer, additional readers) in Rust. Compared to our Dart implementation, this has a few benefits:

1. Shared-memory multithreading: Since each database pool is essentially an Arc in Rust, two isolates can access the same pool independently and share connection locks. In Dart, coordinating this required explicit isolate communication. This also works with different Flutter engines (e.g. a background task and the main UI, which was impossible before).
2. Safer lifetime management: Since pools are managed in Rust, we can use reliable native finalizers to clean them up. This makes it impossible to leak connections in Dart. Even with isolate crashing or hot restarts, existing pool handles are cleaned up.
3. Shared table updates: Registering update, commit and rollback hooks in Rust and then using Dart native APIs to send updates to ports is more efficient than registering the listener in Dart.

The size of the diff really tells the full story here, implementing connection pools via isolate communication is really hard and this removes a lot of complexity from the project.

Changes in behavior and possible regressions

Previously, connections were owned by a single isolate. Now, connections are shared in a pool and may be accessed from different isolates too (but never concurrently, we're still compatible with THREADSAFE=1 mode in SQLite). Due to the way native callables work in Dart, this makes it impossible to register Dart functions that could be called from SQLite:

1. No Dart-based table updates (registering hooks in Rust is a better workaround).
2. No user-defined Dart functions (I had to migrate a few tests for this), it's also a breaking change.
3. No custom VFS (I don't think anyone has ever used that with the asynchronous parts of this package though).

I have also removed the isolate connection factory API. Instead, users should simply create a second instance using the same path on another isolate. The Rust helper will automatically merge the two instances internally.

Other changes in behavior that I observed from tests:

1. On timeouts, we now throw a PoolAbortException instead of a TimeoutException. I think this is fine, especially since we're about to migrate from timeouts to a more general abort mechanism.

Benchmarks

I re-ran benchmarks that are enabled by default using dart run scripts/benchmark.dart:

Benchmark	Before (runs/s)	After (runs/s)
Insert: JSON1	47.994	89.135
writeLock in isolate	7.2002	48.407
Insert: Direct	60.705	79.258
Insert: writeTransaction	114.92	101.85
Insert: executeBatch in isolate	57.336	66.313
Insert: direct write in isolate	12.693	19.417
Insert: writeTransaction no await	205.63	108.18
Insert: computeWithDatabase	31.778	35.707
Insert: computeWithDatabase, prepared	68.894	91.491
Insert: executeBatch	71.798	91.667

Most benchmarks benefit from this change, writeTransaction no await is a notable regression. I didn't profile this, but I assume that's because the non-awaiting benchmark relied on a single isolate owning the database serializing everything (since it's a single thread). The other regression for write transactions is worth investigating as well, but I don't think that's a blocker for this PR.

[simolus3]

Some comments that might help to review this.

[rkistner]

I like this change overall.

To confirm my understanding:

1. This still uses a pool of connections and a pool of Isolates, similar to before.
2. Unlikely before, it is no longer one connection per Isolate - each individual operation gets assigned a connection ref and an Isolate, but that Isolate can use any connection.

Is that correct?

I am curious, since this is not using the Isolate.run optimization, where do the performance improvements come from?

---

I notice there is a still a custom_functions_example.dart, but it crashes hard with Callback invoked after it has been deleted. Is there a way to prevent those custom functions from being created in the APIs, or otherwise throw a better error?

[simolus3]

each individual operation gets assigned a connection ref and an Isolate, but that Isolate can use any connection.

Correct.

I am curious, since this is not using the Isolate.run optimization, where do the performance improvements come from?

That is a good question. I wanted to use Isolate.run initially since that would simplify things even further by removing the compute pool. But it looks like the overhead of spawning new isolates is still fairly substantial. It's not as high as it used to be in older Dart versions, but caused a regression of about 50% for most write benchmarks.

It is worth noting that default benchmarks only measure writes without RETURNING clauses. Isolate.run would have mostly helped for large results sets which could be moved to the calling isolate instead of being copied. I still think that might be an improvement worth investigating at some point (e.g. by providing a hint parameter that we expect large results in which case we'd use Isolate.run internally, or a way for computeWithDatabase to spin up a fresh isolate).

I believe the performance benefits at this point are mostly a result of managing locks in Rust. Some benchmarks that only acquire / release locks did see substantial improvements:

• Write lock: 149k units/s before, 412k after
• Read lock: 87k units/s before, 500k after

Managing locks wasn't particularly expensive before either, but I suspect it's enough to cause the modest improvement.

[simolus3]

Is there a way to prevent those custom functions from being created in the APIs, or otherwise throw a better error?

I'm not aware of a good way to do that without adding APIs to package:sqlite3 (e.g. to open databases in a way that doesn't allow adding functions or listeners). That might be something worth doing, apart from that we only have the changelog entry / docs.