Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changes/unreleased/added-20260610-095806.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: added
body: Support MySQL and PostgreSQL databases in the Migrator and the CLI
time: 2026-06-10T09:58:06.000000+00:00
custom:
Issue: ""
5 changes: 5 additions & 0 deletions .changes/unreleased/changed-20260610-095807.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: changed
body: Move database drivers (sqlite3, mysql, pg) to development dependencies; applications require the driver they use
time: 2026-06-10T09:58:07.000000+00:00
custom:
Issue: ""
5 changes: 5 additions & 0 deletions .changes/unreleased/changed-20260610-095808.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: changed
body: Rename Drift::Migrator::MigrationEntry to Drift::MigrationEntry; update type annotations that reference the nested name
time: 2026-06-10T09:58:08.000000+00:00
custom:
Issue: ""
27 changes: 27 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,35 @@ jobs:

test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:17
env:
POSTGRES_PASSWORD: drift
POSTGRES_DB: drift_test
ports:
- "5432:5432"
options: >-
--health-cmd "pg_isready -h 127.0.0.1 -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
env:
MYSQL_DATABASE_URL: mysql://root:drift@127.0.0.1:3306/drift_test
POSTGRES_DATABASE_URL: postgres://postgres:drift@127.0.0.1:5432/drift_test
steps:
- uses: actions/checkout@v4
- uses: crystal-lang/install-crystal@v1
# crystal-mysql only speaks mysql_native_password, which MySQL 8.4
# disables by default; service containers cannot pass server flags,
# so MySQL starts via docker run instead.
- name: Start MySQL
run: |
docker run --detach --name mysql --publish 3306:3306 \
--env MYSQL_ROOT_PASSWORD=drift --env MYSQL_DATABASE=drift_test \
mysql:8.4 --mysql-native-password=ON --authentication-policy=mysql_native_password,,
- run: shards install
- name: Wait for MySQL
run: |
timeout 90 sh -c 'until docker exec mysql mysqladmin ping -h 127.0.0.1 --silent; do sleep 2; done' || { docker logs mysql; exit 1; }
- run: crystal spec
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
/database
/docs/
/lib/
/.worktrees/
NOTES.txt

# Libraries don't need dependency lock
Expand Down
9 changes: 9 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ src/
├── migration.cr # SQL parser
├── migrator.cr # Migration orchestrator
├── context.cr # Migration collection manager
├── dialect.cr # Dialect base class + selection (from_db)
├── dialects/ # Per-engine SQL (sqlite3, mysql, postgresql)
├── migration_entry.cr # Applied migration record
├── embed.cr # Macro for embedding migrations
├── commands/ # CLI commands (migrate, rollback, reset, status, new, version)
└── support/migrations_loader.cr
Expand Down Expand Up @@ -64,6 +67,12 @@ crystal spec # All tests
crystal spec spec/drift/migration_spec.cr # Specific file
crystal spec --verbose # Verbose

# Cross-engine specs (MySQL/PostgreSQL) skip unless these are set. The app
# container in compose.yaml sets them already; from the host:
docker compose up -d mysql postgres
export MYSQL_DATABASE_URL="mysql://root:drift@127.0.0.1:3306/drift_test"
export POSTGRES_DATABASE_URL="postgres://postgres:drift@127.0.0.1:5432/drift_test"

# Format (enforced by CI)
crystal tool format src/ spec/ # Apply
crystal tool format --check src/ spec/ # Check only
Expand Down
46 changes: 45 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ in reverse order using the information on the previously mentioned table.
## Requirements

Drift CLI is a standalone, self-contained executable capable of connecting to
SQLite databases.
SQLite, MySQL and PostgreSQL databases.

Drift (as library) only depends on Crystal's
[`db`](https://github.com/crystal-lang/crystal-db) common API. To use it with
Expand Down Expand Up @@ -201,6 +201,28 @@ The above is a simplified version of what happens when doing `drift migrate`
in the CLI. For example, you could apply these migrations as part of your
application start process.

#### Supported databases

Drift works with SQLite, MySQL and PostgreSQL. It detects the engine from
the connection and uses the matching SQL to track applied migrations, so
they work the same way independently of the database you choose.

Add the driver for your database to `shard.yml` and require it before Drift:

* SQLite: [crystal-sqlite3](https://github.com/crystal-lang/crystal-sqlite3)
* MySQL: [crystal-mysql](https://github.com/crystal-lang/crystal-mysql)
* PostgreSQL: [crystal-pg](https://github.com/will/crystal-pg)

```crystal
require "sqlite3" # or "mysql" or "pg"
require "drift"
```

Note: MySQL cannot roll back a failed migration batch. Every DDL statement
(Eg. `CREATE TABLE` or `ALTER TABLE`) triggers an implicit commit, so the
migrations applied earlier in that batch stay applied. SQLite and PostgreSQL
support transactional DDL, so a failed batch rolls back as a whole.

Internally, the library interconnects the following elements:

A **migration** (`Drift::Migration`): represents each individual SQL file. It
Expand Down Expand Up @@ -313,6 +335,28 @@ In this pattern, `embed_as` generates an instance method named `context` that
satisfies the abstract method requirement from `Migratable`. Each database class
can embed its own set of migrations while sharing the common migration logic.

## Development

Specs against MySQL and PostgreSQL need a running server, so they are
skipped (shown as pending) unless `MYSQL_DATABASE_URL` and
`POSTGRES_DATABASE_URL` are set. SQLite specs always run, no setup needed.

The development container starts both databases and sets both variables
for you:

```console
$ docker compose run --rm app -- crystal spec
```

To run the specs from the host instead:

```console
$ docker compose up -d mysql postgres
$ export MYSQL_DATABASE_URL="mysql://root:drift@127.0.0.1:3306/drift_test"
$ export POSTGRES_DATABASE_URL="postgres://postgres:drift@127.0.0.1:5432/drift_test"
$ crystal spec
```

## Contribution policy

Inspired by [Litestream](https://github.com/benbjohnson/litestream) and
Expand Down
38 changes: 38 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,47 @@ services:
- OVERMIND_SOCKET=/tmp/overmind.sock
# Disable Shards' postinstall
- SHARDS_OPTS=--skip-postinstall
# Cross-engine specs connect to the sibling database services
- MYSQL_DATABASE_URL=mysql://root:drift@mysql:3306/drift_test
- POSTGRES_DATABASE_URL=postgres://postgres:drift@postgres:5432/drift_test

depends_on:
mysql:
condition: service_healthy
postgres:
condition: service_healthy

# Set these env variables using `export FIXUID=$(id -u) FIXGID=$(id -g)`
user: ${FIXUID:-1000}:${FIXGID:-1000}

volumes:
- .:/workspace/${COMPOSE_PROJECT_NAME}:cached

mysql:
image: mysql:8.4
# crystal-mysql only speaks mysql_native_password, which MySQL 8.4
# disables by default (and 9.x removes entirely).
command: --mysql-native-password=ON --authentication-policy=mysql_native_password,,
environment:
MYSQL_ROOT_PASSWORD: drift
MYSQL_DATABASE: drift_test
ports:
- "3306:3306"
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-pdrift"]
interval: 5s
timeout: 5s
retries: 20

postgres:
image: postgres:17
environment:
POSTGRES_PASSWORD: drift
POSTGRES_DB: drift_test
ports:
- "5432:5432"
healthcheck:
test: ["CMD", "pg_isready", "-h", "127.0.0.1", "-U", "postgres"]
interval: 5s
timeout: 5s
retries: 20
7 changes: 7 additions & 0 deletions shard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ dependencies:
db:
github: crystal-lang/crystal-db
version: ~> 0.15.0
development_dependencies:
sqlite3:
github: crystal-lang/crystal-sqlite3
version: ~> 0.24.0
mysql:
github: crystal-lang/crystal-mysql
version: ~> 0.18.0
pg:
github: will/crystal-pg
version: ~> 0.31.0
101 changes: 101 additions & 0 deletions spec/drift/dialect_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Copyright 2026 Luis Lavena
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require "../spec_helper"

require "sqlite3"
require "mysql"
require "pg"

# A connection crystal-db accepts but no Drift dialect knows about,
# used to exercise the UnsupportedDialectError path without a server.
private class BogusConnection < DB::Connection
def initialize
super(DB::Connection::Options.new)
end

def driver_name : String
"bogus"
end

def build_prepared_statement(query) : DB::Statement
raise NotImplementedError.new("build_prepared_statement")
end

def build_unprepared_statement(query) : DB::Statement
raise NotImplementedError.new("build_unprepared_statement")
end
end

describe Drift::Dialect do
describe ".from_db" do
it "returns SQLite3 dialect for a SQLite connection" do
conn = DB.connect "sqlite3:%3Amemory%3A"
begin
Drift::Dialect.from_db(conn).should be_a(Drift::Dialects::SQLite3)
ensure
conn.close
end
end

it "returns SQLite3 dialect for a SQLite database pool" do
db = DB.open "sqlite3:%3Amemory%3A"
begin
Drift::Dialect.from_db(db).should be_a(Drift::Dialects::SQLite3)
ensure
db.close
end
end

it "raises UnsupportedDialectError for unknown drivers" do
expect_raises(Drift::UnsupportedDialectError, /bogus/) do
Drift::Dialect.from_db(BogusConnection.new)
end
end
end
end

if mysql_url = ENV["MYSQL_DATABASE_URL"]?
describe Drift::Dialect do
it "returns MySQL dialect for a MySQL connection" do
db = DB.open(mysql_url)
begin
Drift::Dialect.from_db(db).should be_a(Drift::Dialects::MySQL)
ensure
db.close
end
end
end
else
describe Drift::Dialect do
pending "MySQL dialect selection (set MYSQL_DATABASE_URL to run)"
end
end

if pg_url = ENV["POSTGRES_DATABASE_URL"]?
describe Drift::Dialect do
it "returns PostgreSQL dialect for a PostgreSQL connection" do
db = DB.open(pg_url)
begin
Drift::Dialect.from_db(db).should be_a(Drift::Dialects::PostgreSQL)
ensure
db.close
end
end
end
else
describe Drift::Dialect do
pending "PostgreSQL dialect selection (set POSTGRES_DATABASE_URL to run)"
end
end
25 changes: 25 additions & 0 deletions spec/drift/migrator_lifecycle_mysql_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Copyright 2026 Luis Lavena
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require "../support/migrator_lifecycle"

require "mysql"

if mysql_url = ENV["MYSQL_DATABASE_URL"]?
it_behaves_like_a_migrator("MySQL", mysql_url)
else
describe "Drift::Migrator (MySQL)" do
pending "set MYSQL_DATABASE_URL to run against a live server"
end
end
25 changes: 25 additions & 0 deletions spec/drift/migrator_lifecycle_postgresql_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Copyright 2026 Luis Lavena
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require "../support/migrator_lifecycle"

require "pg"

if pg_url = ENV["POSTGRES_DATABASE_URL"]?
it_behaves_like_a_migrator("PostgreSQL", pg_url)
else
describe "Drift::Migrator (PostgreSQL)" do
pending "set POSTGRES_DATABASE_URL to run against a live server"
end
end
19 changes: 19 additions & 0 deletions spec/drift/migrator_lifecycle_sqlite3_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 2026 Luis Lavena
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require "../support/migrator_lifecycle"

require "sqlite3"

it_behaves_like_a_migrator("SQLite3", "sqlite3:%3Amemory%3A")
Loading
Loading