add(CI): upgrade between version smoke test

Author: elcreator

.github/docker/ci/README.md

@@ -1,10 +1,19 @@
-# Install checks on MySQL and PostgreSQL
+# Install and upgrade checks
 
-Installs Evolution CMS onto a real database server and asserts the result, so
-the MySQL and PostgreSQL paths of `install/cli-install.php` get the coverage the
-sqlite path already gets from the build workflow.
+Two scripts, one set of assertions:
 
-Run either leg from the repository root:
+- `install-and-smoke.sh` installs this tree onto a database and checks what it
+  put there, then runs the updater over it to catch anything not idempotent.
+- `upgrade-and-smoke.sh` installs an **older release**, copies this tree over it
+  the way unpacking a release archive would, and runs the updater. That is the
+  only check that replays the migration chain over a schema this branch did not
+  create - what every existing site does when it moves to a new release.
+
+Both take a `sqlite`, `mysql` or `pgsql` database from `EVO_DB_TYPE`.
+
+## Installing
+
+Run either database leg from the repository root:
 
 ```sh
 docker compose -f .github/docker/ci/docker-compose.yml --profile mysql up \
@@ -18,10 +27,43 @@ Clean up with `docker compose -f .github/docker/ci/docker-compose.yml --profile
 <mysql|pgsql> down -v`. The container works on a copy of the tree, not a bind
 mount, so a run leaves the working tree alone.
 
+sqlite needs no container at all:
+
+```sh
+EVO_DB_TYPE=sqlite EVO_APP_DIR="$PWD" .github/docker/ci/install-and-smoke.sh
+```
+
 `PHP_VERSION`, `MYSQL_IMAGE_TAG`, `POSTGRES_IMAGE_TAG`, `MYSQL_PORT` and
 `POSTGRES_PORT` override the defaults (PHP 8.3, MySQL 9.7 LTS, PostgreSQL 18.6
 LTS, the servers' own ports).
 
+## Upgrading
+
+`upgrade-and-smoke.sh` needs two trees: the older one it installs and upgrades
+in place, and this one to copy over it. The older tree is left dirty, so give it
+a checkout that can be thrown away.
+
+```sh
+git clone --depth 1 --branch 3.5.7 https://github.com/evolution-cms/evolution /tmp/evo-3.5.7
+EVO_FROM_DIR=/tmp/evo-3.5.7 EVO_NEW_DIR="$PWD" EVO_DB_TYPE=mysql EVO_DB_HOST=127.0.0.1 EVO_DB_USER=root EVO_DB_PASSWORD=secret     .github/docker/ci/upgrade-and-smoke.sh
+```
+
+Two things about the source version are worth knowing:
+
+- **No released version installs on sqlite.** `cli-install.php` offered only
+  `pgsql` and `mysql` through 3.5.7; the sqlite branch is new in this tree. So a
+  sqlite upgrade leg has to take its source from a ref that carries it - the
+  `nightly-3.5.x` tag, or any 3.5.x commit after it landed. A tag works here
+  like any other ref once a release ships sqlite support.
+- **No `--skipComposer` before 3.5.8 either.** An older `composerUpdate()` runs
+  whatever it finds at `core/vendor/bin/composer` and only warns when that is
+  missing, so the script moves the shim aside: `core/vendor` is committed at
+  every tag, and a composer update would test the network rather than the CMS.
+
+The source install is checked for hard failures only. A release run on a newer
+PHP may emit deprecations that are not this branch's to fix, and they are not
+what is under test - the updater's own log is checked strictly.
+
 ## What CI runs
 
 The `install` job in `.github/workflows/ci.yml` does not use the image above.
@@ -36,9 +78,18 @@ same checks; the image is what makes a local run reproducible on a machine with
 no PHP on it, and what pins the PHP version when a version question is the one
 being investigated.
 
-One PHP version, both databases: what the job proves is the installer's MySQL
-and PostgreSQL paths, which do not vary with the PHP minor. 8.3 and 8.4 are both
-covered by the analysis and unit test jobs.
+One PHP version, every database: what the job proves is the installer's database
+paths, which do not vary with the PHP minor. 8.3 and 8.4 are both covered by the
+analysis and unit test jobs. The sqlite leg starts no container.
+
+The upgrade checks live in their own workflow, `.github/workflows/upgrade.yml`,
+off `push` and `pull_request` on purpose: an upgrade regression comes from a
+change to the migrations or the seeders, not from every commit, and each leg
+costs a full install plus a full update. It runs nightly, and on demand with the
+versions to go from and to as inputs (Actions -> Upgrade -> Run workflow):
+`from` takes one or more refs of any repository, `to` a ref of this one or blank
+for the checked out tree, which is the default - so the unreleased version is
+what an upgrade is tested against unless you say otherwise.
 
 ## What a run asserts
 
@@ -50,7 +101,7 @@ covered by the analysis and unit test jobs.
    migrations inside an output buffer it later discards, so the image also
    routes `error_log` to stderr; a warning raised during a migration cannot be
    swallowed.
-3. `smoke.php` checks what reached the database, reading the connection the
+3. `smoke.php --mode=install` checks what reached the database, reading the connection the
    installer just wrote: every table created, the columns the core migrations
    add, the seeded document, template, event names, settings, roles and
    permissions, the admin account with a hashed password and the Administrator
@@ -60,3 +111,18 @@ covered by the analysis and unit test jobs.
    run, which is what catches a migration or seeder that is not idempotent.
 5. The site answers over HTTP: the front page renders the seeded document, and
    the manager renders its login form.
+
+An upgrade run asserts the same things through `smoke.php --mode=upgrade`, with
+the expectations that belong to a fresh install relaxed - the content and the
+settings are whichever ones the older site had, not the ones this installer
+would have chosen. Two checks replace them, and they are the point of the mode:
+
+- the counts recorded from the older site by `--mode=baseline`, taken before the
+  copy, must not have **dropped** anywhere. A seeded catalogue may grow, because
+  topping those up is what the update seeder is for; losing rows is a bug.
+- `site_content`, `users` and `user_attributes` must be **unchanged**. Those are
+  the site, not the release, and an upgrade may not touch them.
+
+`manager_language` and `emailsender` stay exact in both modes: they are answers
+the operator gave the installer, and a seeder writing defaults over them would
+reset a live site.

.github/docker/ci/install-and-smoke.sh

@@ -4,77 +4,37 @@
 # seeded data present, the updater able to replay the chain over it without
 # changing anything, and the site answering over HTTP.
 #
-# Runs as the container entrypoint (see Dockerfile); every knob has a default
-# so `docker run` with only EVO_DB_HOST set does something sensible.
+# Runs as the container entrypoint (see Dockerfile) and, in CI, directly on the
+# runner; every knob has a default so `docker run` with only EVO_DB_HOST set
+# does something sensible. EVO_DB_TYPE=sqlite needs no server at all.
+#
+# For the upgrade of an older release to this tree, see upgrade-and-smoke.sh.
 # pipefail: the installer output goes through tee, and a pipeline that hides
 # the exit code of its first command would report a failed install as a pass.
 set -euo pipefail
 
 APP_DIR=${EVO_APP_DIR:-/var/www/html}
 
-EVO_DB_TYPE=${EVO_DB_TYPE:-mysql}
-EVO_DB_HOST=${EVO_DB_HOST:-127.0.0.1}
-EVO_DB_USER=${EVO_DB_USER:-root}
-EVO_DB_PASSWORD=${EVO_DB_PASSWORD:-secret}
-EVO_DB_NAME=${EVO_DB_NAME:-evolution}
-EVO_DB_PREFIX=${EVO_DB_PREFIX:-evo_}
+export EVO_DB_TYPE=${EVO_DB_TYPE:-mysql}
+export EVO_DB_HOST=${EVO_DB_HOST:-127.0.0.1}
+export EVO_DB_USER=${EVO_DB_USER:-root}
+export EVO_DB_PASSWORD=${EVO_DB_PASSWORD:-secret}
+export EVO_DB_NAME=${EVO_DB_NAME:-evolution}
+export EVO_DB_PREFIX=${EVO_DB_PREFIX:-evo_}
 
 # Exported: smoke.php reads these to check what the installer stored.
 export EVO_ADMIN=${EVO_ADMIN:-admin}
 export EVO_ADMIN_EMAIL=${EVO_ADMIN_EMAIL:-admin@evo.local}
 export EVO_ADMIN_PASSWORD=${EVO_ADMIN_PASSWORD:-Passw0rd123}
 export EVO_LANGUAGE=${EVO_LANGUAGE:-en}
 
+# shellcheck source=lib.sh
+. "$(dirname "$0")/lib.sh"
+
 LOG=/tmp/install.log
 UPDATE_LOG=/tmp/update.log
-PORT=${EVO_HTTP_PORT:-8899}
-
-say() {
-    printf '\n\033[1;36m== %s\033[0m\n' "$1"
-}
-
-fail() {
-    printf '\033[1;31m!! %s\033[0m\n' "$1" >&2
-    exit 1
-}
-
-# Rejects a run that printed anything PHP or PDO would call a problem.
-#
-# php.ini in the image sends diagnostics to stdout and to stderr (see
-# Dockerfile), and both are captured here, so a warning raised inside the output
-# buffer the installer later discards still reaches this log.
-#
-# The patterns are anchored on how PHP formats a diagnostic ("Warning: text in
-# /file on line N") rather than on the bare words, which turn up in plenty of
-# legitimate output - the settings seeder alone writes rows whose names contain
-# "error".
-check_log() {
-    if grep -nE '(^|PHP )(Warning|Notice|Deprecated|Fatal error|Parse error|Recoverable fatal error|Strict Standards):' "$1"; then
-        fail "the $2 emitted PHP diagnostics"
-    fi
-    # SQLSTATE is how both PDO drivers label a failed statement, and the
-    # installer swallows some of those into plain output instead of exiting.
-    if grep -nE 'SQLSTATE\[|Uncaught .*Exception|Migration not found|✖' "$1"; then
-        fail "the $2 reported a database or migration error"
-    fi
-    echo "no diagnostics in $(wc -l < "$1") lines of $2 output"
-}
 
-say "Waiting for ${EVO_DB_TYPE} at ${EVO_DB_HOST}"
-# Compose already gates on a healthcheck, but this script is also meant to be
-# runnable on its own against any reachable server.
-attempt=0
-until php -r '
-    $dsn = $argv[1] === "pgsql"
-        ? "pgsql:host=" . $argv[2] . ";dbname=" . $argv[5]
-        : "mysql:host=" . $argv[2];
-    new PDO($dsn, $argv[3], $argv[4]);
-' "$EVO_DB_TYPE" "$EVO_DB_HOST" "$EVO_DB_USER" "$EVO_DB_PASSWORD" "${EVO_DB_BOOTSTRAP_NAME:-$EVO_DB_USER}" 2>/dev/null; do
-    attempt=$((attempt + 1))
-    [ "$attempt" -lt 60 ] || fail "database never became reachable"
-    sleep 2
-done
-echo "reachable after ${attempt} attempt(s)"
+wait_for_db
 
 say "Installing Evolution CMS (${EVO_DB_TYPE}, database '${EVO_DB_NAME}', prefix '${EVO_DB_PREFIX}')"
 # --skipComposer=y: core/vendor is committed, so there is nothing to install and
@@ -83,16 +43,12 @@ say "Installing Evolution CMS (${EVO_DB_TYPE}, database '${EVO_DB_NAME}', prefix
 # sqlite leg of the build workflow already covers removal.
 # Interactive prompts read from stdin; </dev/null makes an unanswered prompt an
 # immediate failure instead of a hung job.
+mapfile -t db_args < <(installer_db_args)
 cd "$APP_DIR/install"
 set +e
 php cli-install.php \
     --typeInstall=1 \
-    --databaseType="$EVO_DB_TYPE" \
-    --databaseServer="$EVO_DB_HOST" \
-    --database="$EVO_DB_NAME" \
-    --databaseUser="$EVO_DB_USER" \
-    --databasePassword="$EVO_DB_PASSWORD" \
-    --tablePrefix="$EVO_DB_PREFIX" \
+    "${db_args[@]}" \
     --cmsAdmin="$EVO_ADMIN" \
     --cmsAdminEmail="$EVO_ADMIN_EMAIL" \
     --cmsPassword="$EVO_ADMIN_PASSWORD" \
@@ -140,55 +96,6 @@ say "Checking the data survived the update unchanged"
 php "$APP_DIR/.github/docker/ci/smoke.php" "$APP_DIR" || fail "the update changed the installed data"
 
 say "Checking the installed site answers over HTTP"
-php -S "127.0.0.1:${PORT}" -t "$APP_DIR" > /tmp/server.log 2>&1 &
-server=$!
-trap 'kill "$server" 2>/dev/null || true' EXIT
-
-php -r '
-    // The front end and the manager are the two entry points an installed site
-    // has to serve. The manager answers 404 without an Accept-Language header
-    // by design, hence the header on the second request.
-    $port = $argv[1];
-    $get = function (string $path, array $headers = []) use ($port): array {
-        $context = stream_context_create(["http" => [
-            "ignore_errors" => true,
-            "timeout" => 20,
-            "header" => $headers,
-        ]]);
-        $body = @file_get_contents("http://127.0.0.1:{$port}{$path}", false, $context);
-        $status = isset($http_response_header[0]) ? (int) substr($http_response_header[0], 9, 3) : 0;
-        return [$status, (string) $body];
-    };
-
-    for ($i = 0; $i < 30; $i++) {
-        [$status] = $get("/");
-        if ($status !== 0) {
-            break;
-        }
-        sleep(1);
-    }
-
-    $failures = [];
-    [$status, $body] = $get("/");
-    echo "front page: {$status}\n";
-    $status === 200 or $failures[] = "the front page answered {$status}";
-    // The seeded document, rendered through the seeded template - proof the
-    // CMS read its own data rather than merely booting.
-    str_contains($body, "Install Successful!") or $failures[] = "the front page did not render the seeded document";
-
-    [$status, $body] = $get("/manager/index.php", ["Accept-Language: en-US,en;q=0.9"]);
-    echo "manager: {$status}\n";
-    $status === 200 or $failures[] = "the manager answered {$status}";
-    stripos($body, "password") !== false or $failures[] = "the manager did not render its login form";
-
-    if ($failures !== []) {
-        fwrite(STDERR, "  - " . implode("\n  - ", $failures) . "\n");
-        exit(1);
-    }
-    echo "both entry points served the installed site\n";
-' "$PORT" || {
-    tail -40 /tmp/server.log >&2
-    fail "the installed site did not answer correctly"
-}
+http_check "$APP_DIR"
 
 say "OK: ${EVO_DB_TYPE} installation is clean, migrated and seeded"