mwarchiver

mwarchiver restores a public 52Poké Wiki XML snapshot into a local MediaWiki installation for preservation, development, and single-user use. It can create a complete local container stack or import into an existing database.

The XML export is produced by MediaWiki's dumpBackup.php maintenance command.

GitHub Release archives

GitHub Releases contain public 52Poké Wiki snapshots licensed under CC BY-NC-SA 3.0. The release assets are data archives, not mwarchiver binaries. Please also read the 52Poké Wiki machine-reading rules.

ZIM offline archive

mwoffliner-YYYY.MM.DD-zim.zip contains a .zim file generated by mwoffliner. It is intended for Kiwix and other ZIM readers.

MediaWiki XML archive

mwarchiver-YYYY.MM.DD.zip contains 52poke-YYYY.MM.DD.xml.gz, a gzip-compressed MediaWiki XML export. It contains the current revision of pages in these public namespaces:

• Main (0)
• 52W (4)
• File (6)
• MediaWiki (8)
• Template (10)
• Help (12)
• Category (14)

Installation

Install with Go 1.25 or later:

go install github.com/52poke/mwarchiver@main
mwarchiver restore --help

Restoring also requires Docker with Compose support; Podman may be selected with --container-cli.

Restoring an XML snapshot

Start the interactive workflow with a .xml, .xml.gz, .xml.bz2, or .xml.7z path:

mwarchiver restore /absolute/path/to/52poke-YYYY.MM.DD.xml.gz

The workflow lets you choose:

• A new MariaDB database and MediaWiki tables, or an existing database.
• A local 52poke/mediawiki checkout, or the published ghcr.io/52poke/mediawiki:latest image.
• Optional persistent Elasticsearch/CirrusSearch support.
• An optional Timburr/EventBus endpoint. Enabling it also starts a local Kafka broker for Timburr.
• Optional OAuth support and an approved OAuth 1.0a development client.

An administrator account is always created or updated.

Import performance and progress

By default, the XML import restores page and revision content without parsing every page immediately. This is substantially faster, but category membership, backlinks, template links, and other derived link tables are incomplete until they are rebuilt:

mwarchiver restore rebuild-links

If content import has completed but a later stage such as OAuth client creation fails, retry the restore with the same configuration and --skip-import; the existing page and revision data is left untouched.

Using a MediaWiki checkout

When a checkout is selected, the generated stack layers on its devcontainer Compose configuration. The devcontainer owns the wiki build and disposable Memcached service. mwarchiver owns MariaDB and optional Elasticsearch and Kafka because those services have persistent data.

Before initializing or importing the database, mwarchiver updates the checkout's Composer dependencies as www-data. The checkout image maps that account to the invoking user's UID and GID so generated vendor files remain editable on the host.

The workflow creates LocalSettings.php in the checkout and wires the selected services. An existing LocalSettings.php is not replaced without confirmation or --force-settings; its original contents are backed up privately under the restore state directory before replacement.

Elasticsearch indexing is a separate, potentially long-running stage. After importing content and preferably rebuilding derived links, create or replace the CirrusSearch index with:

mwarchiver restore rebuild-search-index

Using the published MediaWiki image

Without a checkout, the generated stack uses ghcr.io/52poke/mediawiki:latest and a minimal local configuration. Object storage, SMTP, Redis, and other credential-dependent production services are not configured.

Non-interactive examples

Create a new database using a checkout, Elasticsearch, EventBus, and OAuth:

mwarchiver restore /absolute/path/to/dump.xml.gz \
  --non-interactive \
  --target new \
  --bind-address 192.0.2.25 \
  --mediawiki-dir /absolute/path/to/mediawiki \
  --elasticsearch \
  --eventbus-url http://host.docker.internal:5001 \
  --oauth \
  --oauth-callback http://localhost:3000/oauth/callback \
  --force-settings

Import into an existing database by supplying credentials through a file. The database host must be reachable from the MediaWiki container; on Linux, host.docker.internal is configured automatically:

mwarchiver restore /absolute/path/to/dump.xml.gz \
  --non-interactive \
  --target existing \
  --db-host host.docker.internal \
  --db-name 52poke_wiki \
  --db-user mediawiki \
  --db-password-file /absolute/path/to/db-password

Review the generated configuration without starting containers:

mwarchiver restore /absolute/path/to/dump.xml.gz \
  --non-interactive \
  --prepare-only

Run mwarchiver restore --help for all database, administrator, image, port, OAuth, and state-directory options.

Managing a prepared restoration

Once a restore has been prepared, manage it without importing the dump again:

mwarchiver restore status
mwarchiver restore down  # removes containers and networks; preserves volumes
mwarchiver restore up    # recreates the previously selected services
mwarchiver restore rebuild-links  # populates deferred link/category tables
mwarchiver restore rebuild-search-index  # creates/populates the CirrusSearch index

Use the same --state-dir with these commands if the restore was prepared in a non-default location. MariaDB, Elasticsearch, and Kafka named volumes persist across down and up, and up restarts the exact services recorded in the saved state.

The generated administrator password is stored in .local/restore/mediawiki-admin-password. OAuth credentials, when requested, are written once to .local/restore/oauth-client.json. MediaWiki's OAuth maintenance utility requires callback-prefix mode, so the configured callback URL is treated as an allowed URL prefix. Generated secrets are mounted with Compose secrets and are not embedded in LocalSettings.php.

License

mwarchiver is licensed under the MIT License.

The archives published in GitHub Releases are licensed under CC BY-NC-SA 3.0. Commercial use and any implication of endorsement by 52Poké are strictly prohibited.