Skip to content
lucascbeyelerPublic

About

A reliable software written in Java to help you in your daily task to backup and restore mails and accounts from Zimbra Open Source Email Platform.

Topics

Resources

Stars

153 stars

Watchers

21 watching

Forks

Repository files navigation

Zmbackup - Backup Script for Zimbra OSE

Zmbackup is a reliable tool developed to help you in your daily task to backup and restore mails and accounts from Zimbra Open Source Email Platform. It's based on another project called Zmbkpose, and completely compatible with the structure if you have plans on migrate from one to another.

Linux Distro Release Build Status

Features

  • Online Backup and Restore - no need to stop the server to do;
  • Backup routines for one, many, or all mailbox, accounts, alias and distribution lists;
  • Restore the routines in your respective places, or inside another account using Restore on Account;
  • Multithreading - Execute each rotine quickly as possible;
  • Have some insights about eacho backup routine;
  • Receive alert everytime a backup session begins;
  • Better internal garbage manager;
  • Filter the accounts that should not be execute with blocked lists;
  • Log management compatible with rsyslog;
  • Sessions stored in a SQLite3 database, with automatic one-time migration from the legacy TXT format;

Backup & Restore Scope

The table below documents what zmbackup covers.

Object Scope Backup Restore Command
Mailbox Per user Yes Yes zmbackup backup mailbox --account user@domain / zmbackup restore mailbox --session <id> --account user@domain
Mailbox All accounts Yes Yes zmbackup backup mailbox / zmbackup restore mailbox --session <id>
LDAP account entry Per user Yes Yes zmbackup backup ldap --account user@domain / zmbackup restore ldap --session <id> --account user@domain
LDAP account entry All accounts Yes Yes zmbackup backup ldap / zmbackup restore ldap --session <id>
Alias Per alias Yes Yes zmbackup backup alias --account alias@domain / zmbackup restore ldap --session <id> --account alias@domain
Distribution list Per list Yes Yes zmbackup backup distlist --account list@domain / zmbackup restore ldap --session <id> --account list@domain
Signature Per user Yes Yes zmbackup backup signature --account user@domain / zmbackup restore ldap --session <id> --account user@domain
Zimbra domain LDAP config Per domain Yes Yes zmbackup backup domain --domain domain.com / zmbackup restore domain --session <id>
Zimbra component passwords Internal services (localconfig.xml) Yes Yes zmbackup backup serverconfig / zmbackup restore serverconfig --session <id>
SSL/TLS certificates Host (/opt/zimbra/ssl by default) Yes Yes zmbackup backup serverconfig / zmbackup restore serverconfig --session <id>
Java Keystores (JKS) Host (/opt/zimbra/conf by default) Yes Yes zmbackup backup serverconfig / zmbackup restore serverconfig --session <id>
Zimbra server config Host — see serverConfig.paths (default /opt/zimbra/conf, /opt/zimbra/ssl) Yes Yes zmbackup backup serverconfig / zmbackup restore serverconfig --session <id>

Notes:

  • A full backup (zmbackup backup full) includes both the mailbox and the LDAP entry for each account by default.
  • An incremental backup (zmbackup backup incremental) also covers the mailbox, but only captures mail received since the last backup session.
  • --into <account> restores a mailbox into a different destination account (restore-on-account).
  • Use zmbackup list to list available session IDs before running a restore.
  • LDAP restores include password hashes. The LDAP backup dumps the full LDAP entry as the LDAP admin, which includes the userPassword attribute (the hashed password). Restoring an LDAP entry will therefore overwrite the account's current password with whatever hash was stored at backup time. Be aware of this before running a restore in production.
  • serverconfig backs up and restores host-level files, not Zimbra objects. It archives every path configured in serverConfig.paths (default: /opt/zimbra/conf and /opt/zimbra/ssl, which together cover localconfig.xml — where every Zimbra component's service password actually lives — TLS certificates, and Java keystores) as a single per-host archive, preserving file permissions. There is no --account/--domain for it — one session covers everything configured. Add /etc/zimbra to serverConfig.paths explicitly if you also want that directory covered; it's not included by default because some installs put root-owned files there, and files this tool's service account (zimbraMailbox.backupUser) can't read are silently skipped rather than causing the backup to fail.
  • Restoring serverconfig writes files directly back to their original host paths, not through any Zimbra API — unlike every other restore in this table. It does not stop or reload Zimbra services for you; restart the affected services (or run zmcertmgr/zmcontrol restart as appropriate) after restoring certificates, keystores, or localconfig.xml. If the archive contains anything that doesn't resolve back under the currently configured serverConfig.paths, the whole restore is refused and nothing is written - that check is a security boundary. A file that is in the allowed paths but that zimbraMailbox.backupUser simply doesn't have permission to overwrite (e.g. a root-owned file under /opt/zimbra/conf/crontabs on a stock install) is skipped individually instead - the restore still reports success, and zmbackup.log/syslog names exactly which files were skipped and why.

Requirements

  • Java 17 (JDK) - required to build and run zmbackup; the Gradle toolchain will download it automatically if it isn't already installed.

Installation

Download the latest package from the "Release" section, or git clone the development branch:

git clone -b master https://github.com/lucascbeyeler/zmbackup.git

Inside the project folder, execute the script install-java.sh and follow all the instructions to install the project. It checks for (and, if missing, installs) a Java 17 JDK, builds zmbackup.jar with the bundled Gradle wrapper, then installs the jar, a thin zmbackup launcher, zmbackup.yaml, the blocked list and a cron file.

If you're moving from an older 1.2.x (bash) install and its WORKDIR still has a sessions.txt, install-java.sh migrates it into the SQLite metadata store automatically at the end of the install (or upgrade), via the zmbackup migrate command. That command is also safe to run by hand at any time - $ zmbackup migrate - and is a no-op once there's nothing left to import (it renames sessions.txt to sessions.txt.migrated after a successful import, so re-running it, e.g. after install-java.sh --force-upgrade, doesn't import the same sessions twice). zmbackup migrate also normalizes a pre-existing SESSION_TYPE=SQLITE3 bash-tool database in place, so either 1.2.x session-store mode carries its history forward.

Before cutting over from 1.2.x, read this:

  • Never run the bash tool and this Java build against the same WORKDIR at the same time. There is no cross-tool locking between them (the bash tool's own checkpid was never actually wired up, and this build's PidLock only protects against other Java invocations) - a leftover bash cron job racing a Java invocation, or two processes writing the same session-ID directory, can corrupt sessions.sqlite3 or a backup's on-disk files. Cut cron over atomically (disable the old bash cron entries in the same change that enables the new ones) rather than leaving both running "just in case."
  • This build enforces TLS/certificate validation the bash tool never did. The bash tool's LDAP calls hardcoded TLS_REQCERT never (trust-all, with no way to turn it off); this build defaults to real CA validation and refuses to start against a self-signed Zimbra certificate unless zimbraLdap.caCertificatePath is set, or both trustAllCertificates: true and the top-level allowInsecure: true are explicitly set (same story for zimbraMailbox's REST connection). If your Zimbra install uses a self-signed certificate, configure caCertificatePath (preferred) before your first real backup, or the LDAP/REST connection will simply fail to start.

zmbackup truncate permanently empties sessions.sqlite3 (every session and account record) and refuses to do anything unless run as zmbackup truncate --force-clean. This is for test/development installs only - never run it against production, since the deleted session/account history cannot be recovered afterward. install-java.sh --remove offers to run it for you (with the same warning) before it removes the rest of the install.

# cd zmbackup
# ./install-java.sh
# su - zimbra
$ zmbackup --version
  zmbackup version: 2.0.1

Building the jar needs internet access on first run (to download Gradle and the project's Maven dependencies). Run ./install-java.sh --remove to uninstall, or ./install-java.sh --force-upgrade to rebuild and redeploy the jar. The upgrade compares /etc/zmbackup/zmbackup.yaml with what this version installs and asks for each setting it is missing, including a CA certificate when one is not set, with the value a new install would use as the default (an unattended upgrade uses that default without asking). It saves the previous file as zmbackup.yaml.<timestamp>.bak before adding your answers, and never changes settings you already have. See ./install-java.sh --help for details.

Carbonio

When the server has /opt/zextras and no /opt/zimbra, the installer treats it as Carbonio: the default user becomes zextras, the install path /opt/zextras and the backup path /opt/zextras/backup, and it looks for the CA certificate under /opt/zextras/ssl/carbonio/ca/ as well. Every default can still be changed at its prompt or through the environment variables below. Carbonio support is experimental and has not been tested against a Carbonio server. Two settings in /etc/zmbackup/zmbackup.yaml are not adjusted for it:

  • serverConfig.paths still defaults to /opt/zimbra/conf and /opt/zimbra/ssl. Set it to /opt/zextras/conf and /opt/zextras/ssl before using zmbackup backup serverconfig.
  • The mailbox REST API port still defaults to 7071. If your server serves it somewhere else, answer the installer's port prompt or set ZMBKP_REST_PORT, or change the port in zimbraMailbox.restBaseUrl after installing.

Checking dependencies without installing them

By default, install-java.sh installs a Java 17 JDK with apt or yum when it doesn't find one. If your server's packages are managed some other way and the installer must not add any, pass --no-install-deps: the installer still checks for the JDK, but when it is missing or older than 17 it prints the command to install it and stops with exit code 3, before anything is built or deployed. Install the JDK yourself and run the installer again. The option combines with --unattended and --force-upgrade.

# ./install-java.sh --no-install-deps

Unattended install (Ansible and other automation)

./install-java.sh --unattended (or -u) runs the same install without a single prompt: passing it accepts the license agreement, and every setting the interactive installer would ask for is read from an environment variable instead (anything left unset keeps its interactive default):

Variable Default
OSE_USER zimbra, or zextras on Carbonio
OSE_INSTALL_DIR /opt/zimbra, or /opt/zextras on Carbonio
OSE_INSTALL_ADDRESS the host's FQDN
OSE_INSTALL_LDAPPASS read from zmlocalconfig -s zimbra_ldap_password
OSE_INSTALL_DOMAIN the first domain from zmprov gad
OSE_DEFAULT_BKP_DIR /opt/zimbra/backup, or /opt/zextras/backup on Carbonio
ZMBKP_METADATA_DIR empty (same as the backup path)
ZMBKP_REST_ADMIN admin@<OSE_INSTALL_DOMAIN>
ZMBKP_REST_ADMIN_PASS required - no default
ZMBKP_REST_PORT 7071 - the port of Zimbra's mailbox REST API, written into zimbraMailbox.restBaseUrl
ZMBKP_MAIL_ALERT admin@<OSE_INSTALL_DOMAIN>
ZMBKP_MAIL_SENDER root@<OSE_INSTALL_DOMAIN>
MAX_PARALLEL_PROCESS 3
ROTATE_TIME 30
LOCK_BACKUP true
ZMBKP_CA_CERT Zimbra's CA file when it validates the server, otherwise none to use the Java trust store. A file you set must be readable by OSE_USER
ZMBKP_SKIP_CRON_CHECK N - set to Y to install where no cron daemon runs, such as an image build

A missing required value, or a ZMBKP_REST_PORT that is not a port from 1 to 65535, stops the installer with exit code 5 before anything is built or deployed, and so does a missing cron daemon, with exit code 3, because the scheduled backups would never run without one. The interactive installer warns about a missing daemon and asks whether to continue. Passwords are masked in the printed summary. --unattended combines with --force-upgrade and with --remove; an unattended --remove never truncates the metadata database and always keeps the backup storage. An unattended --force-upgrade adds any missing setting with the value an unattended install would use, and chooses a CA certificate the same way. A password with no value to fall back on, such as the REST password without ZMBKP_REST_ADMIN_PASS, stays unset with a warning.

# ZMBKP_REST_ADMIN_PASS='admin-password' ./install-java.sh --unattended

The lucascbeyeler.zimbra Ansible role uses this mode to install Zmbackup alongside Zimbra - set zimbra_zmbackup: y there.

Usage

Run zmbackup -h or zmbackup --help for the full, current option list; a summary of the top-level commands:

$ zmbackup --help
Usage: zmbackup [-hv] [--config=<configFile>]
                 [COMMAND]
Commands:
  backup     Back up accounts, aliases, distribution lists, LDAP entries, or domains.
  restore    Restore a backup session (LDAP + mailbox).
  list       List stored backup sessions.
  delete     Delete a stored backup session.
  housekeep  Prune old and empty backup sessions.
  accounts   List Zimbra accounts from LDAP (diagnostic; not a backup operation).
  migrate    Import a bash-tool sessions.txt, and/or normalize a pre-existing bash-tool
             SESSION_TYPE=SQLITE3 database, into the SQLite metadata store.
  truncate   Empty the backup metadata database. TEST/DEV USE ONLY.

backup has one subcommand per object type - full, incremental, mailbox, ldap, alias, distlist, signature, domain - each taking a repeatable --account (or, for domain, --domain) to restrict which objects are backed up, and (except domain) a --domain to restrict discovery to one Zimbra domain. With no --account/--domain, every discovered object is backed up. serverconfig is the exception: it takes no --account/--domain at all (there's nothing to discover) and always archives whatever serverConfig.paths is configured to.

$ zmbackup backup full
$ zmbackup backup full --account user@domain.com
$ zmbackup backup mailbox --domain domain.com
$ zmbackup backup ldap --account user@domain.com
$ zmbackup backup incremental
$ zmbackup backup serverconfig

restore takes --session <sessionId> (check available IDs with zmbackup list first) and, optionally, a repeatable --account/--domain to restrict which objects are restored; with no subcommand it restores both LDAP and mailbox content, or use the ldap, domain, mailbox, or serverconfig subcommands to restore one kind of content on its own. --into <account> restores a mailbox into a different destination account (requires exactly one --account). restore serverconfig takes only --session and writes files back to their original host paths.

$ zmbackup list
$ zmbackup restore --session full-20170621201603
$ zmbackup restore mailbox --session full-20170621201603 --account user@domain.com
$ zmbackup restore mailbox --session full-20170621201603 --account origin@domain.com --into dest@domain.com
$ zmbackup restore serverconfig --session serverconfig-20170621201603

delete removes a stored session; housekeep prunes old and empty sessions:

$ zmbackup delete --session full-20170621201603
$ zmbackup housekeep

See Installation above for migrate and truncate.

Scheduling backups

The installer script automatically creates a cron config file in /etc/cron.d/zmbackup. You can customize backup routines editing that file.

Want to contribute to the project?

Contributions are welcome - please open an issue or pull request.

License

GNU GPL v3.0

View official GNU site http://www.gnu.org/licenses/gpl.html.

Author Information

About

A reliable software written in Java to help you in your daily task to backup and restore mails and accounts from Zimbra Open Source Email Platform.

Topics

Resources

Stars

153 stars

Watchers

21 watching

Forks

Releases

Sponsor this project

Packages

Used by

Contributors

Languages