Skip to content

Backup and Restore

Backups protect your projects, analysis data, and user history. Although the Enterprise server itself should have multiple precautions to protect against data corruption or loss, we still recommend taking regular backups just in case. At minimum, customers should take a backup before updating.

Backup Contents

When you back up your Enterprise server with the backup command included in manage_server, it should include the following data:

  • Databases: Enterprise backend database; WARP database (if enabled)
  • Objects: Project files, snapshots, caches, chat history, and uploads from the object store.
  • Metadata: Extra metadata such as backup format version and server version for compatibility checks

Essentially: All data from within containers should be included.

There are, however, a few things that are not included in this backup. These include:

  • A copy of your config.env
  • Your license.dat and license-bundle.b64 files
  • Your docker-compose.override.yml file
  • All keys/* and secrets/*
  • Any host TLS certificates

Essentially: All data from outside the containers is not included. These files should be backed up separately if you want to preserve custom keys or settings.

When restoring, you will be copying all of your data into a new installation. As a result, it shouldn't be necessary to back up things like secrets as we'll generate and use new passwords and keys for things like database and object store access. But, you may want a copy of your config.env and docker-compose.override.yml at minimum.

Creating a Backup

Note

Ensure your server is actively running before taking a backup. Data is streamed from live services, which can't be done if the containers for those services are stopped.

Creating a backup should hopefully be as simple as running:

./manage_server backup

This should produce a bn_enterprise_backup.tar in the current directory.

In some cases, your host filesystem won't have enough space to hold the backup. In these situations, you may want to mount a separate disk and use one of the following options to stream the backup to the other device:

# Save the backup to a different file directly
./manage_server backup --file /mnt/other_drive/bn_enterprise_backup.tar

# Stream to a different file
./manage_server backup --stdout > /mnt/other_drive/bn_enterprise_backup.tar

Before the backup is attempted, we perform a number of readiness checks to ensure there are no active database migrations and that connectivity between services is good. In rare cases, these readiness checks could get in the way of creating a backup. While not recommended, the following options exist to help in these scenarios:

  • --wait-timeout can be used to control the time the backup command will wait for each step to complete
  • --no-wait will skip these checks entirely

After creating a backup, list its contents to ensure expected files exist:

tar -tf bn_enterprise_backup.tar | head

You should expect the following, at minimum:

  • BACKUP_FORMAT_VERSION: File containing the format version for this backup
  • BACKUP_SERVER_VERSION: File containing the server version that was active when the backup was performed
  • db.backup: The Enterprise database backup
  • media_backup/*: All objects that were backed up from the object store

Warning

The backup archive is all-or-nothing! If it’s interrupted, create a fresh backup. Always verify the resulting file size is non-zero and copy completes cleanly.

Restoring From a Backup

Danger

Restoring erases current server data and replaces it with the backup contents. Ensure you’re restoring to the correct environment and have a fresh backup of the current state if needed.

Note

Ensure your server is actively running before restoring a backup. Data is streamed from the backup directly to live services, which can't be done if the containers for those services are stopped.

Before restoring a backup, consider the following regarding version compatibility:

  • Restoring a backup from older → newer servers are supported.
  • Restoring a backup from newer → older servers are blocked. Upgrade the server first, then restore.
  • Restoring a backup format v0 (v1.0.43 or earlier) file is no longer supported. Contact support (enterprise@vector35.com) if needed.

Restoring a backup should hopefully be as simple as running:

./manage_server restore --file /path/to/bn_enterprise_backup.tar

Once the restore has completed, you must restart the entire service stack. While we do run migrations as part of the restore process, we can't guarantee the server is entirely in the correct state until we re-run service initialization. So, you will need to:

./manage_server stop
./manage_server start --detach

Before the restore is attempted, we perform a number of readiness checks to ensure there are no active database migrations and that connectivity between services is good. In rare cases, these readiness checks could get in the way of restoring a backup. While not recommended, the following options exist to help in these scenarios:

  • --wait-timeout can be used to control the time the restore command will wait for each step to complete
  • --no-wait will skip these checks entirely