Docs / Backup and restore

Backup and restore

Everything SkelaDB needs to recover a database lives in your bucket. A restore always creates a new branch at the point you choose; it never overwrites the branch you restore from.

How branches are protected

Object storage mode. Every commit is acknowledged only after two of the project’s three WAL nodes have it on disk, and each WAL node archives WAL to the bucket before it is trimmed; by default a commit also waits until the WAL is archived. Pages are written to the bucket as immutable layers. Point-in-time restore reaches back 7 days by default on the page server, so any second in the last week can be opened as a branch.

Dedicated disk mode. Each branch archives its WAL continuously (a segment is closed at least every 60 seconds) and takes a base backup every backupIntervalSeconds (daily by default), keeping backupRetainCount (7) full backups and the WAL after the oldest kept one. Before a branch suspends, its latest WAL is archived. A lost disk loses at most the last unarchived seconds of commits.

You can change a branch’s backup interval and retention in the console under Compute or with the API’s PATCH /v1/branches/{name} (backupIntervalSeconds, backupRetainCount, at least one full backup).

Base backups run inside the branch’s compute, so they share its CPU and memory. Large databases get a longer deadline automatically (database size at 20 MiB/s, up to 24 hours). The kubelet’s streaming connection idle timeout (four hours by default) must exceed your longest backup.

Restore a branch

In the console, open Restore, choose the branch (deleted branches are listed while their backups are retained), pick a time or one of the listed restore points, and name the new branch. With the API:

POST /v1/branches
{"name": "acme-restore-1200", "project": "acme",
 "restore": {"from": "acme-main", "at": "2026-10-01T12:00:00Z"}}
  • A running source is asked to archive its latest WAL first, so any time up to the request is reachable.
  • A suspended source is reachable up to the moment it suspended; a later time is refused with the last archived instant.
  • A restore from a raw bucket prefix ("restore": {"prefix": "...", "at": ...}) that asks for a time past the archived WAL fails with the branch condition RestoreTargetUnreachable. Delete that branch and choose an earlier time.

Check your application against the restored branch, then point it there, or copy the rows you need back with pg_dump/psql over a session connection.

Deleted branches and projects

Deleting a branch keeps its backups unless you ask to purge them (DELETE /v1/branches/{name}?purge_backups=true), so it can still be restored. Deleting a project in the console keeps its backups for SKELA_CLOUD_DELETION_RETENTION_DAYS (7 by default) and then purges them. If your bucket has versioning, object lock or replication, the bytes stay until those retention periods end.

Protect the bucket

The bucket is your backup. For production:

  • Turn on object lock (S3 compliance mode, GCS retention policy, Azure immutability) with bucket.sh and SKELA_BUCKET_OBJECT_LOCK_DAYS, and deny the permission to bypass it. Each project’s identity can delete within its own prefix (retention needs it); object lock is what stops a compromised workload from destroying history.
  • Replicate it to a second region with SKELA_BUCKET_REPLICA.
  • Keep the project identities’ grants limited to their own prefix.

The management database and its keys

If you run the console, its management database holds organizations, projects, members, the audit trail and sealed credentials. It is useless without the keys in SKELA_CLOUD_ENCRYPTION_KEYS.

  • Back it up with your provider’s point-in-time recovery or a scheduled pg_dump --format=custom, kept at least as long as your audit retention, and always take one before an upgrade.
  • Escrow every key, including retired keys that older backups still need, in a secret store separate from the database backups, for example a secret manager in another account or sealed copies held by two named people. No single credential should recover both a backup and its keys.
  • Record which key IDs each backup needs. Rotation re-seals values only in the live database, so an older backup may need a retired key.

Restore the management database

Restore into an isolated namespace or cluster first:

  1. Restore the database to a new instance and point a cloud-config Secret at it.

  2. Check the keys before starting anything:

    cloud/render.sh check-keys | kubectl apply -f -
    kubectl -n skela-cloud wait --for=condition=complete job/cloud-check-keys --timeout=30m
    kubectl -n skela-cloud logs job/cloud-check-keys

    For each sealed column it prints how many values exist and how many the configured keys cannot open, never the values, and fails if any value is unreadable. Find the missing key before you continue: replacing keys makes those values permanently unreadable.

  3. Run the migration for the release you run (cloud/render.sh apply), then verify sign-in, permissions and that projects reconcile with their regions before moving traffic to the restored console.

Platform secrets to keep

Keep encrypted copies, outside the cluster, of the skela namespace Secrets: skela-api (API, proxy and metrics tokens), skela-license, skela-project-tokens, skela-proxy-peer, each skela-backup-<project>, and your issuer’s CA. For object storage projects also keep the storage pool ConfigMaps (skela-pool-<project>), their timeline ConfigMaps and signing Secrets: losing them must never lead to new identities being allocated over retained data. disaster-recovery.md covers whole-cluster recovery.