Backups

Restoring a database

Select a restore point, rebuild the database, and find backup files that your retention policy permits you to delete.

restoreBackup() rebuilds a database from a full copy and the change files captured after it. Choose a timestamp and a new destination path so that you can inspect the recovered database before replacing an existing copy.

Sirannon fetches one stored piece at a time, which bounds memory use independently of the database's size. The chain guide explains how Sirannon captures the files that a restore needs.

Ask what a restore would read

db.backupRestorePlan(moment) selects the newest full copy that finished at or before the requested timestamp. It then selects that chain's change files through the same timestamp, in sequence order. The call reads the chain records without downloading the backup files.

The plan's restoresTo is the last selected capture's timestamp, or the full copy's completion timestamp when there are no later captures. A restore stops at a capture boundary, so restoresTo can be earlier than the requested timestamp. A failed or delayed capture can make that gap longer than the configured capture interval.

The example below creates two captures and restores the first. Its in-memory destination exists only to demonstrate recovery within one process; use a durable destination for backups that must outlast a process.

Save this complete helper as memory-destination.ts beside restore.ts.

memory-destination.ts
import type { BackupDestination, BackupPiece } from '@delali/sirannon-db'
 
const stored = new Map<string, Map<number, Uint8Array>>()
 
const piecesFor = (name: string): Map<number, Uint8Array> => {
  const existing = stored.get(name)
  if (existing) return existing
  const pieces = new Map<number, Uint8Array>()
  stored.set(name, pieces)
  return pieces
}
 
export const memory: BackupDestination = {
  async writePiece(name, index, bytes): Promise<void> {
    piecesFor(name).set(index, Uint8Array.from(bytes))
  },
  async writePieceIfAbsent(name, index, bytes): Promise<boolean> {
    const pieces = piecesFor(name)
    if (pieces.has(index)) return false
    pieces.set(index, Uint8Array.from(bytes))
    return true
  },
  async readPiece(name, index): Promise<Uint8Array> {
    const bytes = stored.get(name)?.get(index)
    if (!bytes) throw new Error(`Missing piece ${index} of '${name}'`)
    return Uint8Array.from(bytes)
  },
  async listPieces(name): Promise<BackupPiece[]> {
    return Array.from(stored.get(name) ?? [], ([index, bytes]): BackupPiece => ({
      index,
      byteLength: bytes.byteLength,
    }))
  },
}

Rebuild the database

Save this complete file as restore.ts. Sirannon takes a first backup turn when the database opens, even with intervalMs: 0; zero disables the recurring timer. Explicit captures then save the three orders and the two later orders separately.

restore.ts
import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { setTimeout } from 'node:timers/promises'
import { Sirannon } from '@delali/sirannon-db'
import { restoreBackup } from '@delali/sirannon-db/backup'
import { betterSqlite3 } from '@delali/sirannon-db/driver/better-sqlite3'
import { memory } from './memory-destination.ts'
 
const directory = await mkdtemp(join(tmpdir(), 'sirannon-restore-'))
const source = new Sirannon({ driver: betterSqlite3() })
const check = new Sirannon({ driver: betterSqlite3() })
 
try {
  const db = await source.open('orders', join(directory, 'orders.db'), {
    backups: {
      destination: memory,
      intervalMs: 0,
      onError: (error): void => console.error('Backup failed', error),
    },
  })
  await db.execute('CREATE TABLE orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')
  await db.executeBatch('INSERT INTO orders VALUES (?, ?)', [[1, 12850], [2, 6400], [3, 24999]])
  await db.captureBackupChanges()
  const first = (await db.backupChain())[0]?.changes[0]
  if (!first) throw new Error('The first capture is missing')
  while (Date.now() <= first.capturedAt) await setTimeout(1)
  await db.executeBatch('INSERT INTO orders VALUES (?, ?)', [[4, 1999], [5, 31000]])
  await db.captureBackupChanges()
  const plan = await db.backupRestorePlan(first.capturedAt)
  const location = db.backupLocation()
  await source.shutdown()
 
  const recoveredPath = join(directory, 'recovered.db')
  const report = await restoreBackup({
    destination: location.destination,
    destinationTimeoutMs: location.destinationTimeoutMs,
    chainName: location.chainName,
    driver: betterSqlite3(),
    destPath: recoveredPath,
    moment: first.capturedAt,
  })
  const recovered = await check.open('recovered', recoveredPath)
  const counted = await recovered.queryOne<{ orders: number }>('SELECT COUNT(*) AS orders FROM orders')
  console.log(JSON.stringify({
    plannedChanges: plan.changes.length,
    changesApplied: report.changesApplied,
    orders: counted?.orders,
  }, (key: string, value: unknown): unknown =>
    typeof value === 'number' ? Number(value.toFixed(2)) : value, 2))
} finally {
  await check.shutdown()
  await source.shutdown()
  await rm(directory, { recursive: true, force: true })
}

The clock check separates the captures' millisecond timestamps so that the first restore point excludes the second capture. restoreBackup also checks each file's byte count and recorded fingerprint before accepting it.

The report includes chainId, baseName, destPath, and restoresTo. It also includes changesApplied, framesApplied, batchCount, pieceCount, and bytesFetched, plus startedAt, finishedAt, and durationMs.

A missing change sequence fails with BACKUP_CHAIN_BROKEN. A missing stored piece, incorrect byte count, or fingerprint mismatch fails with BACKUP_DESTINATION_ERROR. A requested timestamp before every available full copy also fails with BACKUP_CHAIN_BROKEN.

For an encrypted chain, enable encryption on the driver and pass encryption: { key: loadKey } to restoreBackup. Define loadKey as described in the encryption guide. Select the key for the chosen chain's base.keyId, including when a later chain uses a different key. The recovered file retains that chain's encryption.

Where a restore writes, and what it leaves behind

Sirannon assembles the database at {destPath}.restoring and renames it to destPath after applying the last batch. Use a separate destination for each concurrent restore, because the temporary path is deterministic.

replaceExisting defaults to false, so an existing destination causes BACKUP_ERROR. Close every connection to the destination before passing replaceExisting: true.

Before replacing a database, Sirannon tries to checkpoint its write-ahead log into the old file. If that succeeds, the old database remains complete until the rename. If Sirannon cannot fold the log into that file, it deletes the old database and its log before the rename. A crash during that final replacement can therefore leave the path empty.

The server restore route closes its database before restoring it. Its operator permission and configuration gate remain necessary even when ordinary SQL requests are enabled.

OptionDefaultBehaviour
destinationRequired.Sirannon fetches backup pieces and records from this object.
driverRequired.Sirannon opens the rebuilt file through this driver.
destPathRequired.Sirannon writes the recovered database to this path.
momentThe current timestamp.Sirannon selects captures at or before these epoch milliseconds.
chainNamesirannon-backup-chainSirannon lists chains under this destination name.
replaceExistingfalseSirannon can replace an existing destination when this is true.
batchSize16Sirannon replays between 1 and 4096 change files before each checkpoint.
destinationTimeoutMs600000Sirannon limits each destination call to this many milliseconds; zero removes the deadline.
onProgressAbsent.Sirannon reports progress after pieces and batches, with a full-copy or changes phase.
encryptionAbsent.Sirannon opens an encrypted chain with this key provider.

Allow free disk for the recovered database, one stored piece, and one batch of change files. When replacing an existing database, include the old database's size as well, because both files coexist before the rename. Lower batchSize to reduce the temporary log's size; larger batches require fewer checkpoints.

Find the backups nobody needs

db.backupPiecesSafeToDelete() returns records that no restore can use. These include change files whose chain lacks a full copy and change files after a sequence gap. Sirannon lists the records without deleting anything.

Pass { restorableFrom: timestamp } to apply a retention window. Sirannon also returns chains older than the newest full copy that finished at or before that timestamp. Keep that boundary full copy and its later change files so that a restore can reach the start of the retention window.

Each BackupChainRecord includes the storage name and pieceCount. Delete every numbered piece of each returned record through your storage client. Retain encryption keys for every chain that remains available for recovery.