> Complete content index: https://sirannon.sondelali.com/llms.txt

# Backup destinations

> Send a copy of an open database to storage you supply, in fixed-size pieces, watch it move, and turn those pieces back into a database file.

URL: https://sirannon.sondelali.com/docs/backup-destinations
Section: Backups
Version: 0.3 (@delali/sirannon-db@0.3.0, latest)

Sirannon writes a backup through a **destination** you supply. A destination is an object with three methods that move bytes, and Sirannon calls them with fixed-size pieces of the copy. Object storage, another machine, and a directory on the same disk all fit behind those three methods, which keeps your credentials and your retry policy inside your own code.

This page builds one file, `copy-to-storage.ts`, step by step. You write a destination backed by a folder, copy a database into it, watch the copy move, and read the pieces back into a database you can query. The [backups page](/docs/backups) covers the copy to a local file that this one builds on, and the [chains page](/docs/backup-chains) covers repeating the copy so that each run sends only what changed.

```bash
rm -rf data && mkdir -p data
```

## What a destination has to do

A destination provides three operations. It stores one piece, it gives one stored piece back, and it lists the pieces it holds under a given name. Sirannon numbers the pieces from zero, and every piece holds `pieceBytes` bytes except the last.

```ts
interface BackupDestination {
  writePiece(name: string, index: number, bytes: Uint8Array): Promise<void>
  readPiece(name: string, index: number): Promise<Uint8Array>
  listPieces(name: string): Promise<{ index: number; byteLength: number }[]>
  writePieceIfAbsent?(name: string, index: number, bytes: Uint8Array): Promise<boolean>
}
```

Four rules follow from how SQLite writes a database, and a destination that breaks one of them produces a backup no restore can use.

- Accept the pieces in any order, because SQLite writes page one last.
- Let a second write to the same name and index replace the piece already stored, because a run resumed after an interruption repeats its last write.
- Return from `readPiece` whatever the most recent `writePiece` to that name and index stored, because Sirannon reads a record back to confirm that no other node replaced it.
- Hold more than one name, because a chain stores its full copy, each change piece, and its own list under a name of its own.

`writePieceIfAbsent` is optional. It stores a piece only where that name and index hold none and reports whether this call is the one that stored it, which is how two nodes of a replication group claim separate places in the same list. The [cluster page](/docs/backups-in-a-cluster) covers when you need it.

## Copy a database into your own storage

The destination below writes each piece as its own file in `./data/pieces`, named after the backup and its index. A destination backed by an object-storage client implements the same three methods with the same signatures. `db.backupTo` then copies the database into it.

```ts title="copy-to-storage.ts"

const pieceDir = './data/pieces'
const pieceFile = (name: string, index: number): string => join(pieceDir, `${name}.${String(index).padStart(6, '0')}`)

const folder: BackupDestination = {
  async writePiece(name, index, bytes) {
    await mkdir(pieceDir, { recursive: true })
    await writeFile(pieceFile(name, index), bytes)
  },
  async readPiece(name, index) {
    return new Uint8Array(await readFile(pieceFile(name, index)))
  },
  async listPieces(name) {
    const entries = await readdir(pieceDir).catch(() => [])
    const pieces: BackupPiece[] = []
    for (const entry of entries) {
      if (!entry.startsWith(`${name}.`)) continue
      const { size } = await stat(join(pieceDir, entry))
      pieces.push({ index: Number(entry.slice(name.length + 1)), byteLength: size })
    }
    return pieces
  },
}

const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })

const db = await sirannon.open('orders', './data/orders.db')

await db.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, customer TEXT NOT NULL, total REAL NOT NULL)')

const ORDER_COUNT = 5000
await db.executeBatch(
  'INSERT INTO orders (customer, total) VALUES (?, ?)',
  Array.from({ length: ORDER_COUNT }, (_, index) => [`Customer ${index + 1}`, (index % 400) + 0.5]),
)

const run = await db.backupTo({
  destination: folder,
  name: 'orders-full.db',
  pieceBytes: 65_536,
})

await sirannon.shutdown()

console.log(
  JSON.stringify(
    {
      copy: {
        kind: run.kind,
        destinationName: run.destinationName,
        pageCount: run.pageCount,
        pageSize: run.pageSize,
        bytesWritten: run.bytesWritten,
        pieceCount: run.pieceCount,
        pieceBytes: run.pieceBytes,
        restarts: run.restarts,
      },
    },
    null,
    2,
  ),
)
```

```json result
{
  "copy": {
    "kind": "full",
    "destinationName": "orders-full.db",
    "pageCount": 39,
    "pageSize": 4096,
    "bytesWritten": 159744,
    "pieceCount": 3,
    "pieceBytes": 65536,
    "restarts": 0
  }
}
```

Five thousand rows come to 39 pages of 4,096 bytes, which is 159,744 bytes, and a piece size of 64 KiB divides that into three pieces. A larger `ORDER_COUNT` raises the page count, the byte total, and the piece count together, while a different `pieceBytes` moves the piece count alone.

The report states more fields than the panel prints. `runId` names the run; `startedAt`, `finishedAt`, `durationMs`, `copyMs`, and `transferMs` time it; `fingerprint` holds the SHA-256 of what it wrote; `route` names which of the two routes the run took; and `chainId` names the chain this copy begins. A copy taken this way begins a chain of one, and the [chains page](/docs/backup-chains) covers how that chain grows.

`destinationName` matters when you come to read the backup back, because a run writes that one name and no other. The journal SQLite opens beside a copy, meanwhile, stays on local disk and never reaches your destination.

## Watch a large copy as it moves

A full copy of a large database to remote storage can take long enough to need a progress display. `onProgress` reports once per copy step while SQLite moves pages, then once per piece while those pieces go out, and `phase` tells the two apart.

Keep the latest report of each phase and print both at the end.

```ts title="copy-to-storage.ts"

import { type BackupDestination, type BackupPiece, Sirannon } from '@delali/sirannon-db' // [!code --]
import { type BackupDestination, type BackupPiece, type BackupProgress, Sirannon } from '@delali/sirannon-db' // [!code ++]

const pieceDir = './data/pieces'
const pieceFile = (name: string, index: number): string => join(pieceDir, `${name}.${String(index).padStart(6, '0')}`)

const folder: BackupDestination = {
  async writePiece(name, index, bytes) {
    await mkdir(pieceDir, { recursive: true })
    await writeFile(pieceFile(name, index), bytes)
  },
  async readPiece(name, index) {
    return new Uint8Array(await readFile(pieceFile(name, index)))
  },
  async listPieces(name) {
    const entries = await readdir(pieceDir).catch(() => [])
    const pieces: BackupPiece[] = []
    for (const entry of entries) {
      if (!entry.startsWith(`${name}.`)) continue
      const { size } = await stat(join(pieceDir, entry))
      pieces.push({ index: Number(entry.slice(name.length + 1)), byteLength: size })
    }
    return pieces
  },
}

const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })

const db = await sirannon.open('orders', './data/orders.db')

await db.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, customer TEXT NOT NULL, total REAL NOT NULL)')

const ORDER_COUNT = 5000
await db.executeBatch(
  'INSERT INTO orders (customer, total) VALUES (?, ?)',
  Array.from({ length: ORDER_COUNT }, (_, index) => [`Customer ${index + 1}`, (index % 400) + 0.5]),
)

// [!code ++:2]
const latest = new Map<BackupProgress['phase'], BackupProgress>()

const run = await db.backupTo({
  destination: folder,
  name: 'orders-full.db',
  pieceBytes: 65_536,
  onProgress: (progress) => latest.set(progress.phase, progress), // [!code ++]
})

await sirannon.shutdown()

console.log(
  JSON.stringify(
    {
      copy: {
        kind: run.kind,
        destinationName: run.destinationName,
        pageCount: run.pageCount,
        pageSize: run.pageSize,
        bytesWritten: run.bytesWritten,
        pieceCount: run.pieceCount,
        pieceBytes: run.pieceBytes,
        restarts: run.restarts,
      },
      // [!code ++:6]
      progress: {
        totalPages: latest.get('copy')?.totalPages,
        remainingPages: latest.get('copy')?.remainingPages,
        piecesWritten: latest.get('transfer')?.piecesWritten,
        bytesWritten: latest.get('transfer')?.bytesWritten,
      },
    },
    null,
    2,
  ),
)
```

```json result
{
  "copy": {
    "kind": "full",
    "destinationName": "orders-full.db",
    "pageCount": 39,
    "pageSize": 4096,
    "bytesWritten": 159744,
    "pieceCount": 3,
    "pieceBytes": 65536,
    "restarts": 0
  },
  "progress": {
    "totalPages": 39,
    "remainingPages": 0,
    "piecesWritten": 3,
    "bytesWritten": 159744
  }
}
```

`totalPages` and `remainingPages` drive a copy bar, while `piecesWritten` against the eventual `pieceCount` drives a transfer bar. On the staged route every transfer report states `remainingPages: 0`, because the copy finishes before any piece goes out; on the streamed route the two phases interleave, so a transfer report states however many pages remain. How many reports fall between the first and the last depends on the route the bytes take and on how fast the disk answers, so read the counters for your display and treat the number of events as unspecified.

`restarts` rising during a copy is worth an alert, because another connection is writing to the source file and SQLite has sent the copy back to page one.

## Turn the pieces back into a database

The pieces are a database file cut into numbered blocks, so putting them back together needs the index of each piece and nothing else. `assembleFromDestination` fetches them one at a time, writes each at `index * pieceBytes`, and checks the result against the report of the run that wrote them.

```ts title="copy-to-storage.ts"

import { assembleFromDestination } from '@delali/sirannon-db/backup' // [!code ++]

const pieceDir = './data/pieces'
const pieceFile = (name: string, index: number): string => join(pieceDir, `${name}.${String(index).padStart(6, '0')}`)

const folder: BackupDestination = {
  async writePiece(name, index, bytes) {
    await mkdir(pieceDir, { recursive: true })
    await writeFile(pieceFile(name, index), bytes)
  },
  async readPiece(name, index) {
    return new Uint8Array(await readFile(pieceFile(name, index)))
  },
  async listPieces(name) {
    const entries = await readdir(pieceDir).catch(() => [])
    const pieces: BackupPiece[] = []
    for (const entry of entries) {
      if (!entry.startsWith(`${name}.`)) continue
      const { size } = await stat(join(pieceDir, entry))
      pieces.push({ index: Number(entry.slice(name.length + 1)), byteLength: size })
    }
    return pieces
  },
}

const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })

const db = await sirannon.open('orders', './data/orders.db')

await db.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, customer TEXT NOT NULL, total REAL NOT NULL)')

const ORDER_COUNT = 5000
await db.executeBatch(
  'INSERT INTO orders (customer, total) VALUES (?, ?)',
  Array.from({ length: ORDER_COUNT }, (_, index) => [`Customer ${index + 1}`, (index % 400) + 0.5]),
)

const latest = new Map<BackupProgress['phase'], BackupProgress>()

const run = await db.backupTo({
  destination: folder,
  name: 'orders-full.db',
  pieceBytes: 65_536,
  onProgress: (progress) => latest.set(progress.phase, progress),
})

await sirannon.shutdown()
// [!code ++:8]

const assembled = await assembleFromDestination(folder, run, './data/rebuilt.db')

const check = new Sirannon({ driver: betterSqlite3() })
const rebuilt = await check.open('rebuilt', './data/rebuilt.db')
const counted = await rebuilt.queryOne<{ orders: number }>('SELECT COUNT(*) AS orders FROM orders')

await check.shutdown()

console.log(
  JSON.stringify(
    {
      copy: {
        kind: run.kind,
        destinationName: run.destinationName,
        pageCount: run.pageCount,
        pageSize: run.pageSize,
        bytesWritten: run.bytesWritten,
        pieceCount: run.pieceCount,
        pieceBytes: run.pieceBytes,
        restarts: run.restarts,
      },
      progress: {
        totalPages: latest.get('copy')?.totalPages,
        remainingPages: latest.get('copy')?.remainingPages,
        piecesWritten: latest.get('transfer')?.piecesWritten,
        bytesWritten: latest.get('transfer')?.bytesWritten,
      },
      // [!code ++:6]
      rebuilt: {
        pieceCount: assembled.pieceCount,
        bytesWritten: assembled.bytesWritten,
        fingerprintMatchesTheRun: assembled.fingerprint === run.fingerprint,
        orders: counted?.orders,
      },
    },
    null,
    2,
  ),
)
```

```json result
{
  "copy": {
    "kind": "full",
    "destinationName": "orders-full.db",
    "pageCount": 39,
    "pageSize": 4096,
    "bytesWritten": 159744,
    "pieceCount": 3,
    "pieceBytes": 65536,
    "restarts": 0
  },
  "progress": {
    "totalPages": 39,
    "remainingPages": 0,
    "piecesWritten": 3,
    "bytesWritten": 159744
  },
  "rebuilt": {
    "pieceCount": 3,
    "bytesWritten": 159744,
    "fingerprintMatchesTheRun": true,
    "orders": 5000
  }
}
```

All 5,000 rows come back, and the digest of the assembled file equals the digest the run recorded. `assembleFromDestination` checks the listing before it opens the local file, so Sirannon refuses a destination that is missing a piece while the path you named still holds whatever it held. Once that file is open, Sirannon removes it after any failure, because a database missing its middle would otherwise stay on disk looking finished.

Four conditions fail the assembly with `BACKUP_DESTINATION_ERROR`: a missing index, a piece past the run's `pieceCount`, a byte total other than `bytesWritten`, and a digest other than the recorded one. The third and fourth catch the case where a later, smaller run reused the same name and left the earlier run's trailing pieces in place.

Use `assembleFromDestination` for a copy you took with `backupTo`. A chain of a full copy plus its change pieces needs `restoreBackup`, which the [restore page](/docs/backup-restore) covers.

## Which route the bytes take

Sirannon takes one of two routes to your destination, and the report names the one it took in `route`.

The **streamed** route passes each piece to your destination as SQLite writes it, so the run needs no local disk at all. It works through a compiled SQLite extension that registers a virtual file system named `sirannon`, and Sirannon publishes that extension as one small package per platform. Installing the library fetches the package matching your `os` and `cpu`, because the library lists all six as optional dependencies pinned to its own version, so the install adds it for you.

The **staged** route writes one local file, then sends that file on in pieces. It needs free local disk the size of the backup, and `localDiskRequired` reads `'equal-to-backup'` when Sirannon would take it.

Which route you get depends on the runtime. Node's own SQLite parses URI file names from version 23 upwards, and Sirannon names its virtual file system through a URI parameter, so the `node` driver streams on Node 23 and later and stages on Node 22. better-sqlite3 parses URI file names only where the operator sets `SQLITE_USE_URI=1` before the module loads.

```bash
SQLITE_USE_URI=1 npx tsx check-route.ts
```

```ts title="check-route.ts"

const sirannon = new Sirannon({ driver: betterSqlite3() })
const db = await sirannon.open('orders', './data/orders.db')

const { streamedCopy, stagedCopy, localDiskRequired } = db.backupCapabilities()

console.log(JSON.stringify({ streamedCopy, stagedCopy, localDiskRequired }, null, 2))

await sirannon.shutdown()
```

```json result
{
  "streamedCopy": true,
  "stagedCopy": true,
  "localDiskRequired": "none"
}
```

The same file without that variable reports `streamedCopy: false` and `localDiskRequired: 'equal-to-backup'`, which is the answer the [backups page](/docs/backups#what-your-runtime-can-copy) prints. The variable turns URI parsing on for every file name that process opens, so SQLite would then read a database path containing a question mark as a URI. Check the paths your application opens before you set it.

On a platform Sirannon publishes no binary for, compile the extension yourself and name it on the driver.

```ts
const driver = betterSqlite3({ vfsExtensionPath: '/opt/sirannon/sirannonvfs.so' })
```

The `node` driver takes the same option. A named path replaces the binary the install fetched, so naming one also pins a single build across a fleet.

The streamed route brings two constraints. It gives SQLite whole 512-byte blocks, so `pieceBytes` has to be a multiple of 512 and any other size fails with `BACKUP_ERROR`. Its fingerprint reads every piece back from your destination, because the run never holds the whole file, so a copy to remote storage does that read over the network. Set `fingerprint: false` where that read is more work than the check is worth.

## Options

| Option | Default | What it does |
| --- | --- | --- |
| `destination` | required | Where the pieces go and where a restore reads them from. |
| `name` | `backup-{ISO timestamp}.db` | Name the pieces are stored under. |
| `chainId` | an identifier the run mints | The chain this copy begins. |
| `pieceBytes` | `16777216` | Bytes one whole piece holds. A streamed copy needs a multiple of 512. |
| `fingerprint` | `true` | Whether the run folds a SHA-256 over what it wrote. |
| `pagesPerStep` | `256` | Pages SQLite moves in one step. |
| `restartLimit` | `3` | Restarts the copy absorbs before it fails with `BACKUP_RESTARTED`. |
| `noProgressStepLimit` | `256` | Steps the copy may take without reaching a page it had not already copied. |
| `stallTimeoutMs` | `30000` | Milliseconds the copy may move no pages before it fails with `BACKUP_STALLED`. |
| `destinationTimeoutMs` | `600000` | Milliseconds one call to your destination may take. Zero leaves the calls unbounded. |
| `stagingDir` | the host temporary directory | Directory the staged route writes its local file in. |
| `onProgress` | none | Called at step resolution during the copy and once per piece during the transfer. |

## When a destination call hangs

A destination that never answers would hold a backup open for ever, because Sirannon counts a pending call as work in progress. `destinationTimeoutMs` bounds every individual call, and a call that passes it fails the run with `BACKUP_DESTINATION_ERROR`. Ten minutes is the default, which suits a large piece over a slow link, and zero removes the bound for a destination you trust to answer or throw on its own.

The same code covers a destination that refuses a call outright, and the message names the piece and the backup name it was stored under, so it points at one object in your storage. Over the [server](/docs/server), `BACKUP_DESTINATION_ERROR` answers `502`, which separates your storage failing from Sirannon failing.