> Complete content index: https://sirannon.sondelali.com/llms.txt

# Distributed replication

> Replicate a SQLite database across nodes over gRPC, bootstrap new replicas automatically, choose write concerns, and fail over through etcd.

URL: https://sirannon.sondelali.com/docs/distributed-replication
Section: Distributed
Version: 0.3 (@delali/sirannon-db@0.3.0, latest)

Sirannon replicates a SQLite database across multiple nodes with change propagation, new-node bootstrapping, write concerns, and coordinator-backed failover. Each node owns its own database file, and Sirannon moves checksummed change batches between them, which keeps every node independent of a shared network filesystem.

The production path is primary-replica. One primary accepts writes, replicas serve reads and can forward writes, and coordinator mode manages authority once you enable failover. Every change includes a <Tooltip tip="A Hybrid Logical Clock combines a node's physical clock with a logical counter, which gives events across nodes timestamps that respect cause and effect even when the clocks disagree.">Hybrid Logical Clock</Tooltip> timestamp, which lets every node order the changes by cause and effect. This page builds two files, `primary.ts` and `replica.ts`, and each code block holds one whole file.

## Create the certificates the nodes present to each other

Replication over gRPC uses mutual TLS, so each node needs its own key and certificate plus the certificate authority that signed both of them. Generate them once with `openssl`, and keep the keys out of your repository. Run this from the directory you start the nodes in so that the `./certs` and `./data` paths in the code below resolve.

```bash
mkdir -p certs data

openssl req -x509 -newkey rsa:2048 -nodes -days 3650 \
  -subj "/CN=Sirannon replication CA" \
  -keyout certs/ca.key -out certs/ca.crt

for node in primary replica; do
  printf 'basicConstraints = CA:FALSE\nkeyUsage = digitalSignature, keyEncipherment\nextendedKeyUsage = serverAuth, clientAuth\nsubjectAltName = DNS:%s.example.com\n' "$node" > "certs/$node.ext"
  openssl req -newkey rsa:2048 -nodes -subj "/CN=$node.example.com" \
    -keyout "certs/$node.key" -out "certs/$node.csr"
  openssl x509 -req -in "certs/$node.csr" -days 825 \
    -CA certs/ca.crt -CAkey certs/ca.key -CAcreateserial \
    -extfile "certs/$node.ext" -out "certs/$node.crt"
done
```

Each node certificate declares both `serverAuth` and `clientAuth`, because a replication peer answers connections from other nodes and opens connections to them. The subject alternative name must match the hostname the other side dials, and `primary.example.com` here therefore matches the endpoint the replica connects to in the next section. Replace both hostnames with your own. Run `openssl verify -CAfile certs/ca.crt certs/primary.crt certs/replica.crt` to confirm the result, which prints `OK` for each file.

## Start the primary

Replication traffic runs over its own transport. Production node-to-node replication uses gRPC with <Tooltip tip="Mutual TLS means both sides of a connection present certificates, so the server verifies the client and the client verifies the server.">mutual TLS</Tooltip>, and an in-memory transport serves tests. Open the primary's database file, take a writer connection that the engine applies changes through, create a `ChangeTracker` for the ordered change log, point a gRPC transport at the local interface, and pass them to a `ReplicationEngine` in the primary role.

```ts title="primary.ts"

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])
```

The engine's second argument is a writer connection that the engine applies incoming changes through, opened here with `driver.open(dbPath)`. The `snapshotConnectionFactory` opens a read-only connection so that a joining node can stream a consistent snapshot without blocking live writes, and `changeTracker` records the ordered change log the engine sends to replicas. The two `CREATE TABLE` statements and the seed define what the primary owns, and `accounts` and `orders` are the two tables a replica copies during its first sync.

## Point a replica at the primary

The replica is its own file. It opens its own database, takes its own writer connection and change tracker, builds a transport in the same way, and starts in the replica role. Two options differ from the primary: `transportConfig.endpoints` names the primary to pull from, and `writeForwarding` lets the replica accept a write and forward it upstream. The `primary.ts` tab still holds the file from the previous step; the `replica.ts` tab is the new file you add now.

<CodeGroup defaultTab="replica.ts">

```ts title="primary.ts"

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])
```

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
})

await replicaEngine.start()
```

</CodeGroup>

## First sync

A new node copies the whole dataset before it starts applying incremental changes. The source streams the schema and table data to it in checksummed batches, then sends a manifest of row counts and primary-key hashes. The new node checks each batch against that manifest, catches up on any writes that arrived while it was copying, and starts serving reads once its replication lag falls below the configured threshold. Its sync state advances through four phases, and `phase` on `engine.status().syncState` reports which one it is in: `pending` before it picks a source, `syncing` while it copies tables, `catching-up` while it applies those buffered writes, and `ready` once it has caught up. A replica refuses reads until it reaches `ready`.

That `await replicaEngine.start()` call from the previous step begins the join. In the `replica.ts` tab, keep polling the sync state until the phase reaches `ready`:

<CodeGroup defaultTab="replica.ts">

```ts title="primary.ts"

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])
```

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
})

await replicaEngine.start()
// [!code ++:10]

for (;;) {
  const { syncState } = replicaEngine.status()
  if (syncState) {
    const { phase, completedTables, totalTables } = syncState
    console.log(`sync ${phase}: ${completedTables.length}/${totalTables} tables copied`)
    if (phase === 'ready') break
  }
  await new Promise((resolve) => setTimeout(resolve, 500))
}
```

</CodeGroup>

The loop prints the phase as each table finishes copying:

```text
sync pending: 0/0 tables copied
sync syncing: 1/2 tables copied
sync syncing: 2/2 tables copied
sync catching-up: 2/2 tables copied
sync ready: 2/2 tables copied
```

Read the same `syncState` object directly to see the detail behind those counts. During the copy it reports the source, the tables finished so far, and the total to expect. `snapshotSeq` stays `null` until the copy completes and catch-up begins, and `startedAt` records when the sync began as a Unix timestamp in milliseconds. The snapshots below come from one sample run of a replica partway through its first sync, and the `startedAt` timestamp and the `snapshotSeq` sequence therefore differ on each run. The file prints them with `console.log`, because `snapshotSeq` is a `bigint` that `JSON.stringify` cannot serialise.

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
})

await replicaEngine.start()

for (;;) {
  const { syncState } = replicaEngine.status()
  if (syncState) {
    const { phase, completedTables, totalTables } = syncState
    console.log(`sync ${phase}: ${completedTables.length}/${totalTables} tables copied`)
    if (phase === 'ready') break
  }
  await new Promise((resolve) => setTimeout(resolve, 500))
}
// [!code ++:4]

const { syncState } = replicaEngine.status()

console.log(syncState)
```

```ts result
{
  phase: 'syncing',
  sourcePeerId: 'primary-us-east-1',
  snapshotSeq: null,
  completedTables: [ 'accounts' ],
  totalTables: 2,
  startedAt: 1751500800000,
  error: null
}
```

Once catch-up finishes, the phase reaches `ready`, `snapshotSeq` holds the sequence the snapshot was taken at, and the replica is safe to read from.

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
})

await replicaEngine.start()

for (;;) {
  const { syncState } = replicaEngine.status()
  if (syncState) {
    const { phase, completedTables, totalTables } = syncState
    console.log(`sync ${phase}: ${completedTables.length}/${totalTables} tables copied`)
    if (phase === 'ready') break
  }
  await new Promise((resolve) => setTimeout(resolve, 500))
}

const { syncState } = replicaEngine.status()

console.log(syncState)
// [!code ++:4]

if (syncState?.phase === 'ready') {
  const orders = await replicaEngine.query('SELECT id, total FROM orders')
}
```

```ts result
{
  phase: 'ready',
  sourcePeerId: 'primary-us-east-1',
  snapshotSeq: 48210n,
  completedTables: [ 'accounts', 'orders' ],
  totalTables: 2,
  startedAt: 1751500800000,
  error: null
}
```

For databases too large to transfer over the network, copy the SQLite file out of band, then start the replica from a known sequence so that it skips the copy and follows changes from that point on. Pass `initialSync: false` and `resumeFromSeq` in the replica's `ReplicationEngine` config.

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
  // [!code ++:2]
  initialSync: false,
  resumeFromSeq: 48210n,
})

await replicaEngine.start()
// [!code --:18]

for (;;) {
  const { syncState } = replicaEngine.status()
  if (syncState) {
    const { phase, completedTables, totalTables } = syncState
    console.log(`sync ${phase}: ${completedTables.length}/${totalTables} tables copied`)
    if (phase === 'ready') break
  }
  await new Promise((resolve) => setTimeout(resolve, 500))
}

const { syncState } = replicaEngine.status()

console.log(syncState)

if (syncState?.phase === 'ready') {
  const orders = await replicaEngine.query('SELECT id, total FROM orders')
}
```

## Write concerns

Writes go through the engine so that it can count acknowledgements before returning. Control how many replicas must acknowledge a write with `writeConcern` on `engine.execute`.

<CodeGroup defaultTab="primary.ts">

```ts title="primary.ts"

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])
// [!code ++:6]

await engine.execute(
  'INSERT INTO orders (id, total) VALUES (?, ?)',
  [1, 4999],
  { writeConcern: { level: 'majority', timeoutMs: 5000 } },
)
```

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
// [!code --:2]
  initialSync: false,
  resumeFromSeq: 48210n,
})

await replicaEngine.start()
```

</CodeGroup>

The levels are `'local'`, `'majority'`, and `'all'`. In static mode, a write that omits `writeConcern` returns after the local commit. In coordinator mode, a write that omits it takes `'majority'`, and a successful majority write is still present after an automatic primary failover, as long as an eligible in-sync replica remains.

## Read concern

A write concern says how widely a write must be acknowledged. A read concern says how current a read must be, which settles two questions about the node answering: whether that node is still part of the group, and whether what it holds is recent enough to serve.

```ts title="primary.ts"

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])

await engine.execute(
  'INSERT INTO orders (id, total) VALUES (?, ?)',
  [1, 4999],
  { writeConcern: { level: 'majority', timeoutMs: 5000 } },
)
// [!code ++:4]

const rows = await engine.query('SELECT id, total FROM orders WHERE id = ?', [1], {
  readConcern: { level: 'linearizable' },
})
```

Each level asks the node to prove more before it answers.

| Level | What the node must prove |
| --- | --- |
| `local` | Nothing. The read returns local state, which a later failover may quarantine. |
| `majority` | The node is in the in-sync set and is neither draining nor repairing. |
| `linearizable` | The read runs on the current primary, after that node proves live authority for its term. |

Coordinator mode enforces these and treats a read that names no level as `majority`, which makes the safe answer the one a read receives without asking. Static mode ignores them entirely, because it has no coordinator to ask and cannot prove membership. That difference matters when you move a service from static to coordinator mode, because reads that always succeeded may start failing, and each failure names a read that was serving stale data before.

A read that cannot meet its level fails, and the error names the reason: `NODE_NOT_IN_SYNC` when the node has fallen out of the in-sync set, `READ_CONCERN_ERROR` when it is draining or repairing, `STALE_PRIMARY` when a linearizable read reaches a node that is no longer primary, and `COORDINATOR_UNAVAILABLE` when the node cannot reach the coordinator to check at all.

`GET /db/:id/cluster` lists the levels each node currently serves, which is what lets the [topology-aware client](/docs/topology-routing) send a `majority` read to a node that can answer it and a `linearizable` read to the primary.

## Conflict resolution

One primary per replication group serialises the ordinary writes. When a receiver applies a batch and finds the target row already present, it passes the local and incoming versions to the configured resolver:

| Strategy | Class | Behaviour |
| --- | --- | --- |
| Last-Writer-Wins | `LWWResolver` | The resolver selects the change with the higher HLC timestamp, breaking ties by node ID. |
| Field-Level Merge | `FieldMergeResolver` | The resolver merges non-overlapping columns and uses per-column HLC metadata for overlapping ones. |
| Primary Wins | `PrimaryWinsResolver` | The resolver selects the version authored by a configured primary node ID. |

Custom resolvers define a single `resolve(ctx)` method, and you can set a resolver per table.

## Coordinator-backed failover

Coordinator mode stores primary authority, node sessions, replication-group state, and the in-sync set in a cluster coordinator, which keeps the record of authority available when a primary is lost. The package includes an etcd adapter. Build the coordinator from the etcd hosts and node credentials, then reference it from a `coordinator` config block that names the cluster, the replication group, and the voting nodes.

The three files under `credentials` come from your etcd cluster and have their own certificate authority. `etcd-ca.crt` is the authority etcd presents, and `orders-node.crt` with `orders-node.key` is the client certificate etcd accepts for this node. Ask whoever runs your etcd cluster for all three, because etcd rejects a certificate signed by the replication authority you created earlier.

<CodeGroup defaultTab="primary.ts">

```ts title="primary.ts"

// [!code ++:2]

const dbPath = './data/orders.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('orders', dbPath)
const writerConn = await driver.open(dbPath)
const tracker = new ChangeTracker()

const transport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/primary.crt',
  tlsKey: './certs/primary.key',
  tlsCaCert: './certs/ca.crt',
})

// [!code ++:10]
const coordinator = createEtcdCoordinator({
  hosts: ['https://etcd-1.internal:2379', 'https://etcd-2.internal:2379'],
  keyPrefix: '/sirannon/orders',
  credentials: {
    rootCertificate: readFileSync('./certs/etcd-ca.crt'),
    privateKey: readFileSync('./certs/orders-node.key'),
    certChain: readFileSync('./certs/orders-node.crt'),
  },
})

const engine = new ReplicationEngine(db, writerConn, {
  nodeId: 'primary-us-east-1',
  topology: new PrimaryReplicaTopology('primary'),
  transport,
  // [!code ++:8]
  coordinator: {
    clusterId: 'commerce-production',
    groupId: 'orders',
    endpoint: 'https://primary-us-east-1.internal/db/orders',
    coordinator,
    votingDataBearingNodeIds: ['primary-us-east-1', 'replica-eu-west-1', 'replica-ap-south-1'],
    controller: true,
  },
  snapshotConnectionFactory: () => driver.open(dbPath, { readonly: true }),
  changeTracker: tracker,
})

await engine.start()

await engine.execute('CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, holder TEXT NOT NULL, balance INTEGER NOT NULL DEFAULT 0)')
await engine.execute('CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, total INTEGER NOT NULL)')

await engine.executeBatch('INSERT INTO accounts (id, holder, balance) VALUES (?, ?, ?)', [
  [1, 'Emma Wright', 5000],
  [2, 'Kenji Tanaka', 2000],
])
// [!code --:10]

await engine.execute(
  'INSERT INTO orders (id, total) VALUES (?, ?)',
  [1, 4999],
  { writeConcern: { level: 'majority', timeoutMs: 5000 } },
)

const rows = await engine.query('SELECT id, total FROM orders WHERE id = ?', [1], {
  readConcern: { level: 'linearizable' },
})
```

```ts title="replica.ts"

const replicaPath = './data/orders-replica.db'
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver })
const replicaDb = await sirannon.open('orders', replicaPath)
const replicaConn = await driver.open(replicaPath)
const replicaTracker = new ChangeTracker()

const replicaTransport = new GrpcReplicationTransport({
  host: '0.0.0.0',
  port: 4200,
  tlsCert: './certs/replica.crt',
  tlsKey: './certs/replica.key',
  tlsCaCert: './certs/ca.crt',
})

const replicaEngine = new ReplicationEngine(replicaDb, replicaConn, {
  nodeId: 'replica-eu-west-1',
  topology: new PrimaryReplicaTopology('replica'),
  transport: replicaTransport,
  transportConfig: { endpoints: ['primary.example.com:4200'] },
  writeForwarding: true,
  changeTracker: replicaTracker,
})

await replicaEngine.start()
```

</CodeGroup>

A primary may accept writes only while it can prove current authority through terms and leases. Replicas reject stale batches and stale forwarded writes, and only an in-sync replica can be promoted. Automatic write failover requires at least three voting data-bearing nodes. With fewer, a survivor cannot prove majority authority. Writes therefore stay unavailable, and no <Tooltip tip="A split brain happens when two nodes each believe they are the primary and both accept writes, leaving the copies with conflicting histories.">split brain</Tooltip> can form.

<Note>
  Sirannon quarantines a returning former primary that holds local-only writes for manual review, and it never merges those
  writes automatically. An operator rebuilds or restores the node before it rejoins.
</Note>

## Schema changes

Replicated <Tooltip tip="DDL (data definition language) covers the statements that change a database's schema, such as CREATE TABLE and DROP INDEX.">DDL</Tooltip> passes a safety allowlist: `CREATE TABLE`, `ALTER TABLE ... ADD COLUMN`, `DROP TABLE`, `CREATE INDEX`, and `DROP INDEX`. The receiver rejects multiple statements, `AS SELECT`, extension loading, `ATTACH`, and dangerous file functions.

## Report the cluster to clients and to your operators

A [topology-aware client](/docs/topology-routing) reads `GET /db/:id/cluster` to find the primary and the nodes it may read from, while `GET /health/ready` reports the same node's replication figures. Both answers come from hooks you supply, and three helpers turn the engine's own status into exactly the shapes those hooks return.

```ts

const endpoints = {
  'primary-us-east-1': 'https://primary-us-east-1.internal/db/orders',
  'replica-eu-west-1': 'https://replica-eu-west-1.internal/db/orders',
  'replica-ap-south-1': 'https://replica-ap-south-1.internal/db/orders',
}

const server = createServer(sirannon, {
  authenticate: identifyCaller,
  authorizeClusterStatus: ({ headers }) => headers.authorization === `Bearer ${process.env.SIRANNON_OPERATOR_TOKEN}`,
  getReplicationStatus: () => toReplicationStatusInfo(engine.status()),
  getClusterStatus: (databaseId) => toClusterStatusInfo(engine.status(), { databaseId, endpoints }),
})
```

`toReplicationStatusInfo` reads the role, the peer count, the local sequence, the health, and, in coordinator mode, the group, the term, the current primary, the in-sync replicas, and the ones lagging. `toClusterStatusInfo` reads the routing metadata a client needs, and it calls `toClusterReadEndpoints` for you.

`readEndpoints` is the part a client acts on. It holds one entry per node that counts towards majority and is serving, leaving out any node the group has quarantined, is draining, or is repairing. A node the group counts as in sync serves both `local` and `majority`, while a node that falls behind serves `local` alone, because the engine answers a `local` read without an in-sync check. A node with no coordinator omits the field, since no coordinator names a group for it to report. Call `toClusterReadEndpoints(coordinatorStatus, endpoints)` yourself where you want that list for something other than the route.

The `endpoints` map is yours to keep current, because Sirannon records node identifiers and never the addresses a client reaches them on. A node missing from the map appears with an empty endpoint. Build the map from the same configuration that names `votingDataBearingNodeIds`.

The [configuration page](/docs/configuration#replication) covers batch sizes, sync timeouts, flow control, and the coordinator options, as does the package's own [configuration reference](https://github.com/assetcorp/sirannon-db/blob/main/docs/configuration.md). The [distributed entitlements example](https://github.com/assetcorp/sirannon-db/tree/main/packages/ts/examples/distributed-entitlements) runs a three-node coordinator-backed cluster with etcd, mutual TLS, and failure injection through Toxiproxy.

A group that already uses coordinator-backed failover can also give one of its nodes the [backups](/docs/backups-in-a-cluster), reading the same membership from the same coordinator.