Networked access

Device sync recovery

Handle the three situations a device can't sync its way out of, checking that the server can sync devices at all, giving a device a full copy of the database from a snapshot, and reconciling the schema through the migration handshake.

Table of Contents

Three situations interrupt the ordinary sync loop. The server can be too old to sync devices at all. The device can hold no copy of the database yet, or it can fall so far behind that resuming is hopeless. And the server's tables can have a shape the device hasn't seen. Each one has its own way back, and this page adds all three to the device.ts that device sync built.

The file carries on from that page, with the lines that only printed the loop working taken out. Every step reprints the whole file, and what's marked is new.

Confirm the server can sync

Before it syncs, a client fetches the server's capabilities and confirms the ones device sync requires. Capability negotiation is that check, and eight of the tokens a server announces are required: sync.push, sync.echo-suppression, sync.ack, sync.resume, sync.snapshot, sync.migrations, sync.schema-gate, and sync.stream-apply. sync.stream-apply covers the rowId, txId, and txEnd fields on each change together with the acknowledgement-paced delivery window, so a device refuses to sync against a server that doesn't announce it.

A ninth token, sync.staged-stream, is optional. A device that sees it asks for packed frames and per-change pacing, and a device that doesn't syncs over one frame per change. The same endpoint carries query.named, query.sql, and the registry digest, which is what a client reads to decide whether it may send SQL or only a registered operation.

If the capabilities endpoint is missing (the server predates device sync) or a required capability is absent, start() aborts with SYNC_UNSUPPORTED and names the gap, so a client never syncs against a server whose WebSocket would ignore the device-sync fields.

start() throws a RemoteError carrying that code, so catch it there and let the app run on its local database alone. A server that is merely unreachable throws CONNECTION_ERROR instead, and the two deserve separate branches because they call for different responses.

device.ts
import { Sirannon } from '@delali/sirannon-db'
import { waSqlite } from '@delali/sirannon-db/driver/wa-sqlite'
import { LWWResolver, RemoteError, SyncController } from '@delali/sirannon-db/client'
 
const driver = waSqlite()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('notes', 'notes.db')
 
await db.execute(
  'CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)'
)
await db.watch('notes')
 
interface NoteRow {
  id: string
  author: string
  body: string
}
 
const renderNotes = async (): Promise<void> => {
  const notes = await db.query<NoteRow>('SELECT id, author, body FROM notes ORDER BY id')
  console.log(`notes on this device (${notes.length}):`)
  for (const note of notes) console.log(`  ${note.author}: ${note.body}`)
}
 
const sync = new SyncController(db, {
  url: 'http://localhost:9876',
  databaseId: 'notes',
  tables: ['notes'],
  resolver: new LWWResolver(),
  onChange: () => {
    void renderNotes()
  },
})
 
const startSync = async (): Promise<boolean> => { 
  try {
    await sync.start()
    return true
  } catch (error) {
    if (error instanceof RemoteError && error.code === 'SYNC_UNSUPPORTED') {
      console.log('this server cannot sync devices:', error.message)
      return false
    }
    if (error instanceof RemoteError && error.code === 'CONNECTION_ERROR') {
      console.log('server unreachable, staying local for now')
      return false
    }
    throw error
  }
}
 
const syncing = await startSync() 
 
if (!syncing) { 
  console.log('running on this device alone for now')
}
 
await renderNotes() 
this server cannot sync devices: The server does not support required device sync capabilities: sync.stream-apply. Upgrade the server.
running on this device alone for now
notes on this device (2):
  Amara Okafor: Draft the release note
  Priya Raghunathan: Book the venue

That run used a server missing sync.stream-apply. The device kept the two notes it had saved earlier and drew them as usual, because every local read and write works whether the controller is running or not.

SYNC_UNSUPPORTED stays until somebody upgrades the server, and the message names every capability that is missing, so log it rather than swallowing it. CONNECTION_ERROR is temporary. The capability check itself tolerates an unreachable server, but opening the pull then fails and leaves the controller stopped, so call start() again when the network returns. A write made while the controller is stopped waits in the change log and pushes on the next successful start.

Give a device a full copy of the database

A device that has fallen too far behind to resume replaces its whole database from a server snapshot. With autoResync on (the default), the controller schedules a download in three cases: on the next start when a load is still pending, when the server signals a resync, and after a failed download. It backs off exponentially between attempts.

None of that covers a device syncing for the first time, so start() doesn't fetch the rows that already exist on the server. A new device subscribes from the current sequence and receives only what happens next, reporting state: 'running' with no error while its tables stay empty. Ask for the first copy yourself with downloadSnapshot(), gated on an empty table so that it runs once per device. Catch it as well, because the call rejects when the copy fails and an unguarded await would take the app's startup down with it.

device.ts
import { Sirannon } from '@delali/sirannon-db'
import { waSqlite } from '@delali/sirannon-db/driver/wa-sqlite'
import { LWWResolver, RemoteError, SyncController } from '@delali/sirannon-db/client'
 
const driver = waSqlite()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('notes', 'notes.db')
 
await db.execute(
  'CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)'
)
await db.watch('notes')
 
interface NoteRow {
  id: string
  author: string
  body: string
}
 
const renderNotes = async (): Promise<void> => {
  const notes = await db.query<NoteRow>('SELECT id, author, body FROM notes ORDER BY id')
  console.log(`notes on this device (${notes.length}):`)
  for (const note of notes) console.log(`  ${note.author}: ${note.body}`)
}
 
const sync = new SyncController(db, {
  url: 'http://localhost:9876',
  databaseId: 'notes',
  tables: ['notes'],
  resolver: new LWWResolver(),
  onChange: () => {
    void renderNotes()
  },
})
 
const startSync = async (): Promise<boolean> => {
  try {
    await sync.start()
    return true
  } catch (error) {
    if (error instanceof RemoteError && error.code === 'SYNC_UNSUPPORTED') {
      console.log('this server cannot sync devices:', error.message)
      return false
    }
    if (error instanceof RemoteError && error.code === 'CONNECTION_ERROR') {
      console.log('server unreachable, staying local for now')
      return false
    }
    throw error
  }
}
 
const syncing = await startSync()
 
if (!syncing) { 
if (syncing) { 
  const notesOnDevice = await db.query<NoteRow>('SELECT id FROM notes')
 
  if (notesOnDevice.length === 0) { 
    try {
      await sync.downloadSnapshot({
        onProgress: (progress) => {
          console.log(`copying ${progress.table}: ${progress.loadedRows}/${progress.totalRows} rows`)
        },
      })
    } catch {
      console.log('the first copy did not arrive, so this device starts empty')
    }
  }
} else {
  console.log('running on this device alone for now')
}
 
await renderNotes()
copying notes: 3/3 rows
notes on this device (3):
  Mei-Ling Chen: Agree the venue shortlist
  Sofia Restrepo: Draft the sponsor email
  Tomas Lindqvist: Confirm the catering

The three notes were already on the server, and the fresh device held none of them until the download ran. downloadSnapshot() takes its progress callback from its own argument, as above, and falls back to the controller's onSnapshotProgress when you leave that argument out, so either place reports the same pages. The controller must already be started, because the download runs against the same connection, and it returns to running when the copy finishes, so live sync carries on with no further action.

A snapshot replaces everything the device already holds. The device marks itself as loading, turns foreign key checks off, stops watching its tables and drops them in reverse dependency order so that a table pointing at another goes first, then rebuilds them from the schema the server sent. Next it copies the rows in one page at a time. Every page arrives with a checksum, and the device compares it before saving the page, failing the load with SNAPSHOT_CHECKSUM_MISMATCH when the two disagree. At the end it writes the server's migration history over its own, sets its position in the change stream to the point the snapshot was taken from, starts watching the tables again, turns foreign key checks back on, and clears the loading mark. Once the load finishes, PRAGMA user_version matches the highest migration version the snapshot carried.

A load interrupted before it finishes leaves the mark in place, and the next start picks it up again. The server refuses to snapshot an in-memory database, answering with SNAPSHOT_UNSUPPORTED.

Hold the app's writes while a copy is loading. During the replacement every read and every write on the local database fails with SNAPSHOT_IN_PROGRESS, and that check runs before the SQL does, so a rejected call leaves the database untouched. The one to look out for is a write that arrives a moment earlier and succeeds, because the copy drops that table and the row with it. Retrying each save is the wrong answer; the next section turns saving off for the duration instead.

Turn saving back on when a copy finishes

A copy your own code starts finishes when downloadSnapshot() resolves, so whatever follows the await runs against a working database. The controller starts copies of its own too, and those hand the application two moments: onResyncRequired fires before one begins, and onSnapshotComplete fires when it ends.

Keep one flag between them and let both callbacks set it. outcome.databaseUsable answers the question behind a Save button, which is whether the local database serves reads and writes again, and it's true after every copy that finished.

A failed copy comes with the reason. outcome.ok is false, outcome.error holds the code and the message, and outcome.retrying is true when the controller has already scheduled another try. A failure before the replacement began leaves the database working, so databaseUsable stays true, while one after it leaves every statement refused until a later copy succeeds. retrying matters most when it is false, because the controller has stopped retrying and asking for the copy again is the application's job.

device.ts
import { Sirannon } from '@delali/sirannon-db'
import { waSqlite } from '@delali/sirannon-db/driver/wa-sqlite'
import { LWWResolver, RemoteError, SyncController } from '@delali/sirannon-db/client'
 
const driver = waSqlite()
const sirannon = new Sirannon({ driver })
const db = await sirannon.open('notes', 'notes.db')
 
await db.execute(
  'CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)'
)
await db.watch('notes')
 
interface NoteRow {
  id: string
  author: string
  body: string
}
 
const renderNotes = async (): Promise<void> => {
  const notes = await db.query<NoteRow>('SELECT id, author, body FROM notes ORDER BY id')
  console.log(`notes on this device (${notes.length}):`)
  for (const note of notes) console.log(`  ${note.author}: ${note.body}`)
}
 
let canSave = true
 
const sync = new SyncController(db, {
  url: 'http://localhost:9876',
  databaseId: 'notes',
  tables: ['notes'],
  resolver: new LWWResolver(),
  onResyncRequired: () => { 
    canSave = false
    console.log('the server asked for a fresh copy, so saving is off')
  },
  onSnapshotComplete: (outcome) => {
    canSave = outcome.databaseUsable
    if (outcome.ok) {
      console.log('the copy finished, so saving is back on')
      return
    }
    console.log(`the copy failed with ${outcome.error.code}, trying again: ${outcome.retrying}`)
  },
  onChange: () => {
    void renderNotes()
  },
})
 
const startSync = async (): Promise<boolean> => {
  try {
    await sync.start()
    return true
  } catch (error) {
    if (error instanceof RemoteError && error.code === 'SYNC_UNSUPPORTED') {
      console.log('this server cannot sync devices:', error.message)
      return false
    }
    if (error instanceof RemoteError && error.code === 'CONNECTION_ERROR') {
      console.log('server unreachable, staying local for now')
      return false
    }
    throw error
  }
}
 
const syncing = await startSync()
 
if (syncing) {
  const notesOnDevice = await db.query<NoteRow>('SELECT id FROM notes')
 
  if (notesOnDevice.length === 0) {
    try {
      await sync.downloadSnapshot({
        onProgress: (progress) => {
          console.log(`copying ${progress.table}: ${progress.loadedRows}/${progress.totalRows} rows`)
        },
      })
    } catch {
      console.log('the first copy did not arrive, so this device starts empty')
    }
  }
} else {
  console.log('running on this device alone for now')
}
 
if (canSave) { 
  await db.execute('INSERT INTO notes (id, author, body) VALUES (?, ?, ?)', [
    'note-amara-1',
    'Amara Okafor',
    'Draft the release note',
  ])
}
 
await renderNotes()
copying notes: 3/3 rows
the copy finished, so saving is back on
notes on this device (4):
  Amara Okafor: Draft the release note
  Mei-Ling Chen: Agree the venue shortlist
  Sofia Restrepo: Draft the sponsor email
  Tomas Lindqvist: Confirm the catering

The run above asked for the copy itself, so only onSnapshotComplete fired. onResyncRequired fires when the server signals a resync or the device falls too far behind to resume, and those are the copies the flag exists for, because no promise in your code marks their beginning or end. A copy you asked for reports a failure twice, once as the rejected call and once through the callback, so handle it in whichever of the two suits the app.

Keep every device on the same schema

The device keeps up with the server's schema by itself. Give it the schema your build contains, and the controller applies anything newer the server has: once while start() runs, and again whenever the server turns away a push or a subscription because the device is behind. Your application writes no migration code for any of that, and the file in your bundle never has to list the newer versions.

What you do supply is the starting point. A device can't begin from an empty database, because it has to create its tables and call watch on them before it syncs, and sync.start() fails outright when they aren't there. Declare the set your build contains, and the handshake covers the rest.

A device's schema version is the number of the highest migration it has run (see migrations). The device sends that number every time it pushes and every time it subscribes, and the server checks it. A server that is ahead refuses the request with MIGRATION_REQUIRED, and the device then fetches the migrations it lacks, checks each one against its checksum, runs them, and sends the request again. When a device is ahead of the server, the server refuses with SCHEMA_AHEAD instead, and only a server deployment clears that.

Three situations can't be repaired that way, and each one forces a full snapshot: the device's migration history has diverged from the server's, the server holds a migration it can't hand over as SQL (one written as a function, or one whose checksum no longer matches), or a baseline has replaced versions the device never ran, which fails with MIGRATION_BASELINE_GAP because the migrations in between no longer exist to run.

Declare one migration set and share it across your server, web, and mobile builds so that every side has the same schema. Put it in a file both builds import.

migrations.ts
import type { Migration } from '@delali/sirannon-db'
 
export const migrations: Migration[] = [
  {
    version: 1,
    name: 'create_notes',
    up: 'CREATE TABLE notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)',
    down: 'DROP TABLE notes',
  },
]

Hand that set to the registry on both sides instead of creating the tables by hand. new Sirannon({ driver, migrations }) applies the set to every database the registry opens, so you can delete the CREATE TABLE call. On the server it does more than that: the server can pass a migration's SQL to a device only when its registry holds that migration, so a server that migrated its own database directly leaves every device behind it with a full snapshot as the only way forward.

server.ts
import { Sirannon } from '@delali/sirannon-db'
import { betterSqlite3 } from '@delali/sirannon-db/driver/better-sqlite3'
import { createServer } from '@delali/sirannon-db/server'
import { migrations } from './migrations'
 
const driver = betterSqlite3()
const sirannon = new Sirannon({ driver }) 
const sirannon = new Sirannon({ driver, migrations }) 
const db = await sirannon.open('notes', 'notes.db')
 
await db.execute( 
  'CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)'
)
await db.watch('notes')
 
const server = createServer(sirannon, { port: 9876, maxUnacknowledgedChanges: 500 })
await server.listen()

The device changes the same way. Its copy of migrations.ts stops at version 1 here, because the server was deployed with version 2 while this build of the app was still going out.

device.ts
import { Sirannon } from '@delali/sirannon-db'
import { waSqlite } from '@delali/sirannon-db/driver/wa-sqlite'
import { LWWResolver, RemoteError, SyncController } from '@delali/sirannon-db/client'
import { migrations } from './migrations'
 
const driver = waSqlite()
const sirannon = new Sirannon({ driver }) 
const sirannon = new Sirannon({ driver, migrations }) 
const db = await sirannon.open('notes', 'notes.db')
 
await db.execute( 
  'CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, author TEXT NOT NULL, body TEXT NOT NULL)'
)
await db.watch('notes')
 
interface NoteRow {
  id: string
  author: string
  body: string
}
 
const renderNotes = async (): Promise<void> => {
  const notes = await db.query<NoteRow>('SELECT id, author, body FROM notes ORDER BY id')
  console.log(`notes on this device (${notes.length}):`)
  for (const note of notes) console.log(`  ${note.author}: ${note.body}`)
}
 
let canSave = true
 
const sync = new SyncController(db, {
  url: 'http://localhost:9876',
  databaseId: 'notes',
  tables: ['notes'],
  resolver: new LWWResolver(),
  onResyncRequired: () => {
    canSave = false
    console.log('the server asked for a fresh copy, so saving is off')
  },
  onSnapshotComplete: (outcome) => {
    canSave = outcome.databaseUsable
    if (outcome.ok) {
      console.log('the copy finished, so saving is back on')
      return
    }
    console.log(`the copy failed with ${outcome.error.code}, trying again: ${outcome.retrying}`)
  },
  onChange: () => {
    void renderNotes()
  },
})
 
const startSync = async (): Promise<boolean> => {
  try {
    await sync.start()
    return true
  } catch (error) {
    if (error instanceof RemoteError && error.code === 'SYNC_UNSUPPORTED') {
      console.log('this server cannot sync devices:', error.message)
      return false
    }
    if (error instanceof RemoteError && error.code === 'CONNECTION_ERROR') {
      console.log('server unreachable, staying local for now')
      return false
    }
    throw error
  }
}
 
const syncing = await startSync()
 
if (syncing) {
  console.log('device schema version:', (await sync.status()).schemaVersion) 
 
  const columns = await db.query<{ name: string }>("SELECT name FROM pragma_table_info('notes')") 
  console.log('device columns:', columns.map((column) => column.name).join(', '))
 
  const notesOnDevice = await db.query<NoteRow>('SELECT id FROM notes')
 
  if (notesOnDevice.length === 0) {
    try {
      await sync.downloadSnapshot({
        onProgress: (progress) => {
          console.log(`copying ${progress.table}: ${progress.loadedRows}/${progress.totalRows} rows`)
        },
      })
    } catch {
      console.log('the first copy did not arrive, so this device starts empty')
    }
  }
} else {
  console.log('running on this device alone for now')
}
 
if (canSave) {
  await db.execute('INSERT INTO notes (id, author, body) VALUES (?, ?, ?)', [
    'note-amara-1',
    'Amara Okafor',
    'Draft the release note',
  ])
}
 
await renderNotes()
device schema version: 2
device columns: id, author, body, pinned
copying notes: 3/3 rows
the copy finished, so saving is back on
notes on this device (4):
  Amara Okafor: Draft the release note
  Mei-Ling Chen: Agree the venue shortlist
  Sofia Restrepo: Draft the sponsor email
  Tomas Lindqvist: Confirm the catering

The device reports version 2 although its own file stops at 1, and it has the pinned column. Both arrived through the handshake while start() ran, before the pull opened, and the app's bundle never changed. The migrations guide covers declaring a set once and applying it to every database that a registry opens.

The device sync page covers the everyday loop these three routes back up, and the Sirannon README covers every feature area.