Distributed

Topology-aware routing

Route reads and writes across the nodes of a replication group from one client, with static or coordinator discovery, read preferences, and a refresh on every error that means the client's view of the cluster is out of date.

Table of Contents

SirannonClient connects to one server URL and stays there. A replication group has a primary that accepts writes and replicas that serve reads, and which node is which changes during a failover. TopologyAwareClient is the client that follows that: it sends every write to the current primary, picks a node for each read, and refreshes its view of the cluster when a node tells it the view is stale.

It has its own entry point.

import { TopologyAwareClient } from '@delali/sirannon-db/client/topology'

Keep this import out of browser bundles. It reaches internal node addresses, reads GET /db/:id/cluster, and carries an operator credential that no browser should hold. SirannonClient fails with INVALID_ARGUMENT when you pass it a routing option such as endpoints or readPreference, and names the entry point you meant, which keeps the two apart when someone reaches for the wrong one.

Route to addresses you configure

Static mode uses the addresses you configured and nothing else. Name the primary and the replicas, and the client routes by preference without asking any node about the group.

static.ts
import { operationRef } from '@delali/sirannon-db'
import { TopologyAwareClient } from '@delali/sirannon-db/client/topology'
 
const ordersByStatus = operationRef<{ status: string }, { id: number; total: number }>('ordersByStatus')
 
const client = new TopologyAwareClient({
  primary: 'https://orders-node-a.internal',
  replicas: ['https://orders-node-b.internal', 'https://orders-node-c.internal'],
  readPreference: 'replica',
})
 
const rows = await client.database('orders').query(ordersByStatus, { status: 'pending' })

This suits a cluster where an operator or an external system promotes a node and updates configuration. It has no failover of its own, so a write keeps going to the address you named as primary until you change it.

Let the cluster say where to go

Coordinator mode asks the cluster itself where each call should go, and treats endpoints as a starter list. The client fetches routing metadata from GET /db/:id/cluster and caches the current primary, the primary term, and the endpoints that are readable right now.

coordinator.ts
import { operationRef } from '@delali/sirannon-db'
import { TopologyAwareClient } from '@delali/sirannon-db/client/topology'
 
const ordersByStatus = operationRef<{ status: string }, { id: number; total: number }>('ordersByStatus')
 
const client = new TopologyAwareClient({
  endpoints: ['https://orders-node-a.internal', 'https://orders-node-b.internal'],
  discovery: 'coordinator',
  readPreference: 'nearest',
  readConcern: 'majority',
  headers: { authorization: `Bearer ${process.env.SIRANNON_TOPOLOGY_TOKEN}` },
})
 
const rows = await client.database('orders').query(ordersByStatus, { status: 'pending' })

readConcern above says how current a read has to be, and distributed replication defines each level. The section after next covers what it does to node selection.

The cluster endpoint answers only a credential the server authorises for it through authorizeClusterStatus, so a client carrying an application credential alone discovers nothing and fails with ROUTING_ERROR. That is the intended split: application clients call registered operations, and only routing clients learn the shape of the cluster.

Where each call goes

Writes always route to the primary. A write with no current primary fails with NO_SAFE_PRIMARY rather than reaching a replica that would refuse it.

Reads route by readPreference:

PreferenceWhere the read goes
primaryThe primary. This is the default.
replicaA replica chosen at random, falling back to the primary when no replica is available.
nearestThe endpoint with the lowest measured round-trip latency.

nearest measures each endpoint and caches the result for a minute, so the probe cost is paid once per minute rather than once per read.

In coordinator mode the read concern narrows that choice before the preference applies. The client keeps only the endpoints the cluster says can serve the level you asked for, then picks from those. A linearizable read skips the list entirely and goes to the current primary, because no other node can prove live authority for the term.

Set the level once on the client and every read carries it. The topology transport applies that client-wide level when it chooses a node, so a per-call readConcern fails with INVALID_ARGUMENT rather than being ignored. The HTTP and WebSocket transports are the other way around: they carry a per-call readConcern to the server, because they never choose between nodes.

When the client's view goes stale

Five error codes mean the client is routing against a cluster that has moved on: STALE_PRIMARY, AUTHORITY_LOST, COORDINATOR_UNAVAILABLE, NO_SAFE_PRIMARY, and CONNECTION_ERROR. In coordinator mode, a call failing with any of them refreshes the routing metadata before it does anything else.

What happens next depends on the call:

  • A write clears its cached transport and raises the error. The client never resends it, because a write whose outcome is unknown is not safe to repeat. Reconcile the state, then decide whether to send it again.
  • A read on a refreshed route runs again on the node the refresh chose.
  • A read that fails on a replica for a connection-level reason takes that replica out of the rotation and retries elsewhere, so one unhealthy replica stops absorbing reads instead of failing them one by one.

Open subscriptions and live queries follow the same refresh, so a failover moves them to the node that now serves them.

Options

OptionDefaultDescription
endpointsnoneStarter addresses for coordinator discovery.
primarynoneThe primary address in static mode.
replicasnoneReplica addresses in static mode.
discovery'static''static' uses the configured addresses, 'coordinator' fetches routing metadata per database.
readPreference'primary''primary', 'replica', or 'nearest'.
readConcernnoneThe level every read carries, applied to node selection. Coordinator mode treats an absent value as 'majority'.

It also takes every client SDK option: transport, headers, webSocketProtocols, autoReconnect, reconnectInterval, and requestTimeout.

The distributed replication guide covers the server side of all this, including how a node reports the levels it can serve.