Tutorial: Building with CleaveDB

CLUSTER queries

In the interactive cleaveshell CLI and application drivers, prefixing queries with CLUSTER transparently routes requests through the Go Coordinator on port 8305. The coordinator fans out the statement across all cluster shards in parallel and returns a consolidated, deduplicated result.

Inspecting cluster topology

Before issuing distributed read queries, verify coordinator connectivity and active shard membership using CLUSTER STATUS:

CLUSTER STATUS

The coordinator responds with its current operational status, role, and registered TCP shard endpoints:

{
  "status": "ok",
  "role": "coordinator",
  "shards": [
    "127.0.0.1:8300",
    "192.168.1.10:8300",
    "192.168.1.11:8300"
  ],
  "version": "4.0.0"
}

This confirms that all three storage nodes are active and reachable by the coordinator.

Distributed query execution

To scatter a query across all shards simultaneously, prefix the CleaveQL statement with CLUSTER:

CLUSTER SCOOP FROM users WHERE age > 21

CLUSTER FIND products WHERE category = "electronics" ARRANGED BY price

Each shard evaluates the query against its local data partition, and the coordinator assembles the resulting documents into a single unified JSON array.

Under the hood: Scatter-Gather lifecycle

When cleaveshell receives a CLUSTER query:

1. TCP Handshake: The client establishes a socket connection to 127.0.0.1:8305.
2. Payload Envelope: The query is packaged into a structured JSON envelope:
   CleaveQLExample · Dispatched request payload

   {
     "action": "scatter_gather",
     "query": "SCOOP FROM users WHERE age > 21",
     "auth": {
       "username": "admin",
       "password": ""
     }
   }

3. Concurrent Fan-Out: The coordinator spawns concurrent Goroutines (one per shard), each with a 5-second deadline context.
4. Deduplication & Merge: Documents are deduplicated by global identifier (gid), result counts are summed, and items are sorted if an arrangement key was specified.

Fault tolerance and partial results

If an individual shard experiences network failure or a timeout during query execution, the coordinator returns available documents from healthy shards and embeds failure diagnostics:

[
  {
    "status": "error",
    "message": "shard 192.168.1.11:8300 error: dial failed",
    "count": 420,
    "documents": [
      { "gid": "users:101", "name": "Alice", "age": 28 },
      { "gid": "users:204", "name": "Bob", "age": 34 }
    ]
  }
]

This allows analytical systems to continue functioning and degrade gracefully even under partial network partitions.