Scaling & Sharding
As your data grows, CleaveDB seamlessly scales horizontally across multiple instances using its built-in distributed Go coordinator.
The Go Coordinator
While the core database engine is written in Rust to squeeze every drop of performance from single-node hardware, the cluster orchestration is handled by a robust Go coordinator. This separation of concerns allows the Rust engine to focus entirely on parsing, planning, and executing queries with memory-safe speed, while the Go coordinator focuses on:
- Cluster Topology Management: Discovering nodes, tracking health, and maintaining cluster state.
- Query Routing: Directing queries to the appropriate shards that contain the necessary tenant data.
- Fault Tolerance: Automatically promoting replicas and rerouting traffic if a primary node fails.
Tenant-Aware Sharding
Unlike traditional databases that shard purely by random hashes, CleaveDB leverages Tenant Isolation for highly optimized data locality. Because all documents and graph bonds are scoped by tenant (e.g. products:david.laptop), the coordinator can guarantee that all data belonging to a single tenant lives on the same shard.
Why Tenant-Aware Sharding?
By keeping a tenant's entire document collection and graph relationships on a single physical node, multi-hop graph traversals (using FOLLOW) execute locally in memory. This eliminates the need for expensive cross-node network calls during complex graph queries.
Deploying a Cluster
To start a multi-node cluster, you first initialize a coordinator node, and then attach worker nodes pointing to the coordinator's address.
# Start the main cluster coordinator
cleavedb --role coordinator --bind 0.0.0.0:8300# Start worker nodes and point them to the coordinator
cleavedb --role worker --coordinator 10.0.0.50:8300
cleavedb --role worker --coordinator 10.0.0.50:8300