Getting Started

Tutorial: Building with CleaveDB

Pipeline (PIPE) — Overview

In complex analytical workflows, data transformations are rarely solved in a single isolated step. Rather than running separate queries and stitching results together in your application layer, CleaveQL introduces the PIPE command for multi-stage stream processing directly within the engine.

Pipeline structure

Every pipeline starts with PIPE FROM <bucket>, followed by one or more stages linked by the THEN keyword:

Stage	Canonical syntax	Pipeline role
Filter	THEN WHERE age > 20	Filters stream by boolean conditions
Sort	THEN ARRANGED BY age GOING DOWN	Sorts stream ascending (`UP`) or descending (`DOWN`)
Limit	THEN LIMIT 3	Caps the maximum records emitted
Project	THEN SHOW name, age	Selects and yields specific fields
Group & Aggregate	THEN GROUP BY city TALLY AS n	Partitions stream and computes reductions

Explore the Pipeline topics

Learn how to compose flexible analytical pipelines using CleaveQL:

Chaining stages: connect multiple processing stages using THEN and write-after-pipe chains.
Filter & sort stages: combine WHERE filters, ARRANGED BY ordering, and LIMIT windows.
Group & aggregate stages: partition streams with GROUP BY and compute multi-metric reductions.