12 the compute to pay for, and less storage
Telemetry cost is the servers that take it in and the storage that keeps it. Quiver needs less of both, on every server in the fleet.
The advantage holds as you scale
We sized each database for 50,000 records a second, then doubled the load. Usage grew in step with the load for both, and Quiver's lead held.
CPU in use
Memory in use
At 100,000/s, ClickHouse kept up only with async inserts, used in these figures (Addendum G).
Telemetry is written far more than it is read
Telemetry is written all day in huge volumes, never changed once written, and read rarely: AI agents query it in short bursts during an investigation, not dashboards refreshing all day. ClickHouse, like most databases, is built for data that is read often and can be changed, and carries the overhead of both. Quiver is built for cheap ingest and cheap, selective queries instead.
- Many concurrent producers (agents, collectors)
- Small batches, all day, every day
- Never updated or deleted row by row
- One trace, service or time window at a time
- Bursts during an investigation, then quiet
Smallest server for the same load, then double it
- 1
Send the same workload
50,000 records a second for 10 minutes, from 32 senders.
- 2
Shrink the compute
Step down from 1 core / 2 GB to half a core / 512 MB.
- 3
Keep the smallest size that keeps up
A size passes if it keeps up, doesn't crash and loses no records.
- 4
Scale up
Double to 100,000 a second and check the gap holds (Addendum G).
The same conditions for both
- Same AWS server and data
- All data in Garage object storage
- 10-minute averages, not peaks
Full setup in Addendum F.
Quiver needs a server half the size
Each box is one 10-minute test at 50,000 records a second. Green kept up with no lost records.
† Quiver's small-server settings; details in Addendum A.
Each engine at its smallest working unit
Quiver at 0.5 core / 512 MB, ClickHouse at 1 core / 1 GB, both at 50,000 rows/s unless noted.
| Measure | Quiver | ClickHouse | To scale | Quiver advantage |
|---|
* Live data is about 100 B per row for both; the gap is replaced ClickHouse parts awaiting deletion (Addendum D).
What happened at twice the load
Quiver kept up on 0.75 core. ClickHouse fell behind with standard inserts and kept up with async inserts, using 72% of the machine against Quiver's 28% (Addendum G).
What about query speed?
ClickHouse answers some lookups faster. Quiver answers them with about a fifth of the CPU and a seventh of the memory, so each node absorbs more of an agent's burst of queries (Addendum E).
Quiver skips the work ClickHouse does on every insert
ClickHouse writes each insert as a new part and keeps merging parts in the background. Quiver writes one Parquet file per flush for all producers and holds no index in memory. Addendum H has the detail.
One file for many writers
All producers share one file and one commit per flush.
No data conversion on the way in
Columnar from the wire to Parquet, with no format conversion.
Far less data rewritten
Fewer, larger files leave far less background merging to do.
No indexes held in memory
Stateless nodes. Files are pruned at query time instead of indexed in memory.
Use Quiver for the data AI agents work with, and ClickHouse for fast analytical queries
Use Quiver for
- Agent-operated telemetry, queried in bursts
- Append-only logs, events and traces
- Many small producers, no batching tier
- Object storage with stateless compute
- Small footprints: edge, per-tenant, sidecar
- Long retention at low cost
Use ClickHouse for
- Sub-second analytical queries over large datasets
- High query concurrency, joins and materialized views
- Frequent updates and deletes
- Hot, curated, aggregated data
The two work together: Quiver holds the raw telemetry agents investigate, and ClickHouse serves the curated slice that needs the fastest queries.
Detailed test results
AEvery server size we tested
† At 512 MB, Quiver runs with its memory budgets sized to the node (smaller compaction target and row groups). ClickHouse at 0.5 core was not tried below 2 GB: it was already CPU-bound, so less memory cannot help.
All cells at 50,000 rows/s for 10 minutes. CPU and memory are 10-minute averages; peak is the highest one-second sample. Quiver's 512 MB runs used its settings for small servers (Addendum F).
| Engine | Settings | CPU cap | Memory cap | Result | Rate achieved | Engine CPU | Storage CPU | Engine memory | Memory peak | Rejected batches |
|---|
BHow much each database can take in on one core
Same producers with no rate limit, engine capped at 1 core / 2 GB, for 10 minutes. ClickHouse rejected batches with error 252 (too many parts) whenever background merges fell behind.
| Measure | Quiver | ClickHouse | Ratio |
|---|
CCPU and memory over the 10-minute test
Quiver at 0.5 core / 512 MB, ClickHouse at 1 core / 1 GB, both at 50,000 rows/s. Ten-second averages; dashed lines are each engine's cap.
| Measure | Quiver | ClickHouse |
|---|
Quiver acknowledges a write once its group-commit flush is durable in object storage, so a typical acknowledgement waits for the 1-second flush window. Telemetry collectors buffer through this. The slowest acknowledgements (p99) are similar for both engines.
DStorage used
Quiver stores each row in 22–26% less object storage than ClickHouse, in 100× fewer objects.
The compressed data itself is about the same size for both, roughly 100 B per row. ClickHouse's bucket is larger because it also holds replaced data waiting to be deleted, and under continuous ingest it always has some.
Measured from the Garage bucket 2 minutes after each 10-minute load ended. Compare storage per row: in the maximum-rate test Quiver took in four times as many rows as ClickHouse in the same 10 minutes, so its totals are larger, while each row takes less space. Live data is what each engine reports as current; the rest is replaced files awaiting deletion (both engines delete after 60 seconds).
| Measure | Quiver 50k unit 30.0M rows | ClickHouse 50k unit 29.9M rows | Quiver max rate 103.9M rows | ClickHouse max rate 25.6M rows |
|---|
Write amplification is bytes written to Garage's disk during the load and settle phases, divided by the bytes stored at the end. It counts the extra rewriting done by merges and compaction.
EQuery cost and speed
Five selective queries typical of telemetry investigation, run 20 times each after a 2-run warm-up, then 3 times each after a restart with caches dropped (cold). Both engines at 1 core / 2 GB unless noted, over the 30 million rows just ingested.
| Query | What it does |
|---|---|
| Q1 | Count and average duration for one service over the last 5 minutes |
| Q2 | Newest 100 errors for one service |
| Q3 | Find every row for one trace ID |
| Q4 | Per-minute count of 5xx responses for one service over 10 minutes |
| Q5 | Latest 1,000 requests from one host over 2 minutes |
| Cost while querying | Quiver | ClickHouse | Ratio | Quiver at 0.5c / 512 MB |
|---|
| Median latency | Quiver warm | ClickHouse warm | Quiver cold | ClickHouse cold | Quiver at 0.5c / 512 MB, warm |
|---|
ClickHouse's warm latencies were measured while its background merges were still running after the load, which likely explains why its cold run, after a restart, was faster. At 512 MB, Quiver's smaller row groups mean more storage reads per query and Garage ran at its 1-core cap, which raises Q1, Q2 and Q4 latency. CPU per query counts all CPU used during the query window, divided by the number of queries.
FTest setup and versions
- Machine
- AWS m7gd.large, us-east-1: 2 vCPU Graviton3, 8 GB, 118 GB local NVMe, Ubuntu 24.04, Docker
- Quiver
- core0 build
35b4016, single node, standalone (no cache tier). Group commit at 50,000 rows or 1 s; replaced files deleted after 60 s. At 512 MB: 32 MB compaction target, 8 MB row groups, 2 concurrent flushes - ClickHouse
- 26.10.3.7 open-source server image (Apache 2.0), self-managed, MergeTree on an S3 disk backed by Garage, ordered by
(service, ts), low-cardinality string columns, ZSTD(3), bloom-filter index on trace ID. Synchronous inserts. Small-node settings: 4 background merge threads, old parts removed after 60 s. Async-insert runs addasync_insert=1withwait_for_async_insert=1 - Object storage
- Garage v2.5.0, single node, capped at 1 core / 1 GB in every run
- Data
- Synthetic log rows with 11 columns (timestamp, service, host, level, trace ID, path, status, duration, message and attributes), about 386 B per row raw. A 1M-row pool replayed with a fresh trace ID per row
- Load client
- Rust load generator, 32 concurrent producers, 5,000-row Arrow batches: Arrow Flight to Quiver, Arrow stream over HTTP to ClickHouse. Client CPU under 0.1 core
- Measurement
- Container CPU and memory sampled every second from the Docker API; bucket size and object count from the Garage admin API. After each run, row counts and timestamp ranges were checked against what the client saw acknowledged
GResults at twice the load (100,000 records a second)
At 100,000 rows/s on a 4-vCPU machine, Quiver kept up on 0.75 core / 512 MB, using 1.9× the CPU for twice the rows, with memory up by about a third. ClickHouse fell behind on 2 cores / 2 GB with standard inserts, reaching 84% of the rate with 1,828 rejected batches. With async inserts it kept up, using 72% of the machine against Quiver's 28%.
The same test at 100,000 rows/s on a 4-vCPU machine (AWS m7gd.xlarge), with Garage at 2 cores / 2 GB for both engines. Quiver passed every size tried, down to 0.75 core / 512 MB. ClickHouse fell behind at 2 cores / 2 GB with standard inserts, so smaller sizes were not run. With async inserts, ClickHouse batches small writes on the server, and it kept up at 2 cores / 2 GB.
| Measure | Quiver | ClickHouse standard inserts | ClickHouse async inserts |
|---|
Whole-machine figures include everything on the machine: engine, Garage, load client and operating system. With async inserts ClickHouse still acknowledges a write only once it is stored (wait_for_async_insert=1); its 9 rejected batches were memory-limit errors. Going from 50,000 to 100,000 rows/s, Quiver's CPU in use rose from 0.53 to 1.01 cores and its memory from 264 to 355 MiB.
HWorkload and design in detail
Logs, events and traces have a distinct profile. They are written continuously and at high volume by many small producers, and they are never updated. They are read selectively, one trace ID, one service or one incident window at a time. Most telemetry is written once and never queried.
Who reads it is changing. ClickHouse is built to make analytical queries fast, so it spends memory and CPU on every insert and in the background to keep each query quick, however often it runs. In FFWD, telemetry is operated by agents. Agents investigate: they pull a trace, search one service's errors, compare a time window, and follow the lead to the next query. They do it in bursts, when something goes wrong, and then go quiet.
That workload rewards the cheapest possible ingest and cheap, selective queries. Quiver is built for exactly that.
ClickHouse writes every insert as a new data part: a set of column, index and metadata files in object storage. It then merges those parts in the background, again and again, reading the data back and rewriting it into larger parts each time, and it keeps its primary index and caches in memory. With many small producers, parts arrive faster than a small server can merge them, and ClickHouse starts rejecting writes.
Quiver is written in Rust on Apache Arrow. Rows arrive over Arrow Flight already in columnar form. Writes from all producers are group-committed into one compressed Parquet file per flush, so the number of files and commits stays flat however many producers there are. Files are never changed after they are written, and compaction combines them within a fixed memory budget. Queries skip irrelevant files using per-file statistics, a clustering key and bloom filters on trace IDs, read at query time, so nothing is held in memory between queries. Nodes keep no state; the data lives only in object storage.