Tune retention

These settings govern what is on disk and how it is shaped. They do different jobs and are worth separating before changing any of them.

How long data is kept

export FANOUT_RETENTION_DAYS=30

Zero means keep everything. Changing it does not delete anything immediately — expired data goes when maintenance next runs, so disk use falls on the maintenance cycle rather than at the moment you change the setting.

Observability API, MCP, and dashboard queries can span up to this retention period. With unlimited retention, each query remains capped at 30 days. Fanout automatically uses wider timeline buckets for windows longer than 24 hours.

The maintenance cycle

export FANOUT_MAINTENANCE_INTERVAL=1h

This is the expensive pass: retention deletes plus full compaction. Lower it to reclaim space sooner, at the cost of running the heavy work more often.

Group-commit target

The target number of telemetry rows that concurrent small requests may share in one atomic Parquet batch:

export FANOUT_INGEST_BATCH_SIZE=50000

A larger target can improve sustained write throughput and create fewer files under concurrent ingest. Every request is still acknowledged only after its Parquet batch is durably published. An individual request is never split merely to satisfy this target: an oversized request remains one atomic batch. The setting controls admission grouping, not the durability boundary.

What the knobs interact with

Maintenance prepares merges while reads continue, then briefly gates readers while it swaps or removes immutable files. New batches keep encoding and need only the short publication lock. Ingest, queries, and maintenance still share CPU, memory bandwidth, and disk, so running maintenance much more often is not free. Change one setting at a time and watch /metrics.

The full list, with defaults and types, is in the storage settings reference.