RunReveal Architecture Deep Dive
Updated: 20 minutes ago
Why ClickHouse
The first thing I wanted to understand about RunReveal wasn't the UI — it was what's actually underneath it.
Most SIEMs I've worked with are built on some flavor of a log indexer:
Splunk has its buckets and indexers,
Sentinel sits on top of Azure Log Analytics workspaces.
Those architectures were designed in an era where the assumption was you'd mostly search for specific terms across a lot of unstructured text.
RunReveal is built directly on ClickHouse, a columnar OLAP database.
The practical difference:
Columnar storage is built for aggregation and analytical queries — count these, group by that, join across time windows — which happens to be exactly the shape of most detection logic and threat hunting. It's the same reason a lot of observability companies quietly migrated off Elasticsearch onto ClickHouse over the last few years. RunReveal just built the SIEM on that foundation from day one instead of bolting it on later.
The logs Table — One Table, Everything
Here's the part that actually surprised me. Almost everything in RunReveal reads from a single underlying table, called simply logs. I ran a DESCRIBE TABLE against it directly and counted 38 columns. The ones that matter most day-to-day:
workspaceID, sourceID, sourceType — which source produced the event
receivedAt — the field you filter on for time ranges; eventTime is separate and represents when the activity actually happened
eventName, eventID — what happened
srcIP / dstIP, plus full GeoIP and ASN fields for both — no separate enrichment lookup needed for basic geo/ASN context
actor (a Map type) — who did it, however the source represents that
tags (Map) and resources (Array) — flexible structured fields for anything source-specific
enrichments (Array of Tuple) — results of any enrichment steps a pipeline applied
rawLog — the original, unparsed event, always kept
On top of that single table, RunReveal maintains source-specific views — aws_cloudtrail_logs, okta_logs, github_logs, aws_vpc_flow_logs, and more — which present the same underlying rows with normalized, source-appropriate column names.
Same DESCRIBE TABLE trick works on those too, if you want to see exactly what a given connector normalizes for you.
Topics, Pipelines, and the Step That Actually Writes Data
This is the part of the architecture that took me the longest to actually understand, because the docs describe it more abstractly than what I found once I inspected a real, live pipeline through the API.
A Pipeline is an ordered list of steps that runs against a Topic (a stream of incoming events).
I pulled a real pipeline from a live workspace and it had five steps, in order:
transform — reshape the event
enrich — attach additional context
filter — drop events matching a condition
detect — a lightweight in-pipeline detection check
destination — the step that decides where the event actually ends up
The insight worth sitting with: a pipeline's Destination step is what actually writes to storage. There's no implicit side-channel where data magically ends up in ClickHouse just because it entered the system. If a pipeline doesn't have a Destination step, older pipelines fall back to RunReveal's default backend, but the docs explicitly warn that fallback isn't something you should rely on — add explicit Destination steps so routing is predictable.'
Destinations, Properly Explained
I want to spend real space on this because it's a genuinely well-designed piece of the platform, and the docs page for it answers a lot of questions I had after just looking at one example pipeline.
Two destination types
ClickHouse destinations are for real-time querying and analysis — this is what powers every RunReveal feature. You can configure more than one, with one marked as default. There's also SPIFFE/mTLS support for certificate-based auth instead of username and password, if that matters to your environment.
Object Storage destinations — S3, Cloudflare R2, Google Cloud Storage, Azure Blob — are for archival, backup, and long-term retention. They're not queryable through RunReveal's own features; they're cold storage for compliance and disaster recovery.
How routing actually works
A few rules that aren't obvious until you read the FAQ section of the docs closely:
Creating a destination in the UI does nothing on its own — it just makes that destination available. Events only flow to it once a pipeline has a Destination step selecting it.
A single pipeline can have multiple Destination steps, which means you can fan the same event out to ClickHouse and S3 at once.
Destination steps don't stop pipeline execution — later steps still run. If you have two Destination steps with no precondition separating them, you'll double-write the same event unless that's actually what you want.
One destination failing doesn't block delivery to the others — errors are tracked independently per destination.
Worth remembering: RunReveal Backend only targets ClickHouse storage. If you want data in S3 too, you need a separate explicit Destination step for that object storage destination — nothing routes there automatically.
Destination health checks
RunReveal can monitor write errors per destination over a rolling interval, and alert your notification channels if the error count crosses a threshold for enough consecutive checks in a row — useful for catching a misconfigured or unreachable destination before it silently drops data for hours. These are configured from the destination's own Health Checks page, not from the create/edit form, which tripped me up the first time I went looking for the setting.
Retention, in Practice
Default retention on ingested data is 550 days.
Disabling a source stops new ingestion but keeps existing config and data intact. Actual deletion of historical data is support-ticket only — this is append-only storage, which lines up with why Object Storage destinations exist as a separate concept from ClickHouse destinations: one is for querying recent-to-mid-term data fast, the other is for keeping everything indefinitely at a lower cost.
Next up: Part 3 covers how you actually query all of this yourself — the Explorer, saved queries, and a set of platform tables that let you query RunReveal's own operational data with the same SQL interface.
------------------------------------------------Dean------------------------------------------------
Comments